2024-04-25T11:08:13.072 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T11:08:13.072 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --owner scheduled_pdonnell@teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/fs/test_o_trunc}} --name pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi --archive /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673128 -- /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673128/orig.config.yaml 2024-04-25T11:08:13.126 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T11:08:13.222 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673128 branch: wip-pdonnell-testing-20240425.015853-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/fs/test_o_trunc}} email: pdonnell@redhat.com first_in_suite: false job_id: '7673128' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240425.015853-debug ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported dry_run: true install: ceph: flavor: default sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240425.015853-debug sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 owner: scheduled_pdonnell@teuthology priority: 85 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1880 sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 sleep_before_teardown: 0 subset: 178/512 suite: fs suite_branch: wip-pdonnell-testing-20240425.015853-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi069.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGiqUzMaYYUAMsM3VFvQqRsWQsAPza1SGMxIK+hcaODlEHquSZ6z2j/zlXU+zBHvm+d+BPVeY4eUSvvH40xobhI= smithi146.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJkYGqhjdEA/bbgTDGkz3qwKHlkij5/2MdMXKdrqe3psxWCpuJeK//m5TAXntJURsSyAn51NTSr/VeiyuXNg5Mo= smithi152.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH9ueY8P6oke5kZrKyEqHHaorUOHlko6LuXu4NwyjIVlJek8f9d2cTypcQmMk9MKwVD0zUEUwvZsH2c+AWFsug8= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - workunit: clients: all: - fs/test_o_trunc.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_10:00:23 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T11:08:13.223 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa; will attempt to use it 2024-04-25T11:08:13.223 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks 2024-04-25T11:08:13.224 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T11:08:13.225 INFO:teuthology.task.internal:Checking packages... 2024-04-25T11:08:13.247 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '994e2f20102a20fa9a27966c14b625cc01884df8' 2024-04-25T11:08:13.247 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T11:08:13.247 INFO:teuthology.packaging:ref: None 2024-04-25T11:08:13.247 INFO:teuthology.packaging:tag: None 2024-04-25T11:08:13.247 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240425.015853-debug 2024-04-25T11:08:13.247 INFO:teuthology.packaging:sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T11:08:13.248 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240425.015853-debug 2024-04-25T11:08:13.466 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3291.g994e2f20 2024-04-25T11:08:13.467 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T11:08:13.474 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T11:08:13.474 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T11:08:13.495 INFO:teuthology.task.internal:Saving configuration 2024-04-25T11:08:13.514 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T11:08:13.521 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T11:08:13.547 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi069.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673128', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 11:02:55.444926', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGiqUzMaYYUAMsM3VFvQqRsWQsAPza1SGMxIK+hcaODlEHquSZ6z2j/zlXU+zBHvm+d+BPVeY4eUSvvH40xobhI='} 2024-04-25T11:08:13.569 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi146.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673128', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 11:02:55.446418', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJkYGqhjdEA/bbgTDGkz3qwKHlkij5/2MdMXKdrqe3psxWCpuJeK//m5TAXntJURsSyAn51NTSr/VeiyuXNg5Mo='} 2024-04-25T11:08:13.592 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi152.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673128', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 11:02:55.447473', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH9ueY8P6oke5kZrKyEqHHaorUOHlko6LuXu4NwyjIVlJek8f9d2cTypcQmMk9MKwVD0zUEUwvZsH2c+AWFsug8='} 2024-04-25T11:08:13.592 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T11:08:13.597 INFO:teuthology.task.internal:roles: ubuntu@smithi069.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-25T11:08:13.597 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-25T11:08:13.597 INFO:teuthology.task.internal:roles: ubuntu@smithi152.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-25T11:08:13.597 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T11:08:13.699 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f2650c0ef70>, signals=[15]) 2024-04-25T11:08:13.700 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T11:08:13.708 INFO:teuthology.task.internal:Opening connections... 2024-04-25T11:08:13.708 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi069.front.sepia.ceph.com 2024-04-25T11:08:13.711 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:08:13.793 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi146.front.sepia.ceph.com 2024-04-25T11:08:13.794 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:08:13.869 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi152.front.sepia.ceph.com 2024-04-25T11:08:13.869 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:08:13.945 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T11:08:13.951 DEBUG:teuthology.orchestra.run.smithi069:> uname -m 2024-04-25T11:08:13.969 INFO:teuthology.orchestra.run.smithi069.stdout:x86_64 2024-04-25T11:08:13.970 DEBUG:teuthology.orchestra.run.smithi069:> cat /etc/os-release 2024-04-25T11:08:14.026 INFO:teuthology.orchestra.run.smithi069.stdout:NAME="CentOS Stream" 2024-04-25T11:08:14.027 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION="9" 2024-04-25T11:08:14.027 INFO:teuthology.orchestra.run.smithi069.stdout:ID="centos" 2024-04-25T11:08:14.027 INFO:teuthology.orchestra.run.smithi069.stdout:ID_LIKE="rhel fedora" 2024-04-25T11:08:14.027 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION_ID="9" 2024-04-25T11:08:14.027 INFO:teuthology.orchestra.run.smithi069.stdout:PLATFORM_ID="platform:el9" 2024-04-25T11:08:14.027 INFO:teuthology.orchestra.run.smithi069.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T11:08:14.027 INFO:teuthology.orchestra.run.smithi069.stdout:ANSI_COLOR="0;31" 2024-04-25T11:08:14.027 INFO:teuthology.orchestra.run.smithi069.stdout:LOGO="fedora-logo-icon" 2024-04-25T11:08:14.027 INFO:teuthology.orchestra.run.smithi069.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T11:08:14.028 INFO:teuthology.orchestra.run.smithi069.stdout:HOME_URL="https://centos.org/" 2024-04-25T11:08:14.028 INFO:teuthology.orchestra.run.smithi069.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T11:08:14.028 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T11:08:14.028 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T11:08:14.028 INFO:teuthology.lock.ops:Updating smithi069.front.sepia.ceph.com on lock server 2024-04-25T11:08:14.048 DEBUG:teuthology.orchestra.run.smithi146:> uname -m 2024-04-25T11:08:14.067 INFO:teuthology.orchestra.run.smithi146.stdout:x86_64 2024-04-25T11:08:14.067 DEBUG:teuthology.orchestra.run.smithi146:> cat /etc/os-release 2024-04-25T11:08:14.126 INFO:teuthology.orchestra.run.smithi146.stdout:NAME="CentOS Stream" 2024-04-25T11:08:14.126 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION="9" 2024-04-25T11:08:14.126 INFO:teuthology.orchestra.run.smithi146.stdout:ID="centos" 2024-04-25T11:08:14.126 INFO:teuthology.orchestra.run.smithi146.stdout:ID_LIKE="rhel fedora" 2024-04-25T11:08:14.127 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION_ID="9" 2024-04-25T11:08:14.127 INFO:teuthology.orchestra.run.smithi146.stdout:PLATFORM_ID="platform:el9" 2024-04-25T11:08:14.127 INFO:teuthology.orchestra.run.smithi146.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T11:08:14.127 INFO:teuthology.orchestra.run.smithi146.stdout:ANSI_COLOR="0;31" 2024-04-25T11:08:14.127 INFO:teuthology.orchestra.run.smithi146.stdout:LOGO="fedora-logo-icon" 2024-04-25T11:08:14.127 INFO:teuthology.orchestra.run.smithi146.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T11:08:14.127 INFO:teuthology.orchestra.run.smithi146.stdout:HOME_URL="https://centos.org/" 2024-04-25T11:08:14.127 INFO:teuthology.orchestra.run.smithi146.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T11:08:14.127 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T11:08:14.127 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T11:08:14.127 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server 2024-04-25T11:08:14.148 DEBUG:teuthology.orchestra.run.smithi152:> uname -m 2024-04-25T11:08:14.166 INFO:teuthology.orchestra.run.smithi152.stdout:x86_64 2024-04-25T11:08:14.166 DEBUG:teuthology.orchestra.run.smithi152:> cat /etc/os-release 2024-04-25T11:08:14.224 INFO:teuthology.orchestra.run.smithi152.stdout:NAME="CentOS Stream" 2024-04-25T11:08:14.224 INFO:teuthology.orchestra.run.smithi152.stdout:VERSION="9" 2024-04-25T11:08:14.224 INFO:teuthology.orchestra.run.smithi152.stdout:ID="centos" 2024-04-25T11:08:14.224 INFO:teuthology.orchestra.run.smithi152.stdout:ID_LIKE="rhel fedora" 2024-04-25T11:08:14.224 INFO:teuthology.orchestra.run.smithi152.stdout:VERSION_ID="9" 2024-04-25T11:08:14.224 INFO:teuthology.orchestra.run.smithi152.stdout:PLATFORM_ID="platform:el9" 2024-04-25T11:08:14.225 INFO:teuthology.orchestra.run.smithi152.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T11:08:14.225 INFO:teuthology.orchestra.run.smithi152.stdout:ANSI_COLOR="0;31" 2024-04-25T11:08:14.225 INFO:teuthology.orchestra.run.smithi152.stdout:LOGO="fedora-logo-icon" 2024-04-25T11:08:14.225 INFO:teuthology.orchestra.run.smithi152.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T11:08:14.225 INFO:teuthology.orchestra.run.smithi152.stdout:HOME_URL="https://centos.org/" 2024-04-25T11:08:14.225 INFO:teuthology.orchestra.run.smithi152.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T11:08:14.225 INFO:teuthology.orchestra.run.smithi152.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T11:08:14.225 INFO:teuthology.orchestra.run.smithi152.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T11:08:14.225 INFO:teuthology.lock.ops:Updating smithi152.front.sepia.ceph.com on lock server 2024-04-25T11:08:14.249 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T11:08:14.259 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T11:08:14.291 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T11:08:14.291 DEBUG:teuthology.orchestra.run.smithi069:> test '!' -e /home/ubuntu/cephtest 2024-04-25T11:08:14.294 DEBUG:teuthology.orchestra.run.smithi146:> test '!' -e /home/ubuntu/cephtest 2024-04-25T11:08:14.296 DEBUG:teuthology.orchestra.run.smithi152:> test '!' -e /home/ubuntu/cephtest 2024-04-25T11:08:14.312 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T11:08:14.318 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T11:08:14.318 DEBUG:teuthology.orchestra.run.smithi069:> test -z $(ls -A /var/lib/ceph) 2024-04-25T11:08:14.350 DEBUG:teuthology.orchestra.run.smithi146:> test -z $(ls -A /var/lib/ceph) 2024-04-25T11:08:14.352 DEBUG:teuthology.orchestra.run.smithi152:> test -z $(ls -A /var/lib/ceph) 2024-04-25T11:08:14.451 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T11:08:14.530 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T11:08:14.545 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-04-25T11:08:14.545 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'mon.c': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-04-25T11:08:14.545 DEBUG:teuthology.orchestra.run.smithi069:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T11:08:14.546 DEBUG:teuthology.orchestra.run.smithi146:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T11:08:14.546 DEBUG:teuthology.orchestra.run.smithi152:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T11:08:14.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:08:14.563 DEBUG:teuthology.orchestra.run.smithi146:> uname -r 2024-04-25T11:08:14.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:08:14.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:08:14.565 DEBUG:teuthology.orchestra.run.smithi152:> uname -r 2024-04-25T11:08:14.565 DEBUG:teuthology.orchestra.run.smithi069:> uname -r 2024-04-25T11:08:14.620 INFO:teuthology.orchestra.run.smithi146.stdout:5.14.0-437.el9.x86_64 2024-04-25T11:08:14.620 INFO:teuthology.task.kernel:Running kernel on smithi146: 5.14.0-437.el9.x86_64 2024-04-25T11:08:14.621 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum install -y kernel 2024-04-25T11:08:14.622 INFO:teuthology.orchestra.run.smithi069.stdout:5.14.0-437.el9.x86_64 2024-04-25T11:08:14.623 INFO:teuthology.task.kernel:Running kernel on smithi069: 5.14.0-437.el9.x86_64 2024-04-25T11:08:14.623 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum install -y kernel 2024-04-25T11:08:14.624 INFO:teuthology.orchestra.run.smithi152.stdout:5.14.0-437.el9.x86_64 2024-04-25T11:08:14.624 INFO:teuthology.task.kernel:Running kernel on smithi152: 5.14.0-437.el9.x86_64 2024-04-25T11:08:14.624 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum install -y kernel 2024-04-25T11:08:16.775 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.0 MB 00:00 2024-04-25T11:08:16.814 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.0 MB 00:00 2024-04-25T11:08:17.893 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - BaseOS 5.0 MB/s | 8.0 MB 00:01 2024-04-25T11:08:18.668 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-04-25T11:08:18.948 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - AppStream 18 MB/s | 19 MB 00:01 2024-04-25T11:08:20.636 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - AppStream 11 MB/s | 19 MB 00:01 2024-04-25T11:08:22.652 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.2 MB 00:00 2024-04-25T11:08:24.837 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - Extras packages 19 kB/s | 16 kB 00:00 2024-04-25T11:08:25.095 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - CRB 7.1 MB/s | 6.2 MB 00:00 2024-04-25T11:08:26.729 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 16 kB 00:00 2024-04-25T11:08:28.865 INFO:teuthology.orchestra.run.smithi069.stdout:Extra Packages for Enterprise Linux 10 MB/s | 21 MB 00:02 2024-04-25T11:08:33.955 INFO:teuthology.orchestra.run.smithi069.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-04-25T11:08:35.827 INFO:teuthology.orchestra.run.smithi069.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T11:08:35.827 INFO:teuthology.orchestra.run.smithi069.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T11:08:35.920 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2024-04-25T11:08:35.921 INFO:teuthology.orchestra.run.smithi069.stdout:Nothing to do. 2024-04-25T11:08:35.921 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2024-04-25T11:08:36.019 DEBUG:teuthology.orchestra.run.smithi069:> echo no | sudo yum reinstall kernel || true 2024-04-25T11:08:36.518 INFO:teuthology.orchestra.run.smithi069.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 11:08:33 AM UTC. 2024-04-25T11:08:36.644 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2024-04-25T11:08:36.650 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-04-25T11:08:36.650 INFO:teuthology.orchestra.run.smithi069.stdout: Package Architecture Version Repository Size 2024-04-25T11:08:36.650 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-04-25T11:08:36.650 INFO:teuthology.orchestra.run.smithi069.stdout:Reinstalling: 2024-04-25T11:08:36.650 INFO:teuthology.orchestra.run.smithi069.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T11:08:36.650 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:08:36.650 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2024-04-25T11:08:36.650 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-04-25T11:08:36.650 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:08:36.651 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 255 k 2024-04-25T11:08:36.651 INFO:teuthology.orchestra.run.smithi069.stdout:Installed size: 0 2024-04-25T11:08:36.651 INFO:teuthology.orchestra.run.smithi069.stderr:Operation aborted. 2024-04-25T11:08:36.707 INFO:teuthology.orchestra.run.smithi069.stdout:Is this ok [y/N]: 2024-04-25T11:08:36.707 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum reinstall -y kernel || true 2024-04-25T11:08:37.204 INFO:teuthology.orchestra.run.smithi069.stdout:Last metadata expiration check: 0:00:04 ago on Thu 25 Apr 2024 11:08:33 AM UTC. 2024-04-25T11:08:37.320 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2024-04-25T11:08:37.321 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-04-25T11:08:37.321 INFO:teuthology.orchestra.run.smithi069.stdout: Package Architecture Version Repository Size 2024-04-25T11:08:37.321 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-04-25T11:08:37.321 INFO:teuthology.orchestra.run.smithi069.stdout:Reinstalling: 2024-04-25T11:08:37.321 INFO:teuthology.orchestra.run.smithi069.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T11:08:37.321 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:08:37.322 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2024-04-25T11:08:37.322 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-04-25T11:08:37.322 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:08:37.322 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 255 k 2024-04-25T11:08:37.322 INFO:teuthology.orchestra.run.smithi069.stdout:Installed size: 0 2024-04-25T11:08:37.322 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading Packages: 2024-04-25T11:08:37.707 INFO:teuthology.orchestra.run.smithi069.stdout:kernel-5.14.0-437.el9.x86_64.rpm 2.6 MB/s | 255 kB 00:00 2024-04-25T11:08:37.707 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2024-04-25T11:08:37.707 INFO:teuthology.orchestra.run.smithi069.stdout:Total 662 kB/s | 255 kB 00:00 2024-04-25T11:08:37.726 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2024-04-25T11:08:37.897 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction check succeeded. 2024-04-25T11:08:37.897 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2024-04-25T11:08:37.902 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded. 2024-04-25T11:08:37.902 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2024-04-25T11:08:38.044 INFO:teuthology.orchestra.run.smithi069.stdout: Preparing : 1/1 2024-04-25T11:08:38.124 INFO:teuthology.orchestra.run.smithi069.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T11:08:38.124 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T11:08:38.763 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T11:08:38.763 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T11:08:39.424 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T11:08:39.424 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:08:39.424 INFO:teuthology.orchestra.run.smithi069.stdout:Reinstalled: 2024-04-25T11:08:39.424 INFO:teuthology.orchestra.run.smithi069.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T11:08:39.424 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:08:39.424 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2024-04-25T11:08:39.592 DEBUG:teuthology.orchestra.run.smithi069:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T11:08:39.639 INFO:teuthology.orchestra.run.smithi069.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T11:08:39.640 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T11:08:39.640 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T11:08:39.640 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-25T11:08:39.640 DEBUG:teuthology.orchestra.run.smithi069:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T11:08:39.672 INFO:teuthology.orchestra.run.smithi069.stdout:ttyS1 2024-04-25T11:08:39.698 DEBUG:teuthology.parallel:result is None 2024-04-25T11:08:41.088 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - CRB 340 kB/s | 6.2 MB 00:18 2024-04-25T11:08:42.672 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - Extras packages 61 kB/s | 16 kB 00:00 2024-04-25T11:08:43.954 INFO:teuthology.orchestra.run.smithi146.stdout:Extra Packages for Enterprise Linux 1.1 MB/s | 21 MB 00:19 2024-04-25T11:08:44.590 INFO:teuthology.orchestra.run.smithi152.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-04-25T11:08:49.010 INFO:teuthology.orchestra.run.smithi146.stdout:lab-extras 36 kB/s | 1.7 kB 00:00 2024-04-25T11:08:49.590 INFO:teuthology.orchestra.run.smithi152.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-04-25T11:08:50.892 INFO:teuthology.orchestra.run.smithi146.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T11:08:50.893 INFO:teuthology.orchestra.run.smithi146.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T11:08:50.978 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-04-25T11:08:50.979 INFO:teuthology.orchestra.run.smithi146.stdout:Nothing to do. 2024-04-25T11:08:50.979 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-04-25T11:08:51.074 DEBUG:teuthology.orchestra.run.smithi146:> echo no | sudo yum reinstall kernel || true 2024-04-25T11:08:51.471 INFO:teuthology.orchestra.run.smithi152.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T11:08:51.471 INFO:teuthology.orchestra.run.smithi152.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T11:08:51.561 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-04-25T11:08:51.562 INFO:teuthology.orchestra.run.smithi152.stdout:Nothing to do. 2024-04-25T11:08:51.562 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-04-25T11:08:51.578 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 11:08:48 AM UTC. 2024-04-25T11:08:51.664 DEBUG:teuthology.orchestra.run.smithi152:> echo no | sudo yum reinstall kernel || true 2024-04-25T11:08:51.699 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-04-25T11:08:51.710 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-04-25T11:08:51.710 INFO:teuthology.orchestra.run.smithi146.stdout: Package Architecture Version Repository Size 2024-04-25T11:08:51.710 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-04-25T11:08:51.710 INFO:teuthology.orchestra.run.smithi146.stdout:Reinstalling: 2024-04-25T11:08:51.710 INFO:teuthology.orchestra.run.smithi146.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T11:08:51.710 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T11:08:51.710 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-04-25T11:08:51.710 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-04-25T11:08:51.710 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T11:08:51.710 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 255 k 2024-04-25T11:08:51.710 INFO:teuthology.orchestra.run.smithi146.stdout:Installed size: 0 2024-04-25T11:08:51.711 INFO:teuthology.orchestra.run.smithi146.stderr:Operation aborted. 2024-04-25T11:08:51.761 INFO:teuthology.orchestra.run.smithi146.stdout:Is this ok [y/N]: 2024-04-25T11:08:51.761 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum reinstall -y kernel || true 2024-04-25T11:08:52.168 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 11:08:49 AM UTC. 2024-04-25T11:08:52.259 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:00:04 ago on Thu 25 Apr 2024 11:08:48 AM UTC. 2024-04-25T11:08:52.288 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-04-25T11:08:52.303 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-04-25T11:08:52.303 INFO:teuthology.orchestra.run.smithi152.stdout: Package Architecture Version Repository Size 2024-04-25T11:08:52.303 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-04-25T11:08:52.303 INFO:teuthology.orchestra.run.smithi152.stdout:Reinstalling: 2024-04-25T11:08:52.303 INFO:teuthology.orchestra.run.smithi152.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T11:08:52.303 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-25T11:08:52.303 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2024-04-25T11:08:52.303 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-04-25T11:08:52.303 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-25T11:08:52.303 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 255 k 2024-04-25T11:08:52.304 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 0 2024-04-25T11:08:52.304 INFO:teuthology.orchestra.run.smithi152.stderr:Operation aborted. 2024-04-25T11:08:52.365 INFO:teuthology.orchestra.run.smithi152.stdout:Is this ok [y/N]: 2024-04-25T11:08:52.365 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum reinstall -y kernel || true 2024-04-25T11:08:52.374 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-04-25T11:08:52.375 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-04-25T11:08:52.376 INFO:teuthology.orchestra.run.smithi146.stdout: Package Architecture Version Repository Size 2024-04-25T11:08:52.376 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-04-25T11:08:52.376 INFO:teuthology.orchestra.run.smithi146.stdout:Reinstalling: 2024-04-25T11:08:52.376 INFO:teuthology.orchestra.run.smithi146.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T11:08:52.376 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T11:08:52.376 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-04-25T11:08:52.376 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-04-25T11:08:52.376 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T11:08:52.376 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 255 k 2024-04-25T11:08:52.376 INFO:teuthology.orchestra.run.smithi146.stdout:Installed size: 0 2024-04-25T11:08:52.376 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2024-04-25T11:08:52.739 INFO:teuthology.orchestra.run.smithi146.stdout:kernel-5.14.0-437.el9.x86_64.rpm 886 kB/s | 255 kB 00:00 2024-04-25T11:08:52.739 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2024-04-25T11:08:52.739 INFO:teuthology.orchestra.run.smithi146.stdout:Total 703 kB/s | 255 kB 00:00 2024-04-25T11:08:52.761 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2024-04-25T11:08:52.860 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 11:08:49 AM UTC. 2024-04-25T11:08:52.965 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2024-04-25T11:08:52.965 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2024-04-25T11:08:52.970 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2024-04-25T11:08:52.970 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2024-04-25T11:08:52.975 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-04-25T11:08:52.976 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-04-25T11:08:52.976 INFO:teuthology.orchestra.run.smithi152.stdout: Package Architecture Version Repository Size 2024-04-25T11:08:52.976 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-04-25T11:08:52.976 INFO:teuthology.orchestra.run.smithi152.stdout:Reinstalling: 2024-04-25T11:08:52.976 INFO:teuthology.orchestra.run.smithi152.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T11:08:52.976 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-25T11:08:52.976 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2024-04-25T11:08:52.977 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-04-25T11:08:52.977 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-25T11:08:52.977 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 255 k 2024-04-25T11:08:52.977 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 0 2024-04-25T11:08:52.977 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2024-04-25T11:08:53.098 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2024-04-25T11:08:53.173 INFO:teuthology.orchestra.run.smithi146.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T11:08:53.173 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T11:08:53.451 INFO:teuthology.orchestra.run.smithi152.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.5 MB/s | 255 kB 00:00 2024-04-25T11:08:53.451 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2024-04-25T11:08:53.451 INFO:teuthology.orchestra.run.smithi152.stdout:Total 537 kB/s | 255 kB 00:00 2024-04-25T11:08:53.470 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2024-04-25T11:08:53.663 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2024-04-25T11:08:53.663 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2024-04-25T11:08:53.668 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2024-04-25T11:08:53.668 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2024-04-25T11:08:53.809 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T11:08:53.809 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T11:08:53.816 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2024-04-25T11:08:53.886 INFO:teuthology.orchestra.run.smithi152.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T11:08:53.886 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T11:08:54.511 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T11:08:54.511 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T11:08:54.511 INFO:teuthology.orchestra.run.smithi146.stdout:Reinstalled: 2024-04-25T11:08:54.511 INFO:teuthology.orchestra.run.smithi146.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T11:08:54.511 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T11:08:54.511 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-04-25T11:08:54.628 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T11:08:54.628 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T11:08:54.676 DEBUG:teuthology.orchestra.run.smithi146:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T11:08:54.723 INFO:teuthology.orchestra.run.smithi146.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T11:08:54.724 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T11:08:54.724 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T11:08:54.724 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-25T11:08:54.724 DEBUG:teuthology.orchestra.run.smithi146:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T11:08:54.758 INFO:teuthology.orchestra.run.smithi146.stdout:ttyS1 2024-04-25T11:08:54.780 DEBUG:teuthology.parallel:result is None 2024-04-25T11:08:55.384 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T11:08:55.384 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-25T11:08:55.384 INFO:teuthology.orchestra.run.smithi152.stdout:Reinstalled: 2024-04-25T11:08:55.384 INFO:teuthology.orchestra.run.smithi152.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T11:08:55.384 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-25T11:08:55.384 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-04-25T11:08:55.564 DEBUG:teuthology.orchestra.run.smithi152:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T11:08:55.611 INFO:teuthology.orchestra.run.smithi152.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T11:08:55.612 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T11:08:55.612 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T11:08:55.612 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-04-25T11:08:55.612 DEBUG:teuthology.orchestra.run.smithi152:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T11:08:55.654 INFO:teuthology.orchestra.run.smithi152.stdout:ttyS1 2024-04-25T11:08:55.673 DEBUG:teuthology.parallel:result is None 2024-04-25T11:08:55.673 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T11:08:55.680 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T11:08:55.680 DEBUG:teuthology.orchestra.run.smithi069:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T11:08:55.682 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T11:08:55.686 DEBUG:teuthology.orchestra.run.smithi152:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T11:08:55.729 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T11:08:55.740 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T11:08:55.746 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T11:08:55.746 DEBUG:teuthology.orchestra.run.smithi069:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T11:08:55.749 DEBUG:teuthology.orchestra.run.smithi146:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T11:08:55.751 DEBUG:teuthology.orchestra.run.smithi152:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T11:08:55.872 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T11:08:55.879 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T11:08:55.879 DEBUG:teuthology.orchestra.run.smithi069:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T11:08:55.881 DEBUG:teuthology.orchestra.run.smithi146:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T11:08:55.883 DEBUG:teuthology.orchestra.run.smithi152:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T11:08:55.912 INFO:teuthology.orchestra.run.smithi146.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T11:08:55.916 INFO:teuthology.orchestra.run.smithi069.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T11:08:55.918 INFO:teuthology.orchestra.run.smithi152.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T11:08:55.923 INFO:teuthology.orchestra.run.smithi146.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T11:08:55.927 INFO:teuthology.orchestra.run.smithi069.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T11:08:55.927 INFO:teuthology.orchestra.run.smithi152.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T11:08:55.929 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T11:08:55.936 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T11:08:55.936 DEBUG:teuthology.orchestra.run.smithi069:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T11:08:55.972 DEBUG:teuthology.orchestra.run.smithi146:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T11:08:55.974 DEBUG:teuthology.orchestra.run.smithi152:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T11:08:55.999 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T11:08:56.009 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T11:08:56.009 DEBUG:teuthology.orchestra.run.smithi069:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T11:08:56.039 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T11:08:56.041 DEBUG:teuthology.orchestra.run.smithi152:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T11:08:56.056 DEBUG:teuthology.orchestra.run.smithi069:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T11:08:56.140 DEBUG:teuthology.orchestra.run.smithi069:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T11:08:56.188 DEBUG:teuthology.orchestra.run.smithi069:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T11:08:56.271 DEBUG:teuthology.orchestra.run.smithi069:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T11:08:56.312 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T11:08:56.313 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T11:08:56.378 DEBUG:teuthology.orchestra.run.smithi146:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T11:08:56.424 DEBUG:teuthology.orchestra.run.smithi146:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T11:08:56.477 DEBUG:teuthology.orchestra.run.smithi146:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T11:08:56.556 DEBUG:teuthology.orchestra.run.smithi146:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T11:08:56.602 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T11:08:56.602 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T11:08:56.669 DEBUG:teuthology.orchestra.run.smithi152:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T11:08:56.712 DEBUG:teuthology.orchestra.run.smithi152:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T11:08:56.760 DEBUG:teuthology.orchestra.run.smithi152:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T11:08:56.843 DEBUG:teuthology.orchestra.run.smithi152:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T11:08:56.882 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-25T11:08:56.882 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T11:08:56.946 DEBUG:teuthology.orchestra.run.smithi069:> sudo service rsyslog restart 2024-04-25T11:08:56.949 DEBUG:teuthology.orchestra.run.smithi146:> sudo service rsyslog restart 2024-04-25T11:08:56.951 DEBUG:teuthology.orchestra.run.smithi152:> sudo service rsyslog restart 2024-04-25T11:08:56.984 INFO:teuthology.orchestra.run.smithi146.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T11:08:56.997 INFO:teuthology.orchestra.run.smithi069.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T11:08:57.020 INFO:teuthology.orchestra.run.smithi152.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T11:08:57.520 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T11:08:57.527 INFO:teuthology.task.internal:Starting timer... 2024-04-25T11:08:57.527 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T11:08:57.537 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T11:08:57.546 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-25T11:08:57.547 DEBUG:teuthology.orchestra.run.smithi069:> sudo service auditd rotate 2024-04-25T11:08:57.608 INFO:teuthology.orchestra.run.smithi069.stdout:Rotating logs: 2024-04-25T11:08:57.610 DEBUG:teuthology.orchestra.run.smithi146:> sudo service auditd rotate 2024-04-25T11:08:57.674 INFO:teuthology.orchestra.run.smithi146.stdout:Rotating logs: 2024-04-25T11:08:57.676 DEBUG:teuthology.orchestra.run.smithi152:> sudo service auditd rotate 2024-04-25T11:08:57.750 INFO:teuthology.orchestra.run.smithi152.stdout:Rotating logs: 2024-04-25T11:08:57.752 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T11:08:57.752 DEBUG:teuthology.orchestra.run.smithi069:> /usr/sbin/getenforce 2024-04-25T11:08:57.774 INFO:teuthology.orchestra.run.smithi069.stdout:Permissive 2024-04-25T11:08:57.775 DEBUG:teuthology.orchestra.run.smithi146:> /usr/sbin/getenforce 2024-04-25T11:08:57.800 INFO:teuthology.orchestra.run.smithi146.stdout:Permissive 2024-04-25T11:08:57.800 DEBUG:teuthology.orchestra.run.smithi152:> /usr/sbin/getenforce 2024-04-25T11:08:57.820 INFO:teuthology.orchestra.run.smithi152.stdout:Permissive 2024-04-25T11:08:57.821 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi069.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi146.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi152.front.sepia.ceph.com': 'permissive'} 2024-04-25T11:08:57.821 DEBUG:teuthology.orchestra.run.smithi069:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T11:08:57.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:08:57.847 DEBUG:teuthology.orchestra.run.smithi146:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T11:08:57.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:08:57.872 DEBUG:teuthology.orchestra.run.smithi152:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T11:08:57.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:08:57.896 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T11:08:57.896 DEBUG:teuthology.orchestra.run.smithi069:> sudo /usr/sbin/setenforce permissive 2024-04-25T11:08:57.920 DEBUG:teuthology.orchestra.run.smithi146:> sudo /usr/sbin/setenforce permissive 2024-04-25T11:08:57.953 DEBUG:teuthology.orchestra.run.smithi152:> sudo /usr/sbin/setenforce permissive 2024-04-25T11:08:57.977 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T11:08:57.986 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T11:08:58.003 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T11:08:58.005 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi069.front.sepia.ceph.com,smithi146.front.sepia.ceph.com,smithi152.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T11:12:37.721 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi069.front.sepia.ceph.com'), Remote(name='ubuntu@smithi146.front.sepia.ceph.com'), Remote(name='ubuntu@smithi152.front.sepia.ceph.com')] 2024-04-25T11:12:37.722 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi069.front.sepia.ceph.com' 2024-04-25T11:12:37.724 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:12:37.800 DEBUG:teuthology.orchestra.run.smithi069:> true 2024-04-25T11:12:37.879 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi069.front.sepia.ceph.com' 2024-04-25T11:12:37.879 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-04-25T11:12:37.880 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:12:37.959 DEBUG:teuthology.orchestra.run.smithi146:> true 2024-04-25T11:12:38.037 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-04-25T11:12:38.037 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com' 2024-04-25T11:12:38.038 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:12:38.115 DEBUG:teuthology.orchestra.run.smithi152:> true 2024-04-25T11:12:38.194 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi152.front.sepia.ceph.com' 2024-04-25T11:12:38.195 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T11:12:38.206 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T11:12:38.207 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T11:12:38.207 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T11:12:38.210 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T11:12:38.210 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T11:12:38.212 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T11:12:38.212 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T11:12:38.239 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T11:12:38.240 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T11:12:38.253 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T11:12:38.253 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T11:12:38.265 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T11:12:38.276 INFO:teuthology.orchestra.run.smithi146.stderr:sudo: ntpd: command not found 2024-04-25T11:12:38.277 INFO:teuthology.orchestra.run.smithi069.stderr:sudo: ntpd: command not found 2024-04-25T11:12:38.278 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T11:12:38.287 INFO:teuthology.orchestra.run.smithi146.stdout:506 Cannot talk to daemon 2024-04-25T11:12:38.288 INFO:teuthology.orchestra.run.smithi069.stdout:506 Cannot talk to daemon 2024-04-25T11:12:38.301 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T11:12:38.301 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T11:12:38.302 INFO:teuthology.orchestra.run.smithi152.stderr:sudo: ntpd: command not found 2024-04-25T11:12:38.313 INFO:teuthology.orchestra.run.smithi152.stdout:506 Cannot talk to daemon 2024-04-25T11:12:38.314 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T11:12:38.315 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T11:12:38.326 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T11:12:38.339 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T11:12:38.367 INFO:teuthology.orchestra.run.smithi069.stderr:bash: line 1: ntpq: command not found 2024-04-25T11:12:38.370 INFO:teuthology.orchestra.run.smithi069.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T11:12:38.370 INFO:teuthology.orchestra.run.smithi069.stdout:=============================================================================== 2024-04-25T11:12:38.370 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:12:38.370 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:12:38.370 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:12:38.370 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:12:38.371 INFO:teuthology.orchestra.run.smithi146.stderr:bash: line 1: ntpq: command not found 2024-04-25T11:12:38.374 INFO:teuthology.orchestra.run.smithi146.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T11:12:38.374 INFO:teuthology.orchestra.run.smithi146.stdout:=============================================================================== 2024-04-25T11:12:38.375 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:12:38.375 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:12:38.375 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:12:38.375 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:12:38.395 INFO:teuthology.orchestra.run.smithi152.stderr:bash: line 1: ntpq: command not found 2024-04-25T11:12:38.399 INFO:teuthology.orchestra.run.smithi152.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T11:12:38.399 INFO:teuthology.orchestra.run.smithi152.stdout:=============================================================================== 2024-04-25T11:12:38.399 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:12:38.399 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:12:38.399 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:12:38.399 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:12:38.400 INFO:teuthology.run_tasks:Running task install... 2024-04-25T11:12:38.408 DEBUG:teuthology.task.install:project ceph 2024-04-25T11:12:38.408 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}} 2024-04-25T11:12:38.408 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'} 2024-04-25T11:12:38.409 INFO:teuthology.task.install:Using flavor: default 2024-04-25T11:12:38.417 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T11:12:38.417 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T11:12:38.417 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'tag': None, 'wait_for_package': False} 2024-04-25T11:12:38.418 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T11:12:38.420 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'tag': None, 'wait_for_package': False} 2024-04-25T11:12:38.420 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T11:12:38.421 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'tag': None, 'wait_for_package': False} 2024-04-25T11:12:38.421 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T11:12:38.573 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/ 2024-04-25T11:12:38.573 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3291.g994e2f20 2024-04-25T11:12:38.579 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/ 2024-04-25T11:12:38.579 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3291.g994e2f20 2024-04-25T11:12:38.680 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/ 2024-04-25T11:12:38.680 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3291.g994e2f20 2024-04-25T11:12:38.685 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T11:12:38.685 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-25T11:12:38.685 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T11:12:38.688 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T11:12:38.688 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T11:12:38.688 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T11:12:38.710 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T11:12:38.710 DEBUG:teuthology.orchestra.run.smithi152:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/994e2f20102a20fa9a27966c14b625cc01884df8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T11:12:38.713 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T11:12:38.713 DEBUG:teuthology.orchestra.run.smithi069:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/994e2f20102a20fa9a27966c14b625cc01884df8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T11:12:38.774 DEBUG:teuthology.orchestra.run.smithi152:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T11:12:38.778 DEBUG:teuthology.orchestra.run.smithi069:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T11:12:38.849 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T11:12:38.849 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T11:12:38.849 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T11:12:38.851 DEBUG:teuthology.orchestra.run.smithi069:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T11:12:38.851 DEBUG:teuthology.orchestra.run.smithi152:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T11:12:38.876 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T11:12:38.876 DEBUG:teuthology.orchestra.run.smithi146:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/994e2f20102a20fa9a27966c14b625cc01884df8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T11:12:38.913 INFO:teuthology.orchestra.run.smithi069.stdout:check_obsoletes = 1 2024-04-25T11:12:38.914 INFO:teuthology.orchestra.run.smithi152.stdout:check_obsoletes = 1 2024-04-25T11:12:38.915 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum clean all 2024-04-25T11:12:38.916 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum clean all 2024-04-25T11:12:38.942 DEBUG:teuthology.orchestra.run.smithi146:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T11:12:39.016 DEBUG:teuthology.orchestra.run.smithi146:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T11:12:39.078 INFO:teuthology.orchestra.run.smithi146.stdout:check_obsoletes = 1 2024-04-25T11:12:39.080 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum clean all 2024-04-25T11:12:39.137 INFO:teuthology.orchestra.run.smithi069.stdout:48 files removed 2024-04-25T11:12:39.140 INFO:teuthology.orchestra.run.smithi152.stdout:48 files removed 2024-04-25T11:12:39.164 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T11:12:39.167 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T11:12:39.304 INFO:teuthology.orchestra.run.smithi146.stdout:48 files removed 2024-04-25T11:12:39.331 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T11:12:39.779 INFO:teuthology.orchestra.run.smithi152.stdout:ceph packages for x86_64 238 kB/s | 84 kB 00:00 2024-04-25T11:12:39.829 INFO:teuthology.orchestra.run.smithi069.stdout:ceph packages for x86_64 206 kB/s | 84 kB 00:00 2024-04-25T11:12:39.912 INFO:teuthology.orchestra.run.smithi146.stdout:ceph packages for x86_64 259 kB/s | 84 kB 00:00 2024-04-25T11:12:40.155 INFO:teuthology.orchestra.run.smithi069.stdout:ceph noarch packages 41 kB/s | 12 kB 00:00 2024-04-25T11:12:40.205 INFO:teuthology.orchestra.run.smithi152.stdout:ceph noarch packages 31 kB/s | 12 kB 00:00 2024-04-25T11:12:40.271 INFO:teuthology.orchestra.run.smithi146.stdout:ceph noarch packages 37 kB/s | 12 kB 00:00 2024-04-25T11:12:40.530 INFO:teuthology.orchestra.run.smithi069.stdout:ceph source packages 5.6 kB/s | 1.9 kB 00:00 2024-04-25T11:12:40.564 INFO:teuthology.orchestra.run.smithi152.stdout:ceph source packages 5.9 kB/s | 1.9 kB 00:00 2024-04-25T11:12:40.638 INFO:teuthology.orchestra.run.smithi146.stdout:ceph source packages 5.7 kB/s | 1.9 kB 00:00 2024-04-25T11:12:41.264 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-25T11:12:41.346 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-04-25T11:12:41.931 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - BaseOS 6.0 MB/s | 8.0 MB 00:01 2024-04-25T11:12:43.565 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - AppStream 15 MB/s | 19 MB 00:01 2024-04-25T11:12:43.664 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - AppStream 15 MB/s | 19 MB 00:01 2024-04-25T11:12:46.007 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - AppStream 6.3 MB/s | 19 MB 00:03 2024-04-25T11:12:47.699 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-25T11:12:47.731 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-04-25T11:12:49.191 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - Extras packages 47 kB/s | 16 kB 00:00 2024-04-25T11:12:49.574 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - Extras packages 24 kB/s | 16 kB 00:00 2024-04-25T11:12:50.443 INFO:teuthology.orchestra.run.smithi069.stdout:Extra Packages for Enterprise Linux 19 MB/s | 21 MB 00:01 2024-04-25T11:12:50.974 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - CRB 4.5 MB/s | 6.2 MB 00:01 2024-04-25T11:12:51.644 INFO:teuthology.orchestra.run.smithi146.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-25T11:12:52.466 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - Extras packages 48 kB/s | 16 kB 00:00 2024-04-25T11:12:54.037 INFO:teuthology.orchestra.run.smithi152.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-04-25T11:12:55.509 INFO:teuthology.orchestra.run.smithi069.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-04-25T11:12:56.625 INFO:teuthology.orchestra.run.smithi146.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-04-25T11:12:57.744 INFO:teuthology.orchestra.run.smithi069.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T11:12:57.745 INFO:teuthology.orchestra.run.smithi069.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T11:12:57.755 INFO:teuthology.orchestra.run.smithi069.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T11:12:57.756 INFO:teuthology.orchestra.run.smithi069.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T11:12:57.757 INFO:teuthology.orchestra.run.smithi069.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T11:12:57.758 INFO:teuthology.orchestra.run.smithi069.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T11:12:57.760 INFO:teuthology.orchestra.run.smithi069.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T11:12:57.762 INFO:teuthology.orchestra.run.smithi069.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T11:12:57.762 INFO:teuthology.orchestra.run.smithi069.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T11:12:57.824 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2024-04-25T11:12:57.833 INFO:teuthology.orchestra.run.smithi069.stdout:======================================================================================= 2024-04-25T11:12:57.833 INFO:teuthology.orchestra.run.smithi069.stdout: Package Arch Version Repository Size 2024-04-25T11:12:57.833 INFO:teuthology.orchestra.run.smithi069.stdout:======================================================================================= 2024-04-25T11:12:57.833 INFO:teuthology.orchestra.run.smithi069.stdout:Installing: 2024-04-25T11:12:57.833 INFO:teuthology.orchestra.run.smithi069.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T11:12:57.833 INFO:teuthology.orchestra.run.smithi069.stdout: ceph x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 6.6 k 2024-04-25T11:12:57.833 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-base x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.4 M 2024-04-25T11:12:57.834 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 863 k 2024-04-25T11:12:57.834 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 147 k 2024-04-25T11:12:57.834 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 1.5 M 2024-04-25T11:12:57.834 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 142 k 2024-04-25T11:12:57.834 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 3.5 M 2024-04-25T11:12:57.834 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 7.4 M 2024-04-25T11:12:57.834 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-rook noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 49 k 2024-04-25T11:12:57.834 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-radosgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 10 M 2024-04-25T11:12:57.834 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-test x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 43 M 2024-04-25T11:12:57.834 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-volume noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 273 k 2024-04-25T11:12:57.834 INFO:teuthology.orchestra.run.smithi069.stdout: cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 762 k 2024-04-25T11:12:57.834 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 224 k 2024-04-25T11:12:57.834 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-shell x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 27 k 2024-04-25T11:12:57.835 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-top noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 18 k 2024-04-25T11:12:57.835 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T11:12:57.835 INFO:teuthology.orchestra.run.smithi069.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T11:12:57.835 INFO:teuthology.orchestra.run.smithi069.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T11:12:57.835 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 32 k 2024-04-25T11:12:57.835 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 756 k 2024-04-25T11:12:57.835 INFO:teuthology.orchestra.run.smithi069.stdout: librados-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 127 k 2024-04-25T11:12:57.835 INFO:teuthology.orchestra.run.smithi069.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T11:12:57.835 INFO:teuthology.orchestra.run.smithi069.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T11:12:57.835 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T11:12:57.835 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T11:12:57.835 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T11:12:57.835 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T11:12:57.835 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cephfs x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 161 k 2024-04-25T11:12:57.836 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rados x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 321 k 2024-04-25T11:12:57.836 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 298 k 2024-04-25T11:12:57.836 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 100 k 2024-04-25T11:12:57.836 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 86 k 2024-04-25T11:12:57.836 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.0 M 2024-04-25T11:12:57.836 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-nbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 169 k 2024-04-25T11:12:57.836 INFO:teuthology.orchestra.run.smithi069.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T11:12:57.836 INFO:teuthology.orchestra.run.smithi069.stdout:Upgrading: 2024-04-25T11:12:57.836 INFO:teuthology.orchestra.run.smithi069.stdout: librados2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.3 M 2024-04-25T11:12:57.836 INFO:teuthology.orchestra.run.smithi069.stdout: librbd1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.1 M 2024-04-25T11:12:57.836 INFO:teuthology.orchestra.run.smithi069.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T11:12:57.836 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T11:12:57.836 INFO:teuthology.orchestra.run.smithi069.stdout:Installing dependencies: 2024-04-25T11:12:57.836 INFO:teuthology.orchestra.run.smithi069.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T11:12:57.837 INFO:teuthology.orchestra.run.smithi069.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T11:12:57.837 INFO:teuthology.orchestra.run.smithi069.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T11:12:57.837 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 20 M 2024-04-25T11:12:57.837 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 34 k 2024-04-25T11:12:57.837 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mds x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 2.3 M 2024-04-25T11:12:57.837 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 249 k 2024-04-25T11:12:57.837 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mon x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 4.9 M 2024-04-25T11:12:57.837 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-osd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 18 M 2024-04-25T11:12:57.837 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 16 k 2024-04-25T11:12:57.837 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-selinux x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 25 k 2024-04-25T11:12:57.837 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T11:12:57.837 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T11:12:57.837 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T11:12:57.838 INFO:teuthology.orchestra.run.smithi069.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T11:12:57.838 INFO:teuthology.orchestra.run.smithi069.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T11:12:57.838 INFO:teuthology.orchestra.run.smithi069.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T11:12:57.838 INFO:teuthology.orchestra.run.smithi069.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T11:12:57.838 INFO:teuthology.orchestra.run.smithi069.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T11:12:57.838 INFO:teuthology.orchestra.run.smithi069.stdout: libcephsqlite x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 158 k 2024-04-25T11:12:57.838 INFO:teuthology.orchestra.run.smithi069.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T11:12:57.838 INFO:teuthology.orchestra.run.smithi069.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T11:12:57.838 INFO:teuthology.orchestra.run.smithi069.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T11:12:57.838 INFO:teuthology.orchestra.run.smithi069.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T11:12:57.838 INFO:teuthology.orchestra.run.smithi069.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T11:12:57.838 INFO:teuthology.orchestra.run.smithi069.stdout: libradosstriper1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 497 k 2024-04-25T11:12:57.838 INFO:teuthology.orchestra.run.smithi069.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T11:12:57.839 INFO:teuthology.orchestra.run.smithi069.stdout: librgw2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.2 M 2024-04-25T11:12:57.839 INFO:teuthology.orchestra.run.smithi069.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T11:12:57.839 INFO:teuthology.orchestra.run.smithi069.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T11:12:57.839 INFO:teuthology.orchestra.run.smithi069.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T11:12:57.839 INFO:teuthology.orchestra.run.smithi069.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T11:12:57.839 INFO:teuthology.orchestra.run.smithi069.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T11:12:57.839 INFO:teuthology.orchestra.run.smithi069.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T11:12:57.839 INFO:teuthology.orchestra.run.smithi069.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T11:12:57.839 INFO:teuthology.orchestra.run.smithi069.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T11:12:57.839 INFO:teuthology.orchestra.run.smithi069.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T11:12:57.839 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T11:12:57.839 INFO:teuthology.orchestra.run.smithi069.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T11:12:57.839 INFO:teuthology.orchestra.run.smithi069.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T11:12:57.840 INFO:teuthology.orchestra.run.smithi069.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T11:12:57.840 INFO:teuthology.orchestra.run.smithi069.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T11:12:57.840 INFO:teuthology.orchestra.run.smithi069.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T11:12:57.840 INFO:teuthology.orchestra.run.smithi069.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T11:12:57.840 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T11:12:57.840 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-argparse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 45 k 2024-04-25T11:12:57.840 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 135 k 2024-04-25T11:12:57.840 INFO:teuthology.orchestra.run.smithi069.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T11:12:57.840 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T11:12:57.840 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T11:12:57.840 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T11:12:57.840 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T11:12:57.840 INFO:teuthology.orchestra.run.smithi069.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T11:12:57.840 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T11:12:57.841 INFO:teuthology.orchestra.run.smithi069.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T11:12:57.841 INFO:teuthology.orchestra.run.smithi069.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T11:12:57.841 INFO:teuthology.orchestra.run.smithi069.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T11:12:57.841 INFO:teuthology.orchestra.run.smithi069.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T11:12:57.841 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T11:12:57.841 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T11:12:57.841 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T11:12:57.841 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T11:12:57.841 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T11:12:57.841 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T11:12:57.841 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T11:12:57.841 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T11:12:57.842 INFO:teuthology.orchestra.run.smithi069.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T11:12:57.842 INFO:teuthology.orchestra.run.smithi069.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T11:12:57.842 INFO:teuthology.orchestra.run.smithi069.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T11:12:57.842 INFO:teuthology.orchestra.run.smithi069.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T11:12:57.842 INFO:teuthology.orchestra.run.smithi069.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T11:12:57.842 INFO:teuthology.orchestra.run.smithi069.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T11:12:57.842 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T11:12:57.842 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T11:12:57.842 INFO:teuthology.orchestra.run.smithi069.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T11:12:57.842 INFO:teuthology.orchestra.run.smithi069.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T11:12:57.842 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T11:12:57.843 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T11:12:57.843 INFO:teuthology.orchestra.run.smithi069.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T11:12:57.843 INFO:teuthology.orchestra.run.smithi069.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T11:12:57.843 INFO:teuthology.orchestra.run.smithi069.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T11:12:57.843 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T11:12:57.843 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T11:12:57.843 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T11:12:57.843 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T11:12:57.843 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T11:12:57.843 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T11:12:57.843 INFO:teuthology.orchestra.run.smithi069.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T11:12:57.843 INFO:teuthology.orchestra.run.smithi069.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T11:12:57.843 INFO:teuthology.orchestra.run.smithi069.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T11:12:57.844 INFO:teuthology.orchestra.run.smithi069.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T11:12:57.844 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T11:12:57.844 INFO:teuthology.orchestra.run.smithi069.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T11:12:57.844 INFO:teuthology.orchestra.run.smithi069.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T11:12:57.844 INFO:teuthology.orchestra.run.smithi069.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T11:12:57.844 INFO:teuthology.orchestra.run.smithi069.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T11:12:57.844 INFO:teuthology.orchestra.run.smithi069.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T11:12:57.844 INFO:teuthology.orchestra.run.smithi069.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T11:12:57.844 INFO:teuthology.orchestra.run.smithi069.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T11:12:57.844 INFO:teuthology.orchestra.run.smithi069.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T11:12:57.844 INFO:teuthology.orchestra.run.smithi069.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T11:12:57.844 INFO:teuthology.orchestra.run.smithi069.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T11:12:57.844 INFO:teuthology.orchestra.run.smithi069.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T11:12:57.844 INFO:teuthology.orchestra.run.smithi069.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T11:12:57.845 INFO:teuthology.orchestra.run.smithi069.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T11:12:57.845 INFO:teuthology.orchestra.run.smithi069.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T11:12:57.845 INFO:teuthology.orchestra.run.smithi069.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T11:12:57.845 INFO:teuthology.orchestra.run.smithi069.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T11:12:57.845 INFO:teuthology.orchestra.run.smithi069.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T11:12:57.845 INFO:teuthology.orchestra.run.smithi069.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T11:12:57.845 INFO:teuthology.orchestra.run.smithi069.stdout:Installing weak dependencies: 2024-04-25T11:12:57.845 INFO:teuthology.orchestra.run.smithi069.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T11:12:57.845 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:12:57.845 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2024-04-25T11:12:57.845 INFO:teuthology.orchestra.run.smithi069.stdout:======================================================================================= 2024-04-25T11:12:57.845 INFO:teuthology.orchestra.run.smithi069.stdout:Install 148 Packages 2024-04-25T11:12:57.845 INFO:teuthology.orchestra.run.smithi069.stdout:Upgrade 4 Packages 2024-04-25T11:12:57.846 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:12:57.846 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 214 M 2024-04-25T11:12:57.846 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading Packages: 2024-04-25T11:12:58.784 INFO:teuthology.orchestra.run.smithi146.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T11:12:58.785 INFO:teuthology.orchestra.run.smithi146.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T11:12:58.796 INFO:teuthology.orchestra.run.smithi146.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T11:12:58.797 INFO:teuthology.orchestra.run.smithi146.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T11:12:58.798 INFO:teuthology.orchestra.run.smithi146.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T11:12:58.799 INFO:teuthology.orchestra.run.smithi146.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T11:12:58.800 INFO:teuthology.orchestra.run.smithi146.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T11:12:58.803 INFO:teuthology.orchestra.run.smithi146.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T11:12:58.804 INFO:teuthology.orchestra.run.smithi146.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T11:12:58.810 INFO:teuthology.orchestra.run.smithi069.stdout:(1/152): ceph-19.0.0-3291.g994e2f20.el9.x86_64. 19 kB/s | 6.6 kB 00:00 2024-04-25T11:12:58.865 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-04-25T11:12:58.873 INFO:teuthology.orchestra.run.smithi146.stdout:======================================================================================= 2024-04-25T11:12:58.873 INFO:teuthology.orchestra.run.smithi146.stdout: Package Arch Version Repository Size 2024-04-25T11:12:58.874 INFO:teuthology.orchestra.run.smithi146.stdout:======================================================================================= 2024-04-25T11:12:58.874 INFO:teuthology.orchestra.run.smithi146.stdout:Installing: 2024-04-25T11:12:58.874 INFO:teuthology.orchestra.run.smithi146.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T11:12:58.874 INFO:teuthology.orchestra.run.smithi146.stdout: ceph x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 6.6 k 2024-04-25T11:12:58.874 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-base x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.4 M 2024-04-25T11:12:58.874 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 863 k 2024-04-25T11:12:58.874 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 147 k 2024-04-25T11:12:58.874 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 1.5 M 2024-04-25T11:12:58.874 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 142 k 2024-04-25T11:12:58.874 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 3.5 M 2024-04-25T11:12:58.874 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 7.4 M 2024-04-25T11:12:58.874 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-rook noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 49 k 2024-04-25T11:12:58.875 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-radosgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 10 M 2024-04-25T11:12:58.875 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-test x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 43 M 2024-04-25T11:12:58.875 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-volume noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 273 k 2024-04-25T11:12:58.875 INFO:teuthology.orchestra.run.smithi146.stdout: cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 762 k 2024-04-25T11:12:58.875 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 224 k 2024-04-25T11:12:58.875 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-shell x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 27 k 2024-04-25T11:12:58.875 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-top noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 18 k 2024-04-25T11:12:58.875 INFO:teuthology.orchestra.run.smithi146.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T11:12:58.875 INFO:teuthology.orchestra.run.smithi146.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T11:12:58.875 INFO:teuthology.orchestra.run.smithi146.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T11:12:58.875 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 32 k 2024-04-25T11:12:58.875 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 756 k 2024-04-25T11:12:58.875 INFO:teuthology.orchestra.run.smithi146.stdout: librados-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 127 k 2024-04-25T11:12:58.875 INFO:teuthology.orchestra.run.smithi146.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T11:12:58.876 INFO:teuthology.orchestra.run.smithi146.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T11:12:58.876 INFO:teuthology.orchestra.run.smithi146.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T11:12:58.876 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T11:12:58.876 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T11:12:58.876 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T11:12:58.876 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cephfs x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 161 k 2024-04-25T11:12:58.876 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rados x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 321 k 2024-04-25T11:12:58.876 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 298 k 2024-04-25T11:12:58.876 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 100 k 2024-04-25T11:12:58.876 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 86 k 2024-04-25T11:12:58.876 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.0 M 2024-04-25T11:12:58.876 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-nbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 169 k 2024-04-25T11:12:58.877 INFO:teuthology.orchestra.run.smithi146.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T11:12:58.877 INFO:teuthology.orchestra.run.smithi146.stdout:Upgrading: 2024-04-25T11:12:58.877 INFO:teuthology.orchestra.run.smithi146.stdout: librados2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.3 M 2024-04-25T11:12:58.877 INFO:teuthology.orchestra.run.smithi146.stdout: librbd1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.1 M 2024-04-25T11:12:58.877 INFO:teuthology.orchestra.run.smithi146.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T11:12:58.877 INFO:teuthology.orchestra.run.smithi146.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T11:12:58.877 INFO:teuthology.orchestra.run.smithi146.stdout:Installing dependencies: 2024-04-25T11:12:58.877 INFO:teuthology.orchestra.run.smithi146.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T11:12:58.877 INFO:teuthology.orchestra.run.smithi146.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T11:12:58.877 INFO:teuthology.orchestra.run.smithi146.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T11:12:58.877 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 20 M 2024-04-25T11:12:58.877 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 34 k 2024-04-25T11:12:58.877 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mds x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 2.3 M 2024-04-25T11:12:58.878 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 249 k 2024-04-25T11:12:58.878 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mon x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 4.9 M 2024-04-25T11:12:58.878 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-osd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 18 M 2024-04-25T11:12:58.878 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 16 k 2024-04-25T11:12:58.878 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-selinux x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 25 k 2024-04-25T11:12:58.878 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T11:12:58.878 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T11:12:58.878 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T11:12:58.878 INFO:teuthology.orchestra.run.smithi146.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T11:12:58.878 INFO:teuthology.orchestra.run.smithi146.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T11:12:58.878 INFO:teuthology.orchestra.run.smithi146.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T11:12:58.878 INFO:teuthology.orchestra.run.smithi146.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T11:12:58.878 INFO:teuthology.orchestra.run.smithi146.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T11:12:58.878 INFO:teuthology.orchestra.run.smithi146.stdout: libcephsqlite x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 158 k 2024-04-25T11:12:58.879 INFO:teuthology.orchestra.run.smithi146.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T11:12:58.879 INFO:teuthology.orchestra.run.smithi146.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T11:12:58.879 INFO:teuthology.orchestra.run.smithi146.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T11:12:58.879 INFO:teuthology.orchestra.run.smithi146.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T11:12:58.879 INFO:teuthology.orchestra.run.smithi146.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T11:12:58.879 INFO:teuthology.orchestra.run.smithi146.stdout: libradosstriper1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 497 k 2024-04-25T11:12:58.879 INFO:teuthology.orchestra.run.smithi146.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T11:12:58.879 INFO:teuthology.orchestra.run.smithi146.stdout: librgw2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.2 M 2024-04-25T11:12:58.879 INFO:teuthology.orchestra.run.smithi146.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T11:12:58.879 INFO:teuthology.orchestra.run.smithi146.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T11:12:58.879 INFO:teuthology.orchestra.run.smithi146.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T11:12:58.879 INFO:teuthology.orchestra.run.smithi146.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T11:12:58.879 INFO:teuthology.orchestra.run.smithi146.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T11:12:58.880 INFO:teuthology.orchestra.run.smithi146.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T11:12:58.880 INFO:teuthology.orchestra.run.smithi146.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T11:12:58.880 INFO:teuthology.orchestra.run.smithi146.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T11:12:58.880 INFO:teuthology.orchestra.run.smithi146.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T11:12:58.880 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T11:12:58.880 INFO:teuthology.orchestra.run.smithi146.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T11:12:58.880 INFO:teuthology.orchestra.run.smithi146.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T11:12:58.880 INFO:teuthology.orchestra.run.smithi146.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T11:12:58.880 INFO:teuthology.orchestra.run.smithi146.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T11:12:58.880 INFO:teuthology.orchestra.run.smithi146.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T11:12:58.880 INFO:teuthology.orchestra.run.smithi146.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T11:12:58.880 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T11:12:58.880 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-argparse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 45 k 2024-04-25T11:12:58.880 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 135 k 2024-04-25T11:12:58.881 INFO:teuthology.orchestra.run.smithi146.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T11:12:58.881 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T11:12:58.881 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T11:12:58.881 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T11:12:58.881 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T11:12:58.881 INFO:teuthology.orchestra.run.smithi146.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T11:12:58.881 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T11:12:58.881 INFO:teuthology.orchestra.run.smithi146.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T11:12:58.881 INFO:teuthology.orchestra.run.smithi146.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T11:12:58.881 INFO:teuthology.orchestra.run.smithi146.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T11:12:58.881 INFO:teuthology.orchestra.run.smithi146.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T11:12:58.881 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T11:12:58.881 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T11:12:58.882 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T11:12:58.882 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T11:12:58.882 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T11:12:58.882 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T11:12:58.882 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T11:12:58.882 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T11:12:58.882 INFO:teuthology.orchestra.run.smithi146.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T11:12:58.882 INFO:teuthology.orchestra.run.smithi146.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T11:12:58.882 INFO:teuthology.orchestra.run.smithi146.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T11:12:58.882 INFO:teuthology.orchestra.run.smithi146.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T11:12:58.882 INFO:teuthology.orchestra.run.smithi146.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T11:12:58.882 INFO:teuthology.orchestra.run.smithi146.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T11:12:58.882 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T11:12:58.882 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T11:12:58.883 INFO:teuthology.orchestra.run.smithi146.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T11:12:58.883 INFO:teuthology.orchestra.run.smithi146.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T11:12:58.883 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T11:12:58.883 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T11:12:58.883 INFO:teuthology.orchestra.run.smithi146.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T11:12:58.883 INFO:teuthology.orchestra.run.smithi146.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T11:12:58.883 INFO:teuthology.orchestra.run.smithi146.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T11:12:58.883 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T11:12:58.883 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T11:12:58.883 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T11:12:58.883 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T11:12:58.883 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T11:12:58.883 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T11:12:58.883 INFO:teuthology.orchestra.run.smithi146.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T11:12:58.884 INFO:teuthology.orchestra.run.smithi146.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T11:12:58.884 INFO:teuthology.orchestra.run.smithi146.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T11:12:58.884 INFO:teuthology.orchestra.run.smithi146.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T11:12:58.884 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T11:12:58.884 INFO:teuthology.orchestra.run.smithi146.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T11:12:58.884 INFO:teuthology.orchestra.run.smithi146.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T11:12:58.884 INFO:teuthology.orchestra.run.smithi146.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T11:12:58.884 INFO:teuthology.orchestra.run.smithi146.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T11:12:58.884 INFO:teuthology.orchestra.run.smithi146.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T11:12:58.884 INFO:teuthology.orchestra.run.smithi146.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T11:12:58.884 INFO:teuthology.orchestra.run.smithi146.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T11:12:58.884 INFO:teuthology.orchestra.run.smithi146.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T11:12:58.884 INFO:teuthology.orchestra.run.smithi146.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T11:12:58.884 INFO:teuthology.orchestra.run.smithi146.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T11:12:58.885 INFO:teuthology.orchestra.run.smithi146.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T11:12:58.885 INFO:teuthology.orchestra.run.smithi146.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T11:12:58.885 INFO:teuthology.orchestra.run.smithi146.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T11:12:58.885 INFO:teuthology.orchestra.run.smithi146.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T11:12:58.885 INFO:teuthology.orchestra.run.smithi146.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T11:12:58.885 INFO:teuthology.orchestra.run.smithi146.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T11:12:58.885 INFO:teuthology.orchestra.run.smithi146.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T11:12:58.885 INFO:teuthology.orchestra.run.smithi146.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T11:12:58.885 INFO:teuthology.orchestra.run.smithi146.stdout:Installing weak dependencies: 2024-04-25T11:12:58.885 INFO:teuthology.orchestra.run.smithi146.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T11:12:58.885 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T11:12:58.885 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-04-25T11:12:58.885 INFO:teuthology.orchestra.run.smithi146.stdout:======================================================================================= 2024-04-25T11:12:58.886 INFO:teuthology.orchestra.run.smithi146.stdout:Install 148 Packages 2024-04-25T11:12:58.886 INFO:teuthology.orchestra.run.smithi146.stdout:Upgrade 4 Packages 2024-04-25T11:12:58.886 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T11:12:58.886 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 214 M 2024-04-25T11:12:58.887 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2024-04-25T11:12:59.060 INFO:teuthology.orchestra.run.smithi152.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-25T11:12:59.344 INFO:teuthology.orchestra.run.smithi069.stdout:(2/152): ceph-fuse-19.0.0-3291.g994e2f20.el9.x8 1.6 MB/s | 863 kB 00:00 2024-04-25T11:12:59.428 INFO:teuthology.orchestra.run.smithi069.stdout:(3/152): ceph-immutable-object-cache-19.0.0-329 1.7 MB/s | 147 kB 00:00 2024-04-25T11:12:59.884 INFO:teuthology.orchestra.run.smithi146.stdout:(1/152): ceph-19.0.0-3291.g994e2f20.el9.x86_64. 20 kB/s | 6.6 kB 00:00 2024-04-25T11:12:59.986 INFO:teuthology.orchestra.run.smithi069.stdout:(4/152): ceph-mds-19.0.0-3291.g994e2f20.el9.x86 4.0 MB/s | 2.3 MB 00:00 2024-04-25T11:13:00.320 INFO:teuthology.orchestra.run.smithi069.stdout:(5/152): ceph-mgr-19.0.0-3291.g994e2f20.el9.x86 4.5 MB/s | 1.5 MB 00:00 2024-04-25T11:13:01.186 INFO:teuthology.orchestra.run.smithi152.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T11:13:01.186 INFO:teuthology.orchestra.run.smithi152.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T11:13:01.197 INFO:teuthology.orchestra.run.smithi152.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T11:13:01.197 INFO:teuthology.orchestra.run.smithi152.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T11:13:01.198 INFO:teuthology.orchestra.run.smithi152.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T11:13:01.199 INFO:teuthology.orchestra.run.smithi152.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T11:13:01.201 INFO:teuthology.orchestra.run.smithi152.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T11:13:01.203 INFO:teuthology.orchestra.run.smithi152.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T11:13:01.204 INFO:teuthology.orchestra.run.smithi152.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T11:13:01.264 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-04-25T11:13:01.273 INFO:teuthology.orchestra.run.smithi152.stdout:======================================================================================= 2024-04-25T11:13:01.274 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repository Size 2024-04-25T11:13:01.274 INFO:teuthology.orchestra.run.smithi152.stdout:======================================================================================= 2024-04-25T11:13:01.274 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2024-04-25T11:13:01.274 INFO:teuthology.orchestra.run.smithi152.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T11:13:01.274 INFO:teuthology.orchestra.run.smithi152.stdout: ceph x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 6.6 k 2024-04-25T11:13:01.274 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-base x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.4 M 2024-04-25T11:13:01.274 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 863 k 2024-04-25T11:13:01.274 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 147 k 2024-04-25T11:13:01.274 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 1.5 M 2024-04-25T11:13:01.274 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 142 k 2024-04-25T11:13:01.274 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 3.5 M 2024-04-25T11:13:01.274 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 7.4 M 2024-04-25T11:13:01.275 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-rook noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 49 k 2024-04-25T11:13:01.275 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-radosgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 10 M 2024-04-25T11:13:01.275 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-test x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 43 M 2024-04-25T11:13:01.275 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-volume noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 273 k 2024-04-25T11:13:01.275 INFO:teuthology.orchestra.run.smithi152.stdout: cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 762 k 2024-04-25T11:13:01.275 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 224 k 2024-04-25T11:13:01.275 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-shell x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 27 k 2024-04-25T11:13:01.275 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-top noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 18 k 2024-04-25T11:13:01.275 INFO:teuthology.orchestra.run.smithi152.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T11:13:01.275 INFO:teuthology.orchestra.run.smithi152.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T11:13:01.275 INFO:teuthology.orchestra.run.smithi152.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T11:13:01.275 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 32 k 2024-04-25T11:13:01.275 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 756 k 2024-04-25T11:13:01.276 INFO:teuthology.orchestra.run.smithi152.stdout: librados-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 127 k 2024-04-25T11:13:01.276 INFO:teuthology.orchestra.run.smithi152.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T11:13:01.276 INFO:teuthology.orchestra.run.smithi152.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T11:13:01.276 INFO:teuthology.orchestra.run.smithi152.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T11:13:01.276 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T11:13:01.276 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T11:13:01.276 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T11:13:01.276 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cephfs x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 161 k 2024-04-25T11:13:01.276 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rados x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 321 k 2024-04-25T11:13:01.277 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 298 k 2024-04-25T11:13:01.277 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 100 k 2024-04-25T11:13:01.277 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 86 k 2024-04-25T11:13:01.277 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.0 M 2024-04-25T11:13:01.277 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-nbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 169 k 2024-04-25T11:13:01.277 INFO:teuthology.orchestra.run.smithi152.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T11:13:01.277 INFO:teuthology.orchestra.run.smithi152.stdout:Upgrading: 2024-04-25T11:13:01.277 INFO:teuthology.orchestra.run.smithi152.stdout: librados2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.3 M 2024-04-25T11:13:01.277 INFO:teuthology.orchestra.run.smithi152.stdout: librbd1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.1 M 2024-04-25T11:13:01.277 INFO:teuthology.orchestra.run.smithi152.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T11:13:01.277 INFO:teuthology.orchestra.run.smithi152.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T11:13:01.277 INFO:teuthology.orchestra.run.smithi152.stdout:Installing dependencies: 2024-04-25T11:13:01.277 INFO:teuthology.orchestra.run.smithi152.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T11:13:01.277 INFO:teuthology.orchestra.run.smithi152.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T11:13:01.278 INFO:teuthology.orchestra.run.smithi152.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T11:13:01.278 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 20 M 2024-04-25T11:13:01.278 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 34 k 2024-04-25T11:13:01.278 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mds x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 2.3 M 2024-04-25T11:13:01.278 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 249 k 2024-04-25T11:13:01.278 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mon x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 4.9 M 2024-04-25T11:13:01.278 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-osd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 18 M 2024-04-25T11:13:01.278 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 16 k 2024-04-25T11:13:01.278 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-selinux x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 25 k 2024-04-25T11:13:01.278 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T11:13:01.278 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T11:13:01.278 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T11:13:01.278 INFO:teuthology.orchestra.run.smithi152.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T11:13:01.279 INFO:teuthology.orchestra.run.smithi152.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T11:13:01.279 INFO:teuthology.orchestra.run.smithi152.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T11:13:01.279 INFO:teuthology.orchestra.run.smithi152.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T11:13:01.279 INFO:teuthology.orchestra.run.smithi152.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T11:13:01.279 INFO:teuthology.orchestra.run.smithi152.stdout: libcephsqlite x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 158 k 2024-04-25T11:13:01.279 INFO:teuthology.orchestra.run.smithi152.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T11:13:01.279 INFO:teuthology.orchestra.run.smithi152.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T11:13:01.279 INFO:teuthology.orchestra.run.smithi152.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T11:13:01.279 INFO:teuthology.orchestra.run.smithi152.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T11:13:01.279 INFO:teuthology.orchestra.run.smithi152.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T11:13:01.279 INFO:teuthology.orchestra.run.smithi152.stdout: libradosstriper1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 497 k 2024-04-25T11:13:01.279 INFO:teuthology.orchestra.run.smithi152.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T11:13:01.279 INFO:teuthology.orchestra.run.smithi152.stdout: librgw2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.2 M 2024-04-25T11:13:01.279 INFO:teuthology.orchestra.run.smithi152.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T11:13:01.280 INFO:teuthology.orchestra.run.smithi152.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T11:13:01.280 INFO:teuthology.orchestra.run.smithi152.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T11:13:01.280 INFO:teuthology.orchestra.run.smithi152.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T11:13:01.280 INFO:teuthology.orchestra.run.smithi152.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T11:13:01.280 INFO:teuthology.orchestra.run.smithi152.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T11:13:01.280 INFO:teuthology.orchestra.run.smithi152.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T11:13:01.280 INFO:teuthology.orchestra.run.smithi152.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T11:13:01.280 INFO:teuthology.orchestra.run.smithi152.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T11:13:01.280 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T11:13:01.280 INFO:teuthology.orchestra.run.smithi152.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T11:13:01.280 INFO:teuthology.orchestra.run.smithi152.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T11:13:01.280 INFO:teuthology.orchestra.run.smithi152.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T11:13:01.280 INFO:teuthology.orchestra.run.smithi152.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T11:13:01.280 INFO:teuthology.orchestra.run.smithi152.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T11:13:01.281 INFO:teuthology.orchestra.run.smithi152.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T11:13:01.281 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T11:13:01.281 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-argparse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 45 k 2024-04-25T11:13:01.281 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 135 k 2024-04-25T11:13:01.281 INFO:teuthology.orchestra.run.smithi152.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T11:13:01.281 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T11:13:01.281 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T11:13:01.281 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T11:13:01.281 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T11:13:01.281 INFO:teuthology.orchestra.run.smithi152.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T11:13:01.281 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T11:13:01.281 INFO:teuthology.orchestra.run.smithi152.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T11:13:01.281 INFO:teuthology.orchestra.run.smithi152.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T11:13:01.282 INFO:teuthology.orchestra.run.smithi152.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T11:13:01.282 INFO:teuthology.orchestra.run.smithi152.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T11:13:01.282 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T11:13:01.282 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T11:13:01.282 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T11:13:01.282 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T11:13:01.282 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T11:13:01.282 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T11:13:01.282 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T11:13:01.282 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T11:13:01.282 INFO:teuthology.orchestra.run.smithi152.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T11:13:01.282 INFO:teuthology.orchestra.run.smithi152.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T11:13:01.282 INFO:teuthology.orchestra.run.smithi152.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T11:13:01.282 INFO:teuthology.orchestra.run.smithi152.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T11:13:01.283 INFO:teuthology.orchestra.run.smithi152.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T11:13:01.283 INFO:teuthology.orchestra.run.smithi152.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T11:13:01.283 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T11:13:01.283 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T11:13:01.283 INFO:teuthology.orchestra.run.smithi152.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T11:13:01.283 INFO:teuthology.orchestra.run.smithi152.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T11:13:01.283 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T11:13:01.283 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T11:13:01.283 INFO:teuthology.orchestra.run.smithi152.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T11:13:01.283 INFO:teuthology.orchestra.run.smithi152.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T11:13:01.283 INFO:teuthology.orchestra.run.smithi152.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T11:13:01.283 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T11:13:01.283 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T11:13:01.283 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T11:13:01.284 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T11:13:01.284 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T11:13:01.284 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T11:13:01.284 INFO:teuthology.orchestra.run.smithi152.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T11:13:01.284 INFO:teuthology.orchestra.run.smithi152.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T11:13:01.284 INFO:teuthology.orchestra.run.smithi152.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T11:13:01.284 INFO:teuthology.orchestra.run.smithi152.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T11:13:01.284 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T11:13:01.284 INFO:teuthology.orchestra.run.smithi152.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T11:13:01.284 INFO:teuthology.orchestra.run.smithi152.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T11:13:01.284 INFO:teuthology.orchestra.run.smithi152.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T11:13:01.284 INFO:teuthology.orchestra.run.smithi152.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T11:13:01.284 INFO:teuthology.orchestra.run.smithi152.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T11:13:01.284 INFO:teuthology.orchestra.run.smithi152.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T11:13:01.285 INFO:teuthology.orchestra.run.smithi152.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T11:13:01.285 INFO:teuthology.orchestra.run.smithi152.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T11:13:01.285 INFO:teuthology.orchestra.run.smithi152.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T11:13:01.285 INFO:teuthology.orchestra.run.smithi152.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T11:13:01.285 INFO:teuthology.orchestra.run.smithi152.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T11:13:01.285 INFO:teuthology.orchestra.run.smithi152.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T11:13:01.285 INFO:teuthology.orchestra.run.smithi152.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T11:13:01.285 INFO:teuthology.orchestra.run.smithi152.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T11:13:01.285 INFO:teuthology.orchestra.run.smithi152.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T11:13:01.285 INFO:teuthology.orchestra.run.smithi152.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T11:13:01.285 INFO:teuthology.orchestra.run.smithi152.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T11:13:01.285 INFO:teuthology.orchestra.run.smithi152.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T11:13:01.285 INFO:teuthology.orchestra.run.smithi152.stdout:Installing weak dependencies: 2024-04-25T11:13:01.285 INFO:teuthology.orchestra.run.smithi152.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T11:13:01.285 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-25T11:13:01.286 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2024-04-25T11:13:01.286 INFO:teuthology.orchestra.run.smithi152.stdout:======================================================================================= 2024-04-25T11:13:01.286 INFO:teuthology.orchestra.run.smithi152.stdout:Install 148 Packages 2024-04-25T11:13:01.286 INFO:teuthology.orchestra.run.smithi152.stdout:Upgrade 4 Packages 2024-04-25T11:13:01.286 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-25T11:13:01.286 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 214 M 2024-04-25T11:13:01.287 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2024-04-25T11:13:01.428 INFO:teuthology.orchestra.run.smithi069.stdout:(6/152): ceph-base-19.0.0-3291.g994e2f20.el9.x8 1.8 MB/s | 5.4 MB 00:02 2024-04-25T11:13:01.545 INFO:teuthology.orchestra.run.smithi069.stdout:(7/152): ceph-mon-19.0.0-3291.g994e2f20.el9.x86 4.0 MB/s | 4.9 MB 00:01 2024-04-25T11:13:02.276 INFO:teuthology.orchestra.run.smithi146.stdout:(2/152): ceph-fuse-19.0.0-3291.g994e2f20.el9.x8 361 kB/s | 863 kB 00:02 2024-04-25T11:13:02.535 INFO:teuthology.orchestra.run.smithi146.stdout:(3/152): ceph-common-19.0.0-3291.g994e2f20.el9. 6.6 MB/s | 20 MB 00:02 2024-04-25T11:13:02.629 INFO:teuthology.orchestra.run.smithi069.stdout:(8/152): ceph-common-19.0.0-3291.g994e2f20.el9. 4.7 MB/s | 20 MB 00:04 2024-04-25T11:13:02.643 INFO:teuthology.orchestra.run.smithi146.stdout:(4/152): ceph-immutable-object-cache-19.0.0-329 401 kB/s | 147 kB 00:00 2024-04-25T11:13:02.712 INFO:teuthology.orchestra.run.smithi069.stdout:(9/152): ceph-selinux-19.0.0-3291.g994e2f20.el9 302 kB/s | 25 kB 00:00 2024-04-25T11:13:02.818 INFO:teuthology.orchestra.run.smithi146.stdout:(5/152): ceph-mds-19.0.0-3291.g994e2f20.el9.x86 8.0 MB/s | 2.3 MB 00:00 2024-04-25T11:13:03.494 INFO:teuthology.orchestra.run.smithi146.stdout:(6/152): ceph-mon-19.0.0-3291.g994e2f20.el9.x86 7.2 MB/s | 4.9 MB 00:00 2024-04-25T11:13:03.602 INFO:teuthology.orchestra.run.smithi146.stdout:(7/152): ceph-base-19.0.0-3291.g994e2f20.el9.x8 1.3 MB/s | 5.4 MB 00:04 2024-04-25T11:13:03.720 INFO:teuthology.orchestra.run.smithi069.stdout:(10/152): ceph-radosgw-19.0.0-3291.g994e2f20.el 4.7 MB/s | 10 MB 00:02 2024-04-25T11:13:03.804 INFO:teuthology.orchestra.run.smithi069.stdout:(11/152): cephfs-mirror-19.0.0-3291.g994e2f20.e 2.6 MB/s | 224 kB 00:00 2024-04-25T11:13:03.879 INFO:teuthology.orchestra.run.smithi069.stdout:(12/152): cephfs-shell-19.0.0-3291.g994e2f20.el 360 kB/s | 27 kB 00:00 2024-04-25T11:13:03.955 INFO:teuthology.orchestra.run.smithi069.stdout:(13/152): libcephfs-devel-19.0.0-3291.g994e2f20 422 kB/s | 32 kB 00:00 2024-04-25T11:13:04.188 INFO:teuthology.orchestra.run.smithi069.stdout:(14/152): libcephfs2-19.0.0-3291.g994e2f20.el9. 3.2 MB/s | 756 kB 00:00 2024-04-25T11:13:04.280 INFO:teuthology.orchestra.run.smithi069.stdout:(15/152): libcephsqlite-19.0.0-3291.g994e2f20.e 1.7 MB/s | 158 kB 00:00 2024-04-25T11:13:04.372 INFO:teuthology.orchestra.run.smithi069.stdout:(16/152): librados-devel-19.0.0-3291.g994e2f20. 1.4 MB/s | 127 kB 00:00 2024-04-25T11:13:04.522 INFO:teuthology.orchestra.run.smithi069.stdout:(17/152): libradosstriper1-19.0.0-3291.g994e2f2 3.2 MB/s | 497 kB 00:00 2024-04-25T11:13:05.069 INFO:teuthology.orchestra.run.smithi146.stdout:(8/152): ceph-mgr-19.0.0-3291.g994e2f20.el9.x86 635 kB/s | 1.5 MB 00:02 2024-04-25T11:13:05.177 INFO:teuthology.orchestra.run.smithi146.stdout:(9/152): ceph-selinux-19.0.0-3291.g994e2f20.el9 233 kB/s | 25 kB 00:00 2024-04-25T11:13:05.669 INFO:teuthology.orchestra.run.smithi152.stdout:(1/152): ceph-19.0.0-3291.g994e2f20.el9.x86_64. 19 kB/s | 6.6 kB 00:00 2024-04-25T11:13:05.773 INFO:teuthology.orchestra.run.smithi069.stdout:(18/152): librgw2-19.0.0-3291.g994e2f20.el9.x86 4.1 MB/s | 5.2 MB 00:01 2024-04-25T11:13:05.828 INFO:teuthology.orchestra.run.smithi146.stdout:(10/152): ceph-osd-19.0.0-3291.g994e2f20.el9.x8 7.6 MB/s | 18 MB 00:02 2024-04-25T11:13:05.831 INFO:teuthology.orchestra.run.smithi069.stdout:(19/152): python3-ceph-argparse-19.0.0-3291.g99 778 kB/s | 45 kB 00:00 2024-04-25T11:13:05.886 INFO:teuthology.orchestra.run.smithi146.stdout:(11/152): cephfs-mirror-19.0.0-3291.g994e2f20.e 3.8 MB/s | 224 kB 00:00 2024-04-25T11:13:05.907 INFO:teuthology.orchestra.run.smithi069.stdout:(20/152): python3-ceph-common-19.0.0-3291.g994e 1.8 MB/s | 135 kB 00:00 2024-04-25T11:13:05.962 INFO:teuthology.orchestra.run.smithi146.stdout:(12/152): cephfs-shell-19.0.0-3291.g994e2f20.el 361 kB/s | 27 kB 00:00 2024-04-25T11:13:05.990 INFO:teuthology.orchestra.run.smithi069.stdout:(21/152): python3-cephfs-19.0.0-3291.g994e2f20. 1.9 MB/s | 161 kB 00:00 2024-04-25T11:13:06.045 INFO:teuthology.orchestra.run.smithi146.stdout:(13/152): libcephfs-devel-19.0.0-3291.g994e2f20 381 kB/s | 32 kB 00:00 2024-04-25T11:13:06.053 INFO:teuthology.orchestra.run.smithi152.stdout:(2/152): ceph-fuse-19.0.0-3291.g994e2f20.el9.x8 2.2 MB/s | 863 kB 00:00 2024-04-25T11:13:06.212 INFO:teuthology.orchestra.run.smithi146.stdout:(14/152): libcephfs2-19.0.0-3291.g994e2f20.el9. 4.4 MB/s | 756 kB 00:00 2024-04-25T11:13:06.288 INFO:teuthology.orchestra.run.smithi146.stdout:(15/152): libcephsqlite-19.0.0-3291.g994e2f20.e 2.1 MB/s | 158 kB 00:00 2024-04-25T11:13:06.316 INFO:teuthology.orchestra.run.smithi069.stdout:(22/152): python3-rados-19.0.0-3291.g994e2f20.e 987 kB/s | 321 kB 00:00 2024-04-25T11:13:06.355 INFO:teuthology.orchestra.run.smithi146.stdout:(16/152): librados-devel-19.0.0-3291.g994e2f20. 1.9 MB/s | 127 kB 00:00 2024-04-25T11:13:06.441 INFO:teuthology.orchestra.run.smithi069.stdout:(23/152): python3-rbd-19.0.0-3291.g994e2f20.el9 2.3 MB/s | 298 kB 00:00 2024-04-25T11:13:06.488 INFO:teuthology.orchestra.run.smithi146.stdout:(17/152): libradosstriper1-19.0.0-3291.g994e2f2 3.6 MB/s | 497 kB 00:00 2024-04-25T11:13:06.499 INFO:teuthology.orchestra.run.smithi069.stdout:(24/152): python3-rgw-19.0.0-3291.g994e2f20.el9 1.7 MB/s | 100 kB 00:00 2024-04-25T11:13:06.600 INFO:teuthology.orchestra.run.smithi069.stdout:(25/152): rbd-fuse-19.0.0-3291.g994e2f20.el9.x8 865 kB/s | 86 kB 00:00 2024-04-25T11:13:06.611 INFO:teuthology.orchestra.run.smithi152.stdout:(3/152): ceph-immutable-object-cache-19.0.0-329 263 kB/s | 147 kB 00:00 2024-04-25T11:13:07.272 INFO:teuthology.orchestra.run.smithi146.stdout:(18/152): ceph-radosgw-19.0.0-3291.g994e2f20.el 2.8 MB/s | 10 MB 00:03 2024-04-25T11:13:07.347 INFO:teuthology.orchestra.run.smithi146.stdout:(19/152): librgw2-19.0.0-3291.g994e2f20.el9.x86 6.0 MB/s | 5.2 MB 00:00 2024-04-25T11:13:07.373 INFO:teuthology.orchestra.run.smithi146.stdout:(20/152): python3-ceph-argparse-19.0.0-3291.g99 452 kB/s | 45 kB 00:00 2024-04-25T11:13:07.403 INFO:teuthology.orchestra.run.smithi152.stdout:(4/152): ceph-mds-19.0.0-3291.g994e2f20.el9.x86 2.9 MB/s | 2.3 MB 00:00 2024-04-25T11:13:07.422 INFO:teuthology.orchestra.run.smithi146.stdout:(21/152): python3-ceph-common-19.0.0-3291.g994e 1.8 MB/s | 135 kB 00:00 2024-04-25T11:13:07.456 INFO:teuthology.orchestra.run.smithi146.stdout:(22/152): python3-cephfs-19.0.0-3291.g994e2f20. 1.9 MB/s | 161 kB 00:00 2024-04-25T11:13:07.514 INFO:teuthology.orchestra.run.smithi146.stdout:(23/152): python3-rados-19.0.0-3291.g994e2f20.e 3.4 MB/s | 321 kB 00:00 2024-04-25T11:13:07.590 INFO:teuthology.orchestra.run.smithi146.stdout:(24/152): python3-rgw-19.0.0-3291.g994e2f20.el9 1.3 MB/s | 100 kB 00:00 2024-04-25T11:13:07.632 INFO:teuthology.orchestra.run.smithi146.stdout:(25/152): python3-rbd-19.0.0-3291.g994e2f20.el9 1.7 MB/s | 298 kB 00:00 2024-04-25T11:13:07.657 INFO:teuthology.orchestra.run.smithi146.stdout:(26/152): rbd-fuse-19.0.0-3291.g994e2f20.el9.x8 1.3 MB/s | 86 kB 00:00 2024-04-25T11:13:07.717 INFO:teuthology.orchestra.run.smithi069.stdout:(26/152): rbd-mirror-19.0.0-3291.g994e2f20.el9. 2.7 MB/s | 3.0 MB 00:01 2024-04-25T11:13:07.741 INFO:teuthology.orchestra.run.smithi146.stdout:(27/152): rbd-nbd-19.0.0-3291.g994e2f20.el9.x86 2.0 MB/s | 169 kB 00:00 2024-04-25T11:13:07.824 INFO:teuthology.orchestra.run.smithi146.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3291.g 411 kB/s | 34 kB 00:00 2024-04-25T11:13:07.850 INFO:teuthology.orchestra.run.smithi069.stdout:(27/152): rbd-nbd-19.0.0-3291.g994e2f20.el9.x86 1.2 MB/s | 169 kB 00:00 2024-04-25T11:13:07.900 INFO:teuthology.orchestra.run.smithi146.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3291.g994e2f2 1.9 MB/s | 142 kB 00:00 2024-04-25T11:13:07.903 INFO:teuthology.orchestra.run.smithi152.stdout:(5/152): ceph-mgr-19.0.0-3291.g994e2f20.el9.x86 3.0 MB/s | 1.5 MB 00:00 2024-04-25T11:13:07.934 INFO:teuthology.orchestra.run.smithi069.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3291.g 410 kB/s | 34 kB 00:00 2024-04-25T11:13:08.042 INFO:teuthology.orchestra.run.smithi069.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3291.g994e2f2 1.3 MB/s | 142 kB 00:00 2024-04-25T11:13:08.558 INFO:teuthology.orchestra.run.smithi146.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3291.g994e2 5.4 MB/s | 3.5 MB 00:00 2024-04-25T11:13:08.634 INFO:teuthology.orchestra.run.smithi146.stdout:(31/152): rbd-mirror-19.0.0-3291.g994e2f20.el9. 3.0 MB/s | 3.0 MB 00:01 2024-04-25T11:13:08.637 INFO:teuthology.orchestra.run.smithi152.stdout:(6/152): ceph-base-19.0.0-3291.g994e2f20.el9.x8 1.6 MB/s | 5.4 MB 00:03 2024-04-25T11:13:08.751 INFO:teuthology.orchestra.run.smithi146.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3291.g99 2.1 MB/s | 249 kB 00:00 2024-04-25T11:13:08.851 INFO:teuthology.orchestra.run.smithi146.stdout:(33/152): ceph-mgr-rook-19.0.0-3291.g994e2f20.e 494 kB/s | 49 kB 00:00 2024-04-25T11:13:08.943 INFO:teuthology.orchestra.run.smithi146.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3291.g9 179 kB/s | 16 kB 00:00 2024-04-25T11:13:09.160 INFO:teuthology.orchestra.run.smithi146.stdout:(35/152): ceph-volume-19.0.0-3291.g994e2f20.el9 1.2 MB/s | 273 kB 00:00 2024-04-25T11:13:09.444 INFO:teuthology.orchestra.run.smithi146.stdout:(36/152): cephadm-19.0.0-3291.g994e2f20.el9.noa 2.6 MB/s | 762 kB 00:00 2024-04-25T11:13:09.467 INFO:teuthology.orchestra.run.smithi069.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3291.g994e2 2.5 MB/s | 3.5 MB 00:01 2024-04-25T11:13:09.527 INFO:teuthology.orchestra.run.smithi146.stdout:(37/152): cephfs-top-19.0.0-3291.g994e2f20.el9. 220 kB/s | 18 kB 00:00 2024-04-25T11:13:09.586 INFO:teuthology.orchestra.run.smithi146.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 1.3 MB/s | 80 kB 00:00 2024-04-25T11:13:09.694 INFO:teuthology.orchestra.run.smithi146.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 7.2 MB/s | 803 kB 00:00 2024-04-25T11:13:09.721 INFO:teuthology.orchestra.run.smithi152.stdout:(7/152): ceph-mon-19.0.0-3291.g994e2f20.el9.x86 2.7 MB/s | 4.9 MB 00:01 2024-04-25T11:13:09.770 INFO:teuthology.orchestra.run.smithi146.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 195 kB 00:00 2024-04-25T11:13:09.845 INFO:teuthology.orchestra.run.smithi146.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.3 MB/s | 253 kB 00:00 2024-04-25T11:13:10.004 INFO:teuthology.orchestra.run.smithi146.stdout:(42/152): ceph-mgr-diskprediction-local-19.0.0- 5.1 MB/s | 7.4 MB 00:01 2024-04-25T11:13:10.037 INFO:teuthology.orchestra.run.smithi146.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 6.4 MB/s | 1.2 MB 00:00 2024-04-25T11:13:10.096 INFO:teuthology.orchestra.run.smithi146.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 106 kB 00:00 2024-04-25T11:13:10.130 INFO:teuthology.orchestra.run.smithi146.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 135 kB 00:00 2024-04-25T11:13:10.188 INFO:teuthology.orchestra.run.smithi146.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.3 MB/s | 125 kB 00:00 2024-04-25T11:13:10.230 INFO:teuthology.orchestra.run.smithi146.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.1 MB/s | 215 kB 00:00 2024-04-25T11:13:10.480 INFO:teuthology.orchestra.run.smithi146.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 2.3 MB/s | 689 kB 00:00 2024-04-25T11:13:10.539 INFO:teuthology.orchestra.run.smithi146.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-04-25T11:13:10.572 INFO:teuthology.orchestra.run.smithi146.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 2.7 MB/s | 943 kB 00:00 2024-04-25T11:13:10.598 INFO:teuthology.orchestra.run.smithi146.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 368 kB/s | 21 kB 00:00 2024-04-25T11:13:10.631 INFO:teuthology.orchestra.run.smithi146.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 313 kB 00:00 2024-04-25T11:13:10.656 INFO:teuthology.orchestra.run.smithi146.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 559 kB/s | 33 kB 00:00 2024-04-25T11:13:10.715 INFO:teuthology.orchestra.run.smithi146.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 313 kB/s | 18 kB 00:00 2024-04-25T11:13:10.774 INFO:teuthology.orchestra.run.smithi146.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 191 kB/s | 11 kB 00:00 2024-04-25T11:13:10.841 INFO:teuthology.orchestra.run.smithi146.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.4 MB/s | 160 kB 00:00 2024-04-25T11:13:10.899 INFO:teuthology.orchestra.run.smithi146.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 778 kB/s | 45 kB 00:00 2024-04-25T11:13:11.025 INFO:teuthology.orchestra.run.smithi146.stdout:(58/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.6 MB/s | 3.0 MB 00:00 2024-04-25T11:13:11.050 INFO:teuthology.orchestra.run.smithi146.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 662 kB 00:00 2024-04-25T11:13:11.109 INFO:teuthology.orchestra.run.smithi146.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 436 kB/s | 25 kB 00:00 2024-04-25T11:13:11.134 INFO:teuthology.orchestra.run.smithi146.stdout:(61/152): libtool-2.4.6-45.el9.x86_64.rpm 5.2 MB/s | 582 kB 00:00 2024-04-25T11:13:11.167 INFO:teuthology.orchestra.run.smithi146.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 811 kB/s | 47 kB 00:00 2024-04-25T11:13:11.192 INFO:teuthology.orchestra.run.smithi146.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-04-25T11:13:11.210 INFO:teuthology.orchestra.run.smithi069.stdout:(31/152): ceph-osd-19.0.0-3291.g994e2f20.el9.x8 1.8 MB/s | 18 MB 00:09 2024-04-25T11:13:11.226 INFO:teuthology.orchestra.run.smithi146.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-04-25T11:13:11.251 INFO:teuthology.orchestra.run.smithi146.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 601 kB/s | 35 kB 00:00 2024-04-25T11:13:11.302 INFO:teuthology.orchestra.run.smithi069.stdout:(32/152): ceph-mgr-diskprediction-local-19.0.0- 4.0 MB/s | 7.4 MB 00:01 2024-04-25T11:13:11.352 INFO:teuthology.orchestra.run.smithi069.stdout:(33/152): ceph-mgr-rook-19.0.0-3291.g994e2f20.e 992 kB/s | 49 kB 00:00 2024-04-25T11:13:11.410 INFO:teuthology.orchestra.run.smithi069.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3291.g9 283 kB/s | 16 kB 00:00 2024-04-25T11:13:11.435 INFO:teuthology.orchestra.run.smithi069.stdout:(35/152): ceph-mgr-modules-core-19.0.0-3291.g99 1.1 MB/s | 249 kB 00:00 2024-04-25T11:13:11.511 INFO:teuthology.orchestra.run.smithi069.stdout:(36/152): ceph-volume-19.0.0-3291.g994e2f20.el9 2.7 MB/s | 273 kB 00:00 2024-04-25T11:13:11.570 INFO:teuthology.orchestra.run.smithi069.stdout:(37/152): cephfs-top-19.0.0-3291.g994e2f20.el9. 312 kB/s | 18 kB 00:00 2024-04-25T11:13:11.577 INFO:teuthology.orchestra.run.smithi146.stdout:(66/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-04-25T11:13:11.685 INFO:teuthology.orchestra.run.smithi146.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 10 MB/s | 4.7 MB 00:00 2024-04-25T11:13:11.696 INFO:teuthology.orchestra.run.smithi152.stdout:(8/152): ceph-radosgw-19.0.0-3291.g994e2f20.el9 5.1 MB/s | 10 MB 00:01 2024-04-25T11:13:11.746 INFO:teuthology.orchestra.run.smithi152.stdout:(9/152): ceph-selinux-19.0.0-3291.g994e2f20.el9 507 kB/s | 25 kB 00:00 2024-04-25T11:13:11.761 INFO:teuthology.orchestra.run.smithi146.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 8.7 MB/s | 1.6 MB 00:00 2024-04-25T11:13:11.762 INFO:teuthology.orchestra.run.smithi069.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 420 kB/s | 80 kB 00:00 2024-04-25T11:13:11.787 INFO:teuthology.orchestra.run.smithi069.stdout:(39/152): cephadm-19.0.0-3291.g994e2f20.el9.noa 2.1 MB/s | 762 kB 00:00 2024-04-25T11:13:11.794 INFO:teuthology.orchestra.run.smithi146.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 8.2 MB/s | 911 kB 00:00 2024-04-25T11:13:11.820 INFO:teuthology.orchestra.run.smithi146.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-04-25T11:13:11.987 INFO:teuthology.orchestra.run.smithi146.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 6.1 MB/s | 1.0 MB 00:00 2024-04-25T11:13:12.095 INFO:teuthology.orchestra.run.smithi146.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 808 kB/s | 87 kB 00:00 2024-04-25T11:13:12.337 INFO:teuthology.orchestra.run.smithi069.stdout:(40/152): ceph-test-19.0.0-3291.g994e2f20.el9.x 4.5 MB/s | 43 MB 00:09 2024-04-25T11:13:12.354 INFO:teuthology.orchestra.run.smithi146.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 10 MB/s | 5.8 MB 00:00 2024-04-25T11:13:12.362 INFO:teuthology.orchestra.run.smithi069.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 339 kB/s | 195 kB 00:00 2024-04-25T11:13:12.387 INFO:teuthology.orchestra.run.smithi069.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 803 kB 00:00 2024-04-25T11:13:12.413 INFO:teuthology.orchestra.run.smithi146.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.1 MB/s | 244 kB 00:00 2024-04-25T11:13:12.454 INFO:teuthology.orchestra.run.smithi069.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-04-25T11:13:12.479 INFO:teuthology.orchestra.run.smithi146.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 3.6 MB/s | 248 kB 00:00 2024-04-25T11:13:12.513 INFO:teuthology.orchestra.run.smithi069.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-25T11:13:12.555 INFO:teuthology.orchestra.run.smithi146.stdout:(76/152): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 6.0 MB 00:00 2024-04-25T11:13:12.580 INFO:teuthology.orchestra.run.smithi069.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-04-25T11:13:12.580 INFO:teuthology.orchestra.run.smithi146.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 255 kB/s | 26 kB 00:00 2024-04-25T11:13:12.605 INFO:teuthology.orchestra.run.smithi069.stdout:(46/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 947 kB/s | 253 kB 00:00 2024-04-25T11:13:12.622 INFO:teuthology.orchestra.run.smithi146.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 286 kB/s | 19 kB 00:00 2024-04-25T11:13:12.639 INFO:teuthology.orchestra.run.smithi069.stdout:(47/152): python3-cryptography-36.0.1-4.el9.x86 4.4 MB/s | 1.2 MB 00:00 2024-04-25T11:13:12.647 INFO:teuthology.orchestra.run.smithi146.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-04-25T11:13:12.673 INFO:teuthology.orchestra.run.smithi069.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-04-25T11:13:12.681 INFO:teuthology.orchestra.run.smithi146.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 593 kB/s | 35 kB 00:00 2024-04-25T11:13:12.739 INFO:teuthology.orchestra.run.smithi069.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-04-25T11:13:12.748 INFO:teuthology.orchestra.run.smithi146.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 250 kB 00:00 2024-04-25T11:13:12.773 INFO:teuthology.orchestra.run.smithi069.stdout:(50/152): automake-1.16.2-8.el9.noarch.rpm 4.0 MB/s | 689 kB 00:00 2024-04-25T11:13:12.806 INFO:teuthology.orchestra.run.smithi069.stdout:(51/152): bison-3.7.4-5.el9.x86_64.rpm 5.5 MB/s | 943 kB 00:00 2024-04-25T11:13:12.815 INFO:teuthology.orchestra.run.smithi146.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.3 MB/s | 222 kB 00:00 2024-04-25T11:13:12.832 INFO:teuthology.orchestra.run.smithi069.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 233 kB/s | 21 kB 00:00 2024-04-25T11:13:12.874 INFO:teuthology.orchestra.run.smithi069.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 489 kB/s | 33 kB 00:00 2024-04-25T11:13:12.882 INFO:teuthology.orchestra.run.smithi146.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-04-25T11:13:12.899 INFO:teuthology.orchestra.run.smithi069.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 2.4 MB/s | 313 kB 00:00 2024-04-25T11:13:12.941 INFO:teuthology.orchestra.run.smithi069.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 273 kB/s | 18 kB 00:00 2024-04-25T11:13:12.949 INFO:teuthology.orchestra.run.smithi146.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 634 kB/s | 42 kB 00:00 2024-04-25T11:13:12.966 INFO:teuthology.orchestra.run.smithi069.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 167 kB/s | 11 kB 00:00 2024-04-25T11:13:13.016 INFO:teuthology.orchestra.run.smithi146.stdout:(85/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 14 MB/s | 5.1 MB 00:00 2024-04-25T11:13:13.024 INFO:teuthology.orchestra.run.smithi069.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 780 kB/s | 45 kB 00:00 2024-04-25T11:13:13.041 INFO:teuthology.orchestra.run.smithi146.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 2.9 MB/s | 269 kB 00:00 2024-04-25T11:13:13.050 INFO:teuthology.orchestra.run.smithi069.stdout:(58/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-04-25T11:13:13.083 INFO:teuthology.orchestra.run.smithi146.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-04-25T11:13:13.100 INFO:teuthology.orchestra.run.smithi069.stdout:(59/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2024-04-25T11:13:13.108 INFO:teuthology.orchestra.run.smithi146.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-04-25T11:13:13.142 INFO:teuthology.orchestra.run.smithi146.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 875 kB/s | 51 kB 00:00 2024-04-25T11:13:13.150 INFO:teuthology.orchestra.run.smithi069.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 509 kB/s | 25 kB 00:00 2024-04-25T11:13:13.167 INFO:teuthology.orchestra.run.smithi146.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 913 kB/s | 54 kB 00:00 2024-04-25T11:13:13.184 INFO:teuthology.orchestra.run.smithi069.stdout:(61/152): librdkafka-1.6.1-102.el9.x86_64.rpm 4.1 MB/s | 662 kB 00:00 2024-04-25T11:13:13.209 INFO:teuthology.orchestra.run.smithi069.stdout:(62/152): libtool-2.4.6-45.el9.x86_64.rpm 3.6 MB/s | 582 kB 00:00 2024-04-25T11:13:13.225 INFO:teuthology.orchestra.run.smithi146.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 717 kB/s | 42 kB 00:00 2024-04-25T11:13:13.235 INFO:teuthology.orchestra.run.smithi069.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 564 kB/s | 47 kB 00:00 2024-04-25T11:13:13.284 INFO:teuthology.orchestra.run.smithi146.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 758 kB/s | 44 kB 00:00 2024-04-25T11:13:13.285 INFO:teuthology.orchestra.run.smithi069.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-04-25T11:13:13.310 INFO:teuthology.orchestra.run.smithi069.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 469 kB/s | 35 kB 00:00 2024-04-25T11:13:13.336 INFO:teuthology.orchestra.run.smithi069.stdout:(66/152): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-04-25T11:13:13.351 INFO:teuthology.orchestra.run.smithi146.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-04-25T11:13:13.410 INFO:teuthology.orchestra.run.smithi146.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 617 kB/s | 36 kB 00:00 2024-04-25T11:13:13.468 INFO:teuthology.orchestra.run.smithi146.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 814 kB/s | 47 kB 00:00 2024-04-25T11:13:13.499 INFO:teuthology.orchestra.run.smithi069.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 9.8 MB/s | 1.6 MB 00:00 2024-04-25T11:13:13.544 INFO:teuthology.orchestra.run.smithi146.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.0 MB/s | 305 kB 00:00 2024-04-25T11:13:13.603 INFO:teuthology.orchestra.run.smithi069.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 15 MB/s | 4.7 MB 00:00 2024-04-25T11:13:13.619 INFO:teuthology.orchestra.run.smithi146.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 770 kB/s | 58 kB 00:00 2024-04-25T11:13:13.661 INFO:teuthology.orchestra.run.smithi069.stdout:(69/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-04-25T11:13:13.694 INFO:teuthology.orchestra.run.smithi146.stdout:(98/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.2 MB/s | 94 kB 00:00 2024-04-25T11:13:13.712 INFO:teuthology.orchestra.run.smithi069.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 4.2 MB/s | 911 kB 00:00 2024-04-25T11:13:13.745 INFO:teuthology.orchestra.run.smithi069.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 939 kB/s | 133 kB 00:00 2024-04-25T11:13:13.753 INFO:teuthology.orchestra.run.smithi146.stdout:(99/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-25T11:13:13.804 INFO:teuthology.orchestra.run.smithi069.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-25T11:13:13.854 INFO:teuthology.orchestra.run.smithi069.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 7.1 MB/s | 1.0 MB 00:00 2024-04-25T11:13:13.904 INFO:teuthology.orchestra.run.smithi146.stdout:(100/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 21 MB/s | 16 MB 00:00 2024-04-25T11:13:13.929 INFO:teuthology.orchestra.run.smithi146.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 256 kB/s | 45 kB 00:00 2024-04-25T11:13:13.962 INFO:teuthology.orchestra.run.smithi152.stdout:(10/152): ceph-common-19.0.0-3291.g994e2f20.el9 2.3 MB/s | 20 MB 00:08 2024-04-25T11:13:13.970 INFO:teuthology.orchestra.run.smithi146.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 335 kB/s | 22 kB 00:00 2024-04-25T11:13:14.046 INFO:teuthology.orchestra.run.smithi146.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 7.2 MB/s | 863 kB 00:00 2024-04-25T11:13:14.079 INFO:teuthology.orchestra.run.smithi152.stdout:(11/152): cephfs-mirror-19.0.0-3291.g994e2f20.e 1.9 MB/s | 224 kB 00:00 2024-04-25T11:13:14.112 INFO:teuthology.orchestra.run.smithi069.stdout:(74/152): postgresql-server-13.14-1.el9.x86_64. 13 MB/s | 5.8 MB 00:00 2024-04-25T11:13:14.138 INFO:teuthology.orchestra.run.smithi069.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 861 kB/s | 244 kB 00:00 2024-04-25T11:13:14.138 INFO:teuthology.orchestra.run.smithi152.stdout:(12/152): cephfs-shell-19.0.0-3291.g994e2f20.el 465 kB/s | 27 kB 00:00 2024-04-25T11:13:14.188 INFO:teuthology.orchestra.run.smithi069.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-04-25T11:13:14.222 INFO:teuthology.orchestra.run.smithi069.stdout:(77/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-04-25T11:13:14.235 INFO:teuthology.orchestra.run.smithi152.stdout:(13/152): libcephfs-devel-19.0.0-3291.g994e2f20 328 kB/s | 32 kB 00:00 2024-04-25T11:13:14.255 INFO:teuthology.orchestra.run.smithi069.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 287 kB/s | 19 kB 00:00 2024-04-25T11:13:14.330 INFO:teuthology.orchestra.run.smithi069.stdout:(79/152): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-04-25T11:13:14.364 INFO:teuthology.orchestra.run.smithi069.stdout:(80/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.2 MB/s | 172 kB 00:00 2024-04-25T11:13:14.381 INFO:teuthology.orchestra.run.smithi069.stdout:(81/152): python3-markupsafe-1.1.1-12.el9.x86_6 276 kB/s | 35 kB 00:00 2024-04-25T11:13:14.413 INFO:teuthology.orchestra.run.smithi146.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.2 MB/s | 548 kB 00:00 2024-04-25T11:13:14.438 INFO:teuthology.orchestra.run.smithi146.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 789 kB/s | 309 kB 00:00 2024-04-25T11:13:14.448 INFO:teuthology.orchestra.run.smithi069.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.9 MB/s | 250 kB 00:00 2024-04-25T11:13:14.473 INFO:teuthology.orchestra.run.smithi069.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-04-25T11:13:14.488 INFO:teuthology.orchestra.run.smithi146.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 258 kB/s | 19 kB 00:00 2024-04-25T11:13:14.532 INFO:teuthology.orchestra.run.smithi069.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 927 kB/s | 77 kB 00:00 2024-04-25T11:13:14.549 INFO:teuthology.orchestra.run.smithi069.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 562 kB/s | 42 kB 00:00 2024-04-25T11:13:14.597 INFO:teuthology.orchestra.run.smithi146.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 240 kB/s | 26 kB 00:00 2024-04-25T11:13:14.597 INFO:teuthology.orchestra.run.smithi152.stdout:(14/152): ceph-osd-19.0.0-3291.g994e2f20.el9.x8 3.0 MB/s | 18 MB 00:05 2024-04-25T11:13:14.622 INFO:teuthology.orchestra.run.smithi152.stdout:(15/152): libcephfs2-19.0.0-3291.g994e2f20.el9. 1.9 MB/s | 756 kB 00:00 2024-04-25T11:13:14.641 INFO:teuthology.orchestra.run.smithi069.stdout:(86/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 5.1 MB 00:00 2024-04-25T11:13:14.682 INFO:teuthology.orchestra.run.smithi069.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-04-25T11:13:14.697 INFO:teuthology.orchestra.run.smithi146.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 486 kB/s | 49 kB 00:00 2024-04-25T11:13:14.708 INFO:teuthology.orchestra.run.smithi069.stdout:(88/152): python3-protobuf-3.14.0-13.el9.noarch 1.5 MB/s | 269 kB 00:00 2024-04-25T11:13:14.714 INFO:teuthology.orchestra.run.smithi152.stdout:(16/152): libcephsqlite-19.0.0-3291.g994e2f20.e 1.3 MB/s | 158 kB 00:00 2024-04-25T11:13:14.740 INFO:teuthology.orchestra.run.smithi152.stdout:(17/152): librados-devel-19.0.0-3291.g994e2f20. 1.1 MB/s | 127 kB 00:00 2024-04-25T11:13:14.758 INFO:teuthology.orchestra.run.smithi069.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-04-25T11:13:14.783 INFO:teuthology.orchestra.run.smithi069.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 507 kB/s | 51 kB 00:00 2024-04-25T11:13:14.808 INFO:teuthology.orchestra.run.smithi069.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 534 kB/s | 54 kB 00:00 2024-04-25T11:13:14.859 INFO:teuthology.orchestra.run.smithi069.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 552 kB/s | 42 kB 00:00 2024-04-25T11:13:14.893 INFO:teuthology.orchestra.run.smithi069.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 526 kB/s | 44 kB 00:00 2024-04-25T11:13:14.932 INFO:teuthology.orchestra.run.smithi152.stdout:(18/152): libradosstriper1-19.0.0-3291.g994e2f2 2.2 MB/s | 497 kB 00:00 2024-04-25T11:13:14.943 INFO:teuthology.orchestra.run.smithi069.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 717 kB/s | 36 kB 00:00 2024-04-25T11:13:14.964 INFO:teuthology.orchestra.run.smithi146.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 252 kB/s | 67 kB 00:00 2024-04-25T11:13:14.968 INFO:teuthology.orchestra.run.smithi069.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.0 MB/s | 221 kB 00:00 2024-04-25T11:13:14.998 INFO:teuthology.orchestra.run.smithi152.stdout:(19/152): python3-ceph-argparse-19.0.0-3291.g99 678 kB/s | 45 kB 00:00 2024-04-25T11:13:15.018 INFO:teuthology.orchestra.run.smithi069.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 630 kB/s | 47 kB 00:00 2024-04-25T11:13:15.066 INFO:teuthology.orchestra.run.smithi152.stdout:(20/152): python3-ceph-common-19.0.0-3291.g994e 2.0 MB/s | 135 kB 00:00 2024-04-25T11:13:15.068 INFO:teuthology.orchestra.run.smithi069.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 3.0 MB/s | 305 kB 00:00 2024-04-25T11:13:15.089 INFO:teuthology.orchestra.run.smithi146.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 1.2 MB/s | 151 kB 00:00 2024-04-25T11:13:15.094 INFO:teuthology.orchestra.run.smithi069.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 767 kB/s | 58 kB 00:00 2024-04-25T11:13:15.142 INFO:teuthology.orchestra.run.smithi152.stdout:(21/152): python3-cephfs-19.0.0-3291.g994e2f20. 2.1 MB/s | 161 kB 00:00 2024-04-25T11:13:15.152 INFO:teuthology.orchestra.run.smithi069.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.1 MB/s | 94 kB 00:00 2024-04-25T11:13:15.178 INFO:teuthology.orchestra.run.smithi069.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 763 kB/s | 64 kB 00:00 2024-04-25T11:13:15.237 INFO:teuthology.orchestra.run.smithi069.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 535 kB/s | 45 kB 00:00 2024-04-25T11:13:15.258 INFO:teuthology.orchestra.run.smithi152.stdout:(22/152): python3-rados-19.0.0-3291.g994e2f20.e 2.7 MB/s | 321 kB 00:00 2024-04-25T11:13:15.262 INFO:teuthology.orchestra.run.smithi069.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 266 kB/s | 22 kB 00:00 2024-04-25T11:13:15.375 INFO:teuthology.orchestra.run.smithi152.stdout:(23/152): python3-rbd-19.0.0-3291.g994e2f20.el9 2.5 MB/s | 298 kB 00:00 2024-04-25T11:13:15.429 INFO:teuthology.orchestra.run.smithi069.stdout:(103/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2024-04-25T11:13:15.442 INFO:teuthology.orchestra.run.smithi152.stdout:(24/152): python3-rgw-19.0.0-3291.g994e2f20.el9 1.5 MB/s | 100 kB 00:00 2024-04-25T11:13:15.509 INFO:teuthology.orchestra.run.smithi152.stdout:(25/152): rbd-fuse-19.0.0-3291.g994e2f20.el9.x8 1.3 MB/s | 86 kB 00:00 2024-04-25T11:13:15.529 INFO:teuthology.orchestra.run.smithi069.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 2.9 MB/s | 863 kB 00:00 2024-04-25T11:13:15.646 INFO:teuthology.orchestra.run.smithi069.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.4 MB/s | 548 kB 00:00 2024-04-25T11:13:15.671 INFO:teuthology.orchestra.run.smithi069.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 138 kB/s | 19 kB 00:00 2024-04-25T11:13:15.689 INFO:teuthology.orchestra.run.smithi146.stdout:(111/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.4 MB/s | 838 kB 00:00 2024-04-25T11:13:15.705 INFO:teuthology.orchestra.run.smithi069.stdout:(107/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-04-25T11:13:15.746 INFO:teuthology.orchestra.run.smithi069.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 347 kB/s | 26 kB 00:00 2024-04-25T11:13:15.830 INFO:teuthology.orchestra.run.smithi069.stdout:(109/152): libarrow-9.0.0-10.el9.x86_64.rpm 28 MB/s | 4.4 MB 00:00 2024-04-25T11:13:15.855 INFO:teuthology.orchestra.run.smithi069.stdout:(110/152): liboath-2.6.7-2.el9.x86_64.rpm 325 kB/s | 49 kB 00:00 2024-04-25T11:13:15.856 INFO:teuthology.orchestra.run.smithi146.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 3.2 MB/s | 548 kB 00:00 2024-04-25T11:13:15.880 INFO:teuthology.orchestra.run.smithi069.stdout:(111/152): libunwind-1.6.2-1.el9.x86_64.rpm 511 kB/s | 67 kB 00:00 2024-04-25T11:13:15.906 INFO:teuthology.orchestra.run.smithi069.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-04-25T11:13:15.915 INFO:teuthology.orchestra.run.smithi146.stdout:(113/152): libarrow-9.0.0-10.el9.x86_64.rpm 3.0 MB/s | 4.4 MB 00:01 2024-04-25T11:13:15.931 INFO:teuthology.orchestra.run.smithi069.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 11 MB/s | 838 kB 00:00 2024-04-25T11:13:15.940 INFO:teuthology.orchestra.run.smithi146.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 521 kB/s | 43 kB 00:00 2024-04-25T11:13:15.956 INFO:teuthology.orchestra.run.smithi069.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 7.2 MB/s | 548 kB 00:00 2024-04-25T11:13:15.990 INFO:teuthology.orchestra.run.smithi146.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 429 kB/s | 32 kB 00:00 2024-04-25T11:13:16.032 INFO:teuthology.orchestra.run.smithi146.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 163 kB/s | 15 kB 00:00 2024-04-25T11:13:16.048 INFO:teuthology.orchestra.run.smithi069.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 308 kB/s | 43 kB 00:00 2024-04-25T11:13:16.066 INFO:teuthology.orchestra.run.smithi146.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-04-25T11:13:16.098 INFO:teuthology.orchestra.run.smithi069.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 227 kB/s | 32 kB 00:00 2024-04-25T11:13:16.124 INFO:teuthology.orchestra.run.smithi069.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 90 kB/s | 15 kB 00:00 2024-04-25T11:13:16.149 INFO:teuthology.orchestra.run.smithi069.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-04-25T11:13:16.158 INFO:teuthology.orchestra.run.smithi146.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-04-25T11:13:16.168 INFO:teuthology.orchestra.run.smithi152.stdout:(26/152): ceph-test-19.0.0-3291.g994e2f20.el9.x 9.8 MB/s | 43 MB 00:04 2024-04-25T11:13:16.174 INFO:teuthology.orchestra.run.smithi069.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-04-25T11:13:16.183 INFO:teuthology.orchestra.run.smithi146.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-04-25T11:13:16.199 INFO:teuthology.orchestra.run.smithi069.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-04-25T11:13:16.224 INFO:teuthology.orchestra.run.smithi069.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 476 kB/s | 35 kB 00:00 2024-04-25T11:13:16.233 INFO:teuthology.orchestra.run.smithi146.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 470 kB/s | 35 kB 00:00 2024-04-25T11:13:16.243 INFO:teuthology.orchestra.run.smithi152.stdout:(27/152): librgw2-19.0.0-3291.g994e2f20.el9.x86 3.4 MB/s | 5.2 MB 00:01 2024-04-25T11:13:16.250 INFO:teuthology.orchestra.run.smithi069.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 4.0 MB/s | 206 kB 00:00 2024-04-25T11:13:16.269 INFO:teuthology.orchestra.run.smithi152.stdout:(28/152): rbd-nbd-19.0.0-3291.g994e2f20.el9.x86 1.7 MB/s | 169 kB 00:00 2024-04-25T11:13:16.275 INFO:teuthology.orchestra.run.smithi146.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 2.2 MB/s | 206 kB 00:00 2024-04-25T11:13:16.292 INFO:teuthology.orchestra.run.smithi069.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 22 MB/s | 2.0 MB 00:00 2024-04-25T11:13:16.294 INFO:teuthology.orchestra.run.smithi152.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3291.g 684 kB/s | 34 kB 00:00 2024-04-25T11:13:16.317 INFO:teuthology.orchestra.run.smithi069.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 1.6 MB/s | 144 kB 00:00 2024-04-25T11:13:16.319 INFO:teuthology.orchestra.run.smithi152.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3291.g994e2f2 2.8 MB/s | 142 kB 00:00 2024-04-25T11:13:16.334 INFO:teuthology.orchestra.run.smithi069.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 130 kB/s | 11 kB 00:00 2024-04-25T11:13:16.361 INFO:teuthology.orchestra.run.smithi069.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 402 kB/s | 18 kB 00:00 2024-04-25T11:13:16.378 INFO:teuthology.orchestra.run.smithi152.stdout:(31/152): rbd-mirror-19.0.0-3291.g994e2f20.el9. 3.5 MB/s | 3.0 MB 00:00 2024-04-25T11:13:16.386 INFO:teuthology.orchestra.run.smithi069.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 335 kB/s | 23 kB 00:00 2024-04-25T11:13:16.411 INFO:teuthology.orchestra.run.smithi069.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 251 kB/s | 19 kB 00:00 2024-04-25T11:13:16.437 INFO:teuthology.orchestra.run.smithi069.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 394 kB/s | 20 kB 00:00 2024-04-25T11:13:16.470 INFO:teuthology.orchestra.run.smithi069.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 12 MB/s | 1.0 MB 00:00 2024-04-25T11:13:16.486 INFO:teuthology.orchestra.run.smithi152.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3291.g99 2.2 MB/s | 249 kB 00:00 2024-04-25T11:13:16.495 INFO:teuthology.orchestra.run.smithi069.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 565 kB/s | 46 kB 00:00 2024-04-25T11:13:16.521 INFO:teuthology.orchestra.run.smithi069.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-04-25T11:13:16.546 INFO:teuthology.orchestra.run.smithi069.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 770 kB/s | 58 kB 00:00 2024-04-25T11:13:16.553 INFO:teuthology.orchestra.run.smithi152.stdout:(33/152): ceph-mgr-rook-19.0.0-3291.g994e2f20.e 741 kB/s | 49 kB 00:00 2024-04-25T11:13:16.571 INFO:teuthology.orchestra.run.smithi069.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-04-25T11:13:16.597 INFO:teuthology.orchestra.run.smithi069.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 330 kB/s | 16 kB 00:00 2024-04-25T11:13:16.612 INFO:teuthology.orchestra.run.smithi152.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3291.g9 281 kB/s | 16 kB 00:00 2024-04-25T11:13:16.622 INFO:teuthology.orchestra.run.smithi069.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-25T11:13:16.647 INFO:teuthology.orchestra.run.smithi069.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 375 kB/s | 28 kB 00:00 2024-04-25T11:13:16.672 INFO:teuthology.orchestra.run.smithi069.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 617 kB/s | 31 kB 00:00 2024-04-25T11:13:16.697 INFO:teuthology.orchestra.run.smithi069.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.5 MB/s | 188 kB 00:00 2024-04-25T11:13:16.723 INFO:teuthology.orchestra.run.smithi069.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 790 kB/s | 59 kB 00:00 2024-04-25T11:13:16.729 INFO:teuthology.orchestra.run.smithi152.stdout:(35/152): ceph-volume-19.0.0-3291.g994e2f20.el9 2.3 MB/s | 273 kB 00:00 2024-04-25T11:13:16.748 INFO:teuthology.orchestra.run.smithi069.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 716 kB/s | 36 kB 00:00 2024-04-25T11:13:16.759 INFO:teuthology.orchestra.run.smithi146.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 299 kB/s | 144 kB 00:00 2024-04-25T11:13:16.773 INFO:teuthology.orchestra.run.smithi069.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 681 kB/s | 51 kB 00:00 2024-04-25T11:13:16.798 INFO:teuthology.orchestra.run.smithi069.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-25T11:13:16.823 INFO:teuthology.orchestra.run.smithi069.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-04-25T11:13:16.849 INFO:teuthology.orchestra.run.smithi069.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 5.6 MB/s | 427 kB 00:00 2024-04-25T11:13:16.851 INFO:teuthology.orchestra.run.smithi146.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 117 kB/s | 11 kB 00:00 2024-04-25T11:13:16.874 INFO:teuthology.orchestra.run.smithi069.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 269 kB/s | 20 kB 00:00 2024-04-25T11:13:16.899 INFO:teuthology.orchestra.run.smithi069.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 3.8 MB/s | 192 kB 00:00 2024-04-25T11:13:16.930 INFO:teuthology.orchestra.run.smithi152.stdout:(36/152): cephadm-19.0.0-3291.g994e2f20.el9.noa 3.7 MB/s | 762 kB 00:00 2024-04-25T11:13:16.941 INFO:teuthology.orchestra.run.smithi069.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-04-25T11:13:16.951 INFO:teuthology.orchestra.run.smithi146.stdout:(124/152): python3-grpcio-1.46.7-10.el9.x86_64. 2.8 MB/s | 2.0 MB 00:00 2024-04-25T11:13:16.976 INFO:teuthology.orchestra.run.smithi146.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 141 kB/s | 18 kB 00:00 2024-04-25T11:13:17.046 INFO:teuthology.orchestra.run.smithi152.stdout:(37/152): ceph-mgr-diskprediction-local-19.0.0- 10 MB/s | 7.4 MB 00:00 2024-04-25T11:13:17.071 INFO:teuthology.orchestra.run.smithi152.stdout:(38/152): cephfs-top-19.0.0-3291.g994e2f20.el9. 129 kB/s | 18 kB 00:00 2024-04-25T11:13:17.130 INFO:teuthology.orchestra.run.smithi152.stdout:(39/152): ceph-mgr-dashboard-19.0.0-3291.g994e2 4.2 MB/s | 3.5 MB 00:00 2024-04-25T11:13:17.214 INFO:teuthology.orchestra.run.smithi152.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 481 kB/s | 80 kB 00:00 2024-04-25T11:13:17.281 INFO:teuthology.orchestra.run.smithi152.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 253 kB 00:00 2024-04-25T11:13:17.291 INFO:teuthology.orchestra.run.smithi069.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 3.9 MB/s | 1.3 MB 00:00 2024-04-25T11:13:17.306 INFO:teuthology.orchestra.run.smithi152.stdout:(42/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 195 kB 00:00 2024-04-25T11:13:17.310 INFO:teuthology.orchestra.run.smithi146.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 65 kB/s | 23 kB 00:00 2024-04-25T11:13:17.336 INFO:teuthology.orchestra.run.smithi146.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 54 kB/s | 19 kB 00:00 2024-04-25T11:13:17.340 INFO:teuthology.orchestra.run.smithi152.stdout:(43/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 803 kB 00:00 2024-04-25T11:13:17.365 INFO:teuthology.orchestra.run.smithi152.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-25T11:13:17.377 INFO:teuthology.orchestra.run.smithi146.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 293 kB/s | 20 kB 00:00 2024-04-25T11:13:17.391 INFO:teuthology.orchestra.run.smithi152.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-25T11:13:17.392 INFO:teuthology.orchestra.run.smithi069.stdout:(150/152): librados2-19.0.0-3291.g994e2f20.el9. 6.5 MB/s | 3.3 MB 00:00 2024-04-25T11:13:17.415 INFO:teuthology.orchestra.run.smithi152.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-25T11:13:17.450 INFO:teuthology.orchestra.run.smithi152.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-04-25T11:13:17.452 INFO:teuthology.orchestra.run.smithi146.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 617 kB/s | 46 kB 00:00 2024-04-25T11:13:17.491 INFO:teuthology.orchestra.run.smithi152.stdout:(48/152): python3-cryptography-36.0.1-4.el9.x86 5.8 MB/s | 1.2 MB 00:00 2024-04-25T11:13:17.528 INFO:teuthology.orchestra.run.smithi146.stdout:(130/152): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-04-25T11:13:17.559 INFO:teuthology.orchestra.run.smithi152.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-04-25T11:13:17.561 INFO:teuthology.orchestra.run.smithi146.stdout:(131/152): python3-kubernetes-26.1.0-2.el9.noar 4.5 MB/s | 1.0 MB 00:00 2024-04-25T11:13:17.592 INFO:teuthology.orchestra.run.smithi152.stdout:(50/152): automake-1.16.2-8.el9.noarch.rpm 3.8 MB/s | 689 kB 00:00 2024-04-25T11:13:17.595 INFO:teuthology.orchestra.run.smithi146.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 864 kB/s | 58 kB 00:00 2024-04-25T11:13:17.637 INFO:teuthology.orchestra.run.smithi146.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-04-25T11:13:17.642 INFO:teuthology.orchestra.run.smithi152.stdout:(51/152): bison-3.7.4-5.el9.x86_64.rpm 4.8 MB/s | 943 kB 00:00 2024-04-25T11:13:17.658 INFO:teuthology.orchestra.run.smithi069.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 6.5 MB/s | 2.4 MB 00:00 2024-04-25T11:13:17.662 INFO:teuthology.orchestra.run.smithi146.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 245 kB/s | 16 kB 00:00 2024-04-25T11:13:17.668 INFO:teuthology.orchestra.run.smithi152.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 197 kB/s | 21 kB 00:00 2024-04-25T11:13:17.709 INFO:teuthology.orchestra.run.smithi152.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 489 kB/s | 33 kB 00:00 2024-04-25T11:13:17.712 INFO:teuthology.orchestra.run.smithi146.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-25T11:13:17.735 INFO:teuthology.orchestra.run.smithi152.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 2.1 MB/s | 313 kB 00:00 2024-04-25T11:13:17.738 INFO:teuthology.orchestra.run.smithi146.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 375 kB/s | 28 kB 00:00 2024-04-25T11:13:17.777 INFO:teuthology.orchestra.run.smithi152.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 272 kB/s | 18 kB 00:00 2024-04-25T11:13:17.796 INFO:teuthology.orchestra.run.smithi146.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 368 kB/s | 31 kB 00:00 2024-04-25T11:13:17.802 INFO:teuthology.orchestra.run.smithi152.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 166 kB/s | 11 kB 00:00 2024-04-25T11:13:17.847 INFO:teuthology.orchestra.run.smithi146.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-04-25T11:13:17.861 INFO:teuthology.orchestra.run.smithi152.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 777 kB/s | 45 kB 00:00 2024-04-25T11:13:17.880 INFO:teuthology.orchestra.run.smithi146.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 710 kB/s | 59 kB 00:00 2024-04-25T11:13:17.886 INFO:teuthology.orchestra.run.smithi152.stdout:(58/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-04-25T11:13:17.914 INFO:teuthology.orchestra.run.smithi146.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 535 kB/s | 36 kB 00:00 2024-04-25T11:13:17.944 INFO:teuthology.orchestra.run.smithi152.stdout:(59/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2024-04-25T11:13:17.947 INFO:teuthology.orchestra.run.smithi146.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 764 kB/s | 51 kB 00:00 2024-04-25T11:13:17.989 INFO:teuthology.orchestra.run.smithi146.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-25T11:13:17.995 INFO:teuthology.orchestra.run.smithi152.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 509 kB/s | 25 kB 00:00 2024-04-25T11:13:18.045 INFO:teuthology.orchestra.run.smithi152.stdout:(61/152): librdkafka-1.6.1-102.el9.x86_64.rpm 3.5 MB/s | 662 kB 00:00 2024-04-25T11:13:18.048 INFO:teuthology.orchestra.run.smithi146.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 894 kB/s | 90 kB 00:00 2024-04-25T11:13:18.063 INFO:teuthology.orchestra.run.smithi152.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 708 kB/s | 47 kB 00:00 2024-04-25T11:13:18.075 INFO:teuthology.orchestra.run.smithi069.stdout:(152/152): librbd1-19.0.0-3291.g994e2f20.el9.x8 2.6 MB/s | 3.1 MB 00:01 2024-04-25T11:13:18.081 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2024-04-25T11:13:18.081 INFO:teuthology.orchestra.run.smithi069.stdout:Total 11 MB/s | 214 MB 00:20 2024-04-25T11:13:18.096 INFO:teuthology.orchestra.run.smithi152.stdout:(63/152): libtool-2.4.6-45.el9.x86_64.rpm 2.7 MB/s | 582 kB 00:00 2024-04-25T11:13:18.106 INFO:teuthology.orchestra.run.smithi146.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-04-25T11:13:18.132 INFO:teuthology.orchestra.run.smithi146.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 240 kB/s | 20 kB 00:00 2024-04-25T11:13:18.146 INFO:teuthology.orchestra.run.smithi152.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-04-25T11:13:18.163 INFO:teuthology.orchestra.run.smithi152.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 529 kB/s | 35 kB 00:00 2024-04-25T11:13:18.197 INFO:teuthology.orchestra.run.smithi152.stdout:(66/152): lua-5.4.4-4.el9.x86_64.rpm 1.4 MB/s | 188 kB 00:00 2024-04-25T11:13:18.198 INFO:teuthology.orchestra.run.smithi146.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-04-25T11:13:18.330 INFO:teuthology.orchestra.run.smithi152.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-04-25T11:13:18.390 INFO:teuthology.orchestra.run.smithi146.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 6.1 MB/s | 1.6 MB 00:00 2024-04-25T11:13:18.431 INFO:teuthology.orchestra.run.smithi152.stdout:(68/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-04-25T11:13:18.497 INFO:teuthology.orchestra.run.smithi152.stdout:(69/152): openblas-openmp-0.3.21-2.el9.x86_64.r 14 MB/s | 4.7 MB 00:00 2024-04-25T11:13:18.556 INFO:teuthology.orchestra.run.smithi152.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 3.9 MB/s | 911 kB 00:00 2024-04-25T11:13:18.581 INFO:teuthology.orchestra.run.smithi152.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 886 kB/s | 133 kB 00:00 2024-04-25T11:13:18.640 INFO:teuthology.orchestra.run.smithi152.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-25T11:13:18.698 INFO:teuthology.orchestra.run.smithi152.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 7.1 MB/s | 1.0 MB 00:00 2024-04-25T11:13:18.741 INFO:teuthology.orchestra.run.smithi146.stdout:(148/152): librbd1-19.0.0-3291.g994e2f20.el9.x8 8.8 MB/s | 3.1 MB 00:00 2024-04-25T11:13:18.816 INFO:teuthology.orchestra.run.smithi152.stdout:(74/152): postgresql-server-13.14-1.el9.x86_64. 18 MB/s | 5.8 MB 00:00 2024-04-25T11:13:18.841 INFO:teuthology.orchestra.run.smithi152.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.7 MB/s | 244 kB 00:00 2024-04-25T11:13:18.866 INFO:teuthology.orchestra.run.smithi146.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 11 MB/s | 1.3 MB 00:00 2024-04-25T11:13:18.891 INFO:teuthology.orchestra.run.smithi152.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-04-25T11:13:18.917 INFO:teuthology.orchestra.run.smithi152.stdout:(77/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-04-25T11:13:18.959 INFO:teuthology.orchestra.run.smithi152.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 286 kB/s | 19 kB 00:00 2024-04-25T11:13:19.009 INFO:teuthology.orchestra.run.smithi152.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.8 MB/s | 172 kB 00:00 2024-04-25T11:13:19.033 INFO:teuthology.orchestra.run.smithi146.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 14 MB/s | 2.4 MB 00:00 2024-04-25T11:13:19.034 INFO:teuthology.orchestra.run.smithi152.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 461 kB/s | 35 kB 00:00 2024-04-25T11:13:19.083 INFO:teuthology.orchestra.run.smithi146.stdout:(151/152): librados2-19.0.0-3291.g994e2f20.el9. 3.8 MB/s | 3.3 MB 00:00 2024-04-25T11:13:19.110 INFO:teuthology.orchestra.run.smithi152.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 6.0 MB 00:00 2024-04-25T11:13:19.177 INFO:teuthology.orchestra.run.smithi152.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.7 MB/s | 250 kB 00:00 2024-04-25T11:13:19.186 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2024-04-25T11:13:19.202 INFO:teuthology.orchestra.run.smithi152.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-04-25T11:13:19.252 INFO:teuthology.orchestra.run.smithi152.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 77 kB 00:00 2024-04-25T11:13:19.277 INFO:teuthology.orchestra.run.smithi152.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 561 kB/s | 42 kB 00:00 2024-04-25T11:13:19.332 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction check succeeded. 2024-04-25T11:13:19.332 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2024-04-25T11:13:19.344 INFO:teuthology.orchestra.run.smithi152.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 2.9 MB/s | 269 kB 00:00 2024-04-25T11:13:19.370 INFO:teuthology.orchestra.run.smithi152.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-04-25T11:13:19.437 INFO:teuthology.orchestra.run.smithi152.stdout:(88/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-04-25T11:13:19.479 INFO:teuthology.orchestra.run.smithi152.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 469 kB/s | 51 kB 00:00 2024-04-25T11:13:19.504 INFO:teuthology.orchestra.run.smithi152.stdout:(90/152): python3-pyasn1-modules-0.4.8-6.el9.no 1.7 MB/s | 279 kB 00:00 2024-04-25T11:13:19.529 INFO:teuthology.orchestra.run.smithi152.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 581 kB/s | 54 kB 00:00 2024-04-25T11:13:19.571 INFO:teuthology.orchestra.run.smithi152.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 621 kB/s | 42 kB 00:00 2024-04-25T11:13:19.596 INFO:teuthology.orchestra.run.smithi152.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 659 kB/s | 44 kB 00:00 2024-04-25T11:13:19.647 INFO:teuthology.orchestra.run.smithi152.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 717 kB/s | 36 kB 00:00 2024-04-25T11:13:19.672 INFO:teuthology.orchestra.run.smithi152.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.1 MB/s | 221 kB 00:00 2024-04-25T11:13:19.714 INFO:teuthology.orchestra.run.smithi152.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 707 kB/s | 47 kB 00:00 2024-04-25T11:13:19.923 INFO:teuthology.orchestra.run.smithi152.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 1.2 MB/s | 305 kB 00:00 2024-04-25T11:13:19.940 INFO:teuthology.orchestra.run.smithi152.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 256 kB/s | 58 kB 00:00 2024-04-25T11:13:20.007 INFO:teuthology.orchestra.run.smithi152.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.1 MB/s | 94 kB 00:00 2024-04-25T11:13:20.032 INFO:teuthology.orchestra.run.smithi152.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 693 kB/s | 64 kB 00:00 2024-04-25T11:13:20.091 INFO:teuthology.orchestra.run.smithi152.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 537 kB/s | 45 kB 00:00 2024-04-25T11:13:20.116 INFO:teuthology.orchestra.run.smithi152.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 265 kB/s | 22 kB 00:00 2024-04-25T11:13:20.267 INFO:teuthology.orchestra.run.smithi152.stdout:(103/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 21 MB/s | 16 MB 00:00 2024-04-25T11:13:20.367 INFO:teuthology.orchestra.run.smithi152.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 3.1 MB/s | 863 kB 00:00 2024-04-25T11:13:20.675 INFO:teuthology.orchestra.run.smithi152.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 69 kB/s | 19 kB 00:00 2024-04-25T11:13:20.801 INFO:teuthology.orchestra.run.smithi152.stdout:(106/152): abseil-cpp-20211102.0-3.el9.x86_64.r 801 kB/s | 548 kB 00:00 2024-04-25T11:13:20.826 INFO:teuthology.orchestra.run.smithi152.stdout:(107/152): gperftools-libs-2.9.1-2.el9.x86_64.r 579 kB/s | 309 kB 00:00 2024-04-25T11:13:20.926 INFO:teuthology.orchestra.run.smithi152.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 207 kB/s | 26 kB 00:00 2024-04-25T11:13:20.951 INFO:teuthology.orchestra.run.smithi152.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 387 kB/s | 49 kB 00:00 2024-04-25T11:13:21.035 INFO:teuthology.orchestra.run.smithi152.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 620 kB/s | 67 kB 00:00 2024-04-25T11:13:21.058 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded. 2024-04-25T11:13:21.059 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2024-04-25T11:13:21.069 INFO:teuthology.orchestra.run.smithi152.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-04-25T11:13:21.244 INFO:teuthology.orchestra.run.smithi152.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.9 MB/s | 838 kB 00:00 2024-04-25T11:13:21.295 INFO:teuthology.orchestra.run.smithi152.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 2.4 MB/s | 548 kB 00:00 2024-04-25T11:13:21.353 INFO:teuthology.orchestra.run.smithi152.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 400 kB/s | 43 kB 00:00 2024-04-25T11:13:21.378 INFO:teuthology.orchestra.run.smithi152.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 384 kB/s | 32 kB 00:00 2024-04-25T11:13:21.445 INFO:teuthology.orchestra.run.smithi152.stdout:(116/152): libarrow-9.0.0-10.el9.x86_64.rpm 5.7 MB/s | 4.4 MB 00:00 2024-04-25T11:13:21.471 INFO:teuthology.orchestra.run.smithi152.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 128 kB/s | 15 kB 00:00 2024-04-25T11:13:21.496 INFO:teuthology.orchestra.run.smithi152.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2024-04-25T11:13:21.538 INFO:teuthology.orchestra.run.smithi152.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-04-25T11:13:21.563 INFO:teuthology.orchestra.run.smithi152.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-04-25T11:13:21.588 INFO:teuthology.orchestra.run.smithi152.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 382 kB/s | 35 kB 00:00 2024-04-25T11:13:21.622 INFO:teuthology.orchestra.run.smithi152.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.4 MB/s | 206 kB 00:00 2024-04-25T11:13:21.681 INFO:teuthology.orchestra.run.smithi152.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-04-25T11:13:21.706 INFO:teuthology.orchestra.run.smithi152.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 128 kB/s | 11 kB 00:00 2024-04-25T11:13:21.756 INFO:teuthology.orchestra.run.smithi152.stdout:(125/152): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-04-25T11:13:21.782 INFO:teuthology.orchestra.run.smithi152.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-04-25T11:13:21.807 INFO:teuthology.orchestra.run.smithi152.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 230 kB/s | 23 kB 00:00 2024-04-25T11:13:21.841 INFO:teuthology.orchestra.run.smithi152.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 231 kB/s | 19 kB 00:00 2024-04-25T11:13:21.874 INFO:teuthology.orchestra.run.smithi152.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 212 kB/s | 20 kB 00:00 2024-04-25T11:13:21.958 INFO:teuthology.orchestra.run.smithi152.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 395 kB/s | 46 kB 00:00 2024-04-25T11:13:21.983 INFO:teuthology.orchestra.run.smithi152.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 725 kB/s | 79 kB 00:00 2024-04-25T11:13:22.025 INFO:teuthology.orchestra.run.smithi146.stdout:(152/152): ceph-test-19.0.0-3291.g994e2f20.el9. 2.6 MB/s | 43 MB 00:16 2024-04-25T11:13:22.029 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2024-04-25T11:13:22.029 INFO:teuthology.orchestra.run.smithi146.stdout:Total 9.3 MB/s | 214 MB 00:23 2024-04-25T11:13:22.034 INFO:teuthology.orchestra.run.smithi152.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 764 kB/s | 58 kB 00:00 2024-04-25T11:13:22.067 INFO:teuthology.orchestra.run.smithi152.stdout:(133/152): python3-kubernetes-26.1.0-2.el9.noar 3.9 MB/s | 1.0 MB 00:00 2024-04-25T11:13:22.092 INFO:teuthology.orchestra.run.smithi152.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-04-25T11:13:22.118 INFO:teuthology.orchestra.run.smithi152.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 195 kB/s | 16 kB 00:00 2024-04-25T11:13:22.152 INFO:teuthology.orchestra.run.smithi152.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2024-04-25T11:13:22.194 INFO:teuthology.orchestra.run.smithi152.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 280 kB/s | 28 kB 00:00 2024-04-25T11:13:22.219 INFO:teuthology.orchestra.run.smithi152.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 305 kB/s | 31 kB 00:00 2024-04-25T11:13:22.261 INFO:teuthology.orchestra.run.smithi152.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-04-25T11:13:22.277 INFO:teuthology.orchestra.run.smithi152.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 707 kB/s | 59 kB 00:00 2024-04-25T11:13:22.295 INFO:teuthology.orchestra.run.smithi152.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 473 kB/s | 36 kB 00:00 2024-04-25T11:13:22.328 INFO:teuthology.orchestra.run.smithi152.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 758 kB/s | 51 kB 00:00 2024-04-25T11:13:22.370 INFO:teuthology.orchestra.run.smithi152.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-04-25T11:13:22.396 INFO:teuthology.orchestra.run.smithi152.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 891 kB/s | 90 kB 00:00 2024-04-25T11:13:22.438 INFO:teuthology.orchestra.run.smithi152.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-04-25T11:13:22.463 INFO:teuthology.orchestra.run.smithi152.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 216 kB/s | 20 kB 00:00 2024-04-25T11:13:22.496 INFO:teuthology.orchestra.run.smithi152.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-04-25T11:13:22.630 INFO:teuthology.orchestra.run.smithi152.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 8.2 MB/s | 1.6 MB 00:00 2024-04-25T11:13:22.831 INFO:teuthology.orchestra.run.smithi152.stdout:(149/152): librados2-19.0.0-3291.g994e2f20.el9. 9.1 MB/s | 3.3 MB 00:00 2024-04-25T11:13:23.039 INFO:teuthology.orchestra.run.smithi152.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 3.3 MB/s | 1.3 MB 00:00 2024-04-25T11:13:23.103 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2024-04-25T11:13:23.246 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2024-04-25T11:13:23.247 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2024-04-25T11:13:23.272 INFO:teuthology.orchestra.run.smithi152.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 5.4 MB/s | 2.4 MB 00:00 2024-04-25T11:13:23.489 INFO:teuthology.orchestra.run.smithi152.stdout:(152/152): librbd1-19.0.0-3291.g994e2f20.el9.x8 3.1 MB/s | 3.1 MB 00:00 2024-04-25T11:13:23.495 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2024-04-25T11:13:23.495 INFO:teuthology.orchestra.run.smithi152.stdout:Total 9.6 MB/s | 214 MB 00:22 2024-04-25T11:13:24.591 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2024-04-25T11:13:24.734 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2024-04-25T11:13:24.734 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2024-04-25T11:13:25.026 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2024-04-25T11:13:25.027 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2024-04-25T11:13:26.459 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2024-04-25T11:13:26.459 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2024-04-25T11:13:27.142 INFO:teuthology.orchestra.run.smithi069.stdout: Preparing : 1/1 2024-04-25T11:13:27.245 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T11:13:27.435 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T11:13:27.489 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T11:13:27.539 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T11:13:27.564 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T11:13:27.608 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T11:13:27.940 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T11:13:28.064 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T11:13:28.215 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T11:13:28.240 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T11:13:28.323 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T11:13:28.353 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T11:13:28.502 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T11:13:28.550 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T11:13:28.578 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T11:13:28.683 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T11:13:28.712 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T11:13:28.758 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T11:13:28.780 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T11:13:28.803 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T11:13:29.005 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T11:13:29.038 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T11:13:29.144 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T11:13:29.246 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T11:13:29.287 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T11:13:29.322 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T11:13:29.396 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T11:13:29.446 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T11:13:29.499 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T11:13:29.530 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T11:13:29.559 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T11:13:29.629 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T11:13:29.688 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T11:13:29.719 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T11:13:29.763 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T11:13:29.840 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T11:13:29.874 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 31/156 2024-04-25T11:13:29.917 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 32/156 2024-04-25T11:13:29.946 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 33/156 2024-04-25T11:13:30.023 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T11:13:30.069 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T11:13:30.142 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T11:13:30.277 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T11:13:30.330 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T11:13:30.855 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T11:13:30.934 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T11:13:30.963 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T11:13:31.005 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T11:13:31.044 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 43/156 2024-04-25T11:13:31.085 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T11:13:31.148 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2024-04-25T11:13:31.239 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T11:13:31.242 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T11:13:31.277 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T11:13:31.296 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T11:13:31.331 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T11:13:31.351 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T11:13:31.364 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T11:13:31.390 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T11:13:31.424 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T11:13:31.432 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T11:13:31.463 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T11:13:31.500 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T11:13:31.546 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T11:13:31.583 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T11:13:31.626 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T11:13:31.668 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T11:13:31.768 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T11:13:31.813 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T11:13:31.848 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T11:13:31.881 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T11:13:31.920 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T11:13:31.921 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T11:13:32.014 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T11:13:32.050 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T11:13:32.054 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T11:13:32.094 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T11:13:32.100 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T11:13:32.119 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T11:13:32.144 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T11:13:32.191 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T11:13:32.221 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T11:13:32.243 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T11:13:32.341 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2024-04-25T11:13:32.370 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T11:13:32.371 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T11:13:32.401 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T11:13:32.418 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T11:13:32.426 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T11:13:32.429 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T11:13:32.445 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T11:13:32.463 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T11:13:32.502 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T11:13:32.532 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T11:13:32.534 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T11:13:32.551 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T11:13:32.564 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T11:13:32.580 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T11:13:32.598 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T11:13:32.625 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T11:13:32.632 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T11:13:32.647 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T11:13:32.667 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T11:13:32.673 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T11:13:32.696 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T11:13:32.708 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T11:13:32.737 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T11:13:32.762 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T11:13:32.795 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T11:13:32.806 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T11:13:32.838 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T11:13:32.864 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T11:13:32.893 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T11:13:32.897 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T11:13:32.942 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T11:13:33.011 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T11:13:33.040 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T11:13:33.088 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T11:13:33.102 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T11:13:33.138 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T11:13:33.141 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T11:13:33.173 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T11:13:33.247 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T11:13:33.272 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T11:13:33.297 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T11:13:33.357 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T11:13:33.380 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T11:13:33.405 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T11:13:33.417 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T11:13:33.437 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T11:13:33.506 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T11:13:33.540 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T11:13:33.546 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T11:13:33.607 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T11:13:33.645 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T11:13:33.689 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T11:13:33.690 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T11:13:33.741 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T11:13:33.766 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T11:13:33.769 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T11:13:33.808 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 31/156 2024-04-25T11:13:33.812 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T11:13:33.851 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 32/156 2024-04-25T11:13:33.870 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T11:13:33.873 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T11:13:33.879 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 33/156 2024-04-25T11:13:33.894 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T11:13:33.895 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T11:13:33.941 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T11:13:33.950 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T11:13:33.963 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T11:13:33.988 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T11:13:33.995 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T11:13:34.003 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T11:13:34.052 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T11:13:34.077 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T11:13:34.139 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T11:13:34.187 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T11:13:34.188 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T11:13:34.211 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T11:13:34.221 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T11:13:34.263 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T11:13:34.322 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T11:13:34.365 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T11:13:34.393 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T11:13:34.428 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T11:13:34.463 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T11:13:34.480 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T11:13:34.498 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T11:13:34.572 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T11:13:34.622 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T11:13:34.666 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T11:13:34.699 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T11:13:34.726 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T11:13:34.784 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T11:13:34.788 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T11:13:34.849 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T11:13:34.859 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T11:13:34.879 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T11:13:34.887 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T11:13:34.924 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T11:13:34.930 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T11:13:34.968 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 43/156 2024-04-25T11:13:35.001 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T11:13:35.010 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T11:13:35.034 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 31/156 2024-04-25T11:13:35.078 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 32/156 2024-04-25T11:13:35.106 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 33/156 2024-04-25T11:13:35.116 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T11:13:35.176 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T11:13:35.177 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T11:13:35.230 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T11:13:35.230 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T11:13:35.233 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T11:13:35.285 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T11:13:35.304 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T11:13:35.357 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T11:13:35.388 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T11:13:35.425 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T11:13:35.439 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T11:13:35.472 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T11:13:35.491 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T11:13:35.507 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T11:13:35.550 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T11:13:35.592 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T11:13:35.728 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T11:13:35.773 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T11:13:35.803 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T11:13:35.844 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T11:13:35.938 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T11:13:35.978 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T11:13:36.009 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T11:13:36.013 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T11:13:36.035 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T11:13:36.060 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T11:13:36.095 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T11:13:36.123 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T11:13:36.133 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T11:13:36.166 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T11:13:36.204 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 43/156 2024-04-25T11:13:36.246 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T11:13:36.261 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T11:13:36.300 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T11:13:36.328 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T11:13:36.363 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T11:13:36.397 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T11:13:36.401 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T11:13:36.433 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T11:13:36.459 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T11:13:36.463 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T11:13:36.466 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T11:13:36.489 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T11:13:36.521 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T11:13:36.522 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T11:13:36.530 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T11:13:36.558 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T11:13:36.564 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T11:13:36.591 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T11:13:36.594 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T11:13:36.598 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T11:13:36.625 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T11:13:36.662 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T11:13:36.693 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T11:13:36.709 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T11:13:36.737 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T11:13:36.745 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T11:13:36.768 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T11:13:36.788 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T11:13:36.815 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T11:13:36.829 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T11:13:36.912 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T11:13:36.913 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T11:13:36.969 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T11:13:36.971 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T11:13:36.972 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T11:13:37.027 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T11:13:37.058 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T11:13:37.061 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T11:13:37.092 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 100/156 2024-04-25T11:13:37.099 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T11:13:37.128 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T11:13:37.194 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T11:13:37.224 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T11:13:37.265 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T11:13:37.290 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T11:13:37.315 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T11:13:37.392 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T11:13:37.521 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T11:13:37.559 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T11:13:37.587 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T11:13:37.620 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T11:13:37.621 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T11:13:37.649 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T11:13:37.661 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T11:13:37.693 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T11:13:37.718 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T11:13:37.720 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T11:13:37.748 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T11:13:37.750 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 104/156 2024-04-25T11:13:37.752 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T11:13:37.785 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T11:13:37.790 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T11:13:37.824 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T11:13:37.865 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T11:13:37.884 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T11:13:37.900 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T11:13:37.943 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T11:13:37.987 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T11:13:38.030 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T11:13:38.031 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T11:13:38.070 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T11:13:38.071 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T11:13:38.117 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T11:13:38.216 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T11:13:38.250 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T11:13:38.264 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T11:13:38.285 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T11:13:38.364 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T11:13:38.982 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T11:13:39.003 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T11:13:39.027 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T11:13:39.062 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T11:13:39.124 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T11:13:39.161 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T11:13:39.228 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T11:13:39.316 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T11:13:39.364 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T11:13:39.543 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T11:13:39.570 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T11:13:39.651 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T11:13:40.004 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T11:13:40.047 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T11:13:40.281 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T11:13:40.357 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T11:13:40.411 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T11:13:40.438 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T11:13:40.466 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T11:13:40.499 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T11:13:40.610 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T11:13:40.658 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T11:13:40.744 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T11:13:40.744 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T11:13:40.744 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T11:13:40.744 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:13:40.776 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T11:13:40.820 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T11:13:40.876 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T11:13:40.945 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T11:13:40.976 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 100/156 2024-04-25T11:13:41.012 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T11:13:41.499 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T11:13:41.531 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T11:13:41.603 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T11:13:41.624 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 104/156 2024-04-25T11:13:41.630 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T11:13:41.718 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T11:13:41.746 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T11:13:41.779 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T11:13:42.102 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T11:13:42.159 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T11:13:42.250 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T11:13:42.280 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 100/156 2024-04-25T11:13:42.317 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T11:13:42.804 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T11:13:42.829 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T11:13:42.891 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T11:13:42.914 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 104/156 2024-04-25T11:13:43.906 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T11:13:43.946 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T11:13:44.513 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T11:13:44.558 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T11:13:44.649 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T11:13:44.649 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T11:13:44.649 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T11:13:44.650 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T11:13:44.667 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T11:13:45.154 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T11:13:45.186 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T11:13:45.749 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T11:13:45.790 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T11:13:45.886 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T11:13:45.886 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T11:13:45.886 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T11:13:45.886 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-25T11:13:45.916 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T11:13:52.082 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T11:13:52.082 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /sys 2024-04-25T11:13:52.082 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /proc 2024-04-25T11:13:52.082 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /mnt 2024-04-25T11:13:52.082 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /var/tmp 2024-04-25T11:13:52.082 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /home 2024-04-25T11:13:52.082 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /root 2024-04-25T11:13:52.082 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /tmp 2024-04-25T11:13:52.082 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:13:52.274 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T11:13:52.289 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T11:13:52.290 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:13:52.290 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T11:13:52.290 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:13:53.866 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T11:13:53.891 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T11:13:53.891 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:13:53.891 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T11:13:53.891 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T11:13:53.891 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T11:13:53.891 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:13:54.116 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T11:13:54.139 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T11:13:54.139 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:13:54.139 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T11:13:54.139 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T11:13:54.139 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T11:13:54.140 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:13:54.600 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T11:13:54.622 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T11:13:54.623 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:13:54.623 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T11:13:54.623 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T11:13:54.623 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T11:13:54.623 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:13:54.731 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T11:13:54.787 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T11:13:54.842 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T11:13:55.028 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T11:13:55.076 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T11:13:55.145 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T11:13:55.242 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T11:13:55.382 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T11:13:55.417 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T11:13:55.450 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T11:13:55.477 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T11:13:55.929 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T11:13:55.959 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T11:13:55.991 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 124/156 2024-04-25T11:13:56.052 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 125/156 2024-04-25T11:13:56.091 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T11:13:56.202 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T11:13:56.203 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /sys 2024-04-25T11:13:56.203 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /proc 2024-04-25T11:13:56.203 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /mnt 2024-04-25T11:13:56.203 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /var/tmp 2024-04-25T11:13:56.203 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /home 2024-04-25T11:13:56.203 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /root 2024-04-25T11:13:56.203 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /tmp 2024-04-25T11:13:56.203 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T11:13:56.387 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T11:13:56.402 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T11:13:56.402 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:13:56.402 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T11:13:56.403 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T11:13:57.128 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T11:13:57.159 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T11:13:57.646 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T11:13:57.646 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /sys 2024-04-25T11:13:57.646 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /proc 2024-04-25T11:13:57.646 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /mnt 2024-04-25T11:13:57.647 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /var/tmp 2024-04-25T11:13:57.647 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /home 2024-04-25T11:13:57.647 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /root 2024-04-25T11:13:57.647 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /tmp 2024-04-25T11:13:57.647 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-25T11:13:57.825 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T11:13:57.840 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T11:13:57.840 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:13:57.841 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T11:13:57.841 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-25T11:13:57.965 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T11:13:57.990 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T11:13:57.990 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:13:57.990 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T11:13:57.990 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T11:13:57.991 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T11:13:57.991 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T11:13:58.127 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T11:13:58.159 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T11:13:58.224 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T11:13:58.246 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T11:13:58.246 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:13:58.246 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T11:13:58.246 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T11:13:58.247 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T11:13:58.247 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T11:13:58.261 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T11:13:58.418 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 129/156 2024-04-25T11:13:58.461 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T11:13:58.484 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T11:13:58.484 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:13:58.484 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T11:13:58.484 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T11:13:58.485 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T11:13:58.485 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:13:58.529 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T11:13:58.540 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T11:13:58.638 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 132/156 2024-04-25T11:13:58.693 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 133/156 2024-04-25T11:13:58.708 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T11:13:58.730 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T11:13:58.730 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:13:58.730 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T11:13:58.730 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T11:13:58.730 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T11:13:58.730 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T11:13:58.830 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T11:13:58.886 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T11:13:58.942 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T11:13:59.129 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T11:13:59.175 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T11:13:59.251 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T11:13:59.340 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T11:13:59.419 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T11:13:59.445 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T11:13:59.445 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:13:59.445 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T11:13:59.445 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T11:13:59.445 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T11:13:59.445 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-25T11:13:59.482 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T11:13:59.525 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T11:13:59.557 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T11:13:59.584 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T11:13:59.631 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T11:13:59.664 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T11:13:59.670 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T11:13:59.687 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T11:13:59.687 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:13:59.687 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T11:13:59.687 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T11:13:59.687 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T11:13:59.687 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:13:59.692 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T11:13:59.692 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:13:59.692 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T11:13:59.693 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T11:13:59.693 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T11:13:59.693 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-25T11:13:59.732 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T11:13:59.753 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T11:13:59.753 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:13:59.753 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T11:13:59.753 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:13:59.799 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T11:13:59.821 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T11:13:59.821 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:13:59.822 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T11:13:59.822 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T11:13:59.822 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T11:13:59.822 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:14:00.012 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T11:14:00.050 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T11:14:00.073 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 124/156 2024-04-25T11:14:00.127 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 125/156 2024-04-25T11:14:00.133 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T11:14:00.154 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T11:14:00.155 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T11:14:00.155 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:14:00.155 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T11:14:00.155 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T11:14:00.155 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T11:14:00.155 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:14:00.164 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T11:14:00.177 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T11:14:00.177 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:14:00.177 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T11:14:00.177 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T11:14:00.177 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T11:14:00.177 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-25T11:14:00.284 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T11:14:00.333 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T11:14:00.389 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T11:14:00.574 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T11:14:00.622 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T11:14:00.702 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T11:14:00.789 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T11:14:00.928 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T11:14:00.973 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T11:14:01.004 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T11:14:01.032 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T11:14:01.201 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T11:14:01.241 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T11:14:01.418 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T11:14:01.448 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T11:14:01.480 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 124/156 2024-04-25T11:14:01.542 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 125/156 2024-04-25T11:14:01.571 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T11:14:02.213 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T11:14:02.241 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T11:14:02.343 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T11:14:02.508 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 129/156 2024-04-25T11:14:02.535 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T11:14:02.558 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T11:14:02.558 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:14:02.558 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T11:14:02.558 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T11:14:02.558 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T11:14:02.558 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T11:14:02.602 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T11:14:02.611 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T11:14:02.613 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T11:14:02.648 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T11:14:02.712 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 132/156 2024-04-25T11:14:02.760 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 133/156 2024-04-25T11:14:03.612 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T11:14:03.649 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T11:14:03.699 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T11:14:03.730 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T11:14:03.753 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T11:14:03.754 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T11:14:03.754 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:14:03.754 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T11:14:03.754 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T11:14:03.755 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T11:14:03.755 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T11:14:03.797 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T11:14:03.819 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T11:14:03.819 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:14:03.819 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T11:14:03.820 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T11:14:03.865 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T11:14:03.887 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T11:14:03.887 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:14:03.887 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T11:14:03.887 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T11:14:03.887 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T11:14:03.887 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T11:14:03.932 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 129/156 2024-04-25T11:14:03.959 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T11:14:03.983 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T11:14:03.983 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:14:03.984 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T11:14:03.984 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T11:14:03.984 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T11:14:03.984 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-25T11:14:04.027 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T11:14:04.038 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T11:14:04.137 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 132/156 2024-04-25T11:14:04.183 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 133/156 2024-04-25T11:14:04.199 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T11:14:04.221 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T11:14:04.221 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:14:04.221 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T11:14:04.221 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T11:14:04.221 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T11:14:04.221 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T11:14:04.428 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 139/156 2024-04-25T11:14:04.470 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 140/156 2024-04-25T11:14:04.510 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 141/156 2024-04-25T11:14:04.542 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T11:14:04.570 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T11:14:04.757 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 144/156 2024-04-25T11:14:04.807 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T11:14:04.837 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 146/156 2024-04-25T11:14:05.131 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T11:14:05.178 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T11:14:05.201 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T11:14:05.201 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:14:05.201 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T11:14:05.201 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T11:14:05.201 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T11:14:05.201 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-25T11:14:05.247 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T11:14:05.269 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T11:14:05.269 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:14:05.269 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T11:14:05.269 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-25T11:14:05.314 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T11:14:05.336 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T11:14:05.337 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:14:05.337 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T11:14:05.337 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T11:14:05.337 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T11:14:05.337 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-25T11:14:05.419 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T11:14:05.451 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T11:14:05.471 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T11:14:05.656 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T11:14:05.678 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T11:14:05.679 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:14:05.679 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T11:14:05.679 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T11:14:05.679 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T11:14:05.679 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-25T11:14:06.320 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 148/156 2024-04-25T11:14:06.414 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T11:14:06.455 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T11:14:06.536 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T11:14:06.568 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T11:14:06.568 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T11:14:06.652 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T11:14:06.685 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T11:14:06.685 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T11:14:06.782 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T11:14:06.851 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T11:14:08.486 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 139/156 2024-04-25T11:14:08.527 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 140/156 2024-04-25T11:14:08.567 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 141/156 2024-04-25T11:14:08.600 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T11:14:08.627 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T11:14:08.818 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 144/156 2024-04-25T11:14:08.856 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T11:14:08.886 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 146/156 2024-04-25T11:14:09.500 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T11:14:09.542 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T11:14:09.561 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T11:14:09.961 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 139/156 2024-04-25T11:14:10.017 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 140/156 2024-04-25T11:14:10.051 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 141/156 2024-04-25T11:14:10.083 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T11:14:10.111 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T11:14:10.295 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 144/156 2024-04-25T11:14:10.349 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T11:14:10.379 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 146/156 2024-04-25T11:14:10.397 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 148/156 2024-04-25T11:14:10.480 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T11:14:10.512 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T11:14:10.602 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T11:14:10.651 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T11:14:10.651 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T11:14:10.734 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T11:14:10.767 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T11:14:10.768 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T11:14:10.858 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T11:14:10.924 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T11:14:10.994 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T11:14:11.038 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T11:14:11.057 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T11:14:11.914 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 148/156 2024-04-25T11:14:12.010 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T11:14:12.051 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T11:14:12.133 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T11:14:12.181 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T11:14:12.181 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T11:14:12.249 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T11:14:12.282 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T11:14:12.282 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T11:14:12.385 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T11:14:12.455 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T11:14:12.573 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T11:14:12.574 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 1/156 2024-04-25T11:14:12.574 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2/156 2024-04-25T11:14:12.574 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 3/156 2024-04-25T11:14:12.574 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 4/156 2024-04-25T11:14:12.574 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 5/156 2024-04-25T11:14:12.574 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 6/156 2024-04-25T11:14:12.574 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 7/156 2024-04-25T11:14:12.574 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T11:14:12.574 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T11:14:12.574 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T11:14:12.574 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 11/156 2024-04-25T11:14:12.574 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 12/156 2024-04-25T11:14:12.575 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T11:14:12.575 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 14/156 2024-04-25T11:14:12.575 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 15/156 2024-04-25T11:14:12.575 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 16/156 2024-04-25T11:14:12.575 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 17/156 2024-04-25T11:14:12.575 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 18/156 2024-04-25T11:14:12.575 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 19/156 2024-04-25T11:14:12.575 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 20/156 2024-04-25T11:14:12.575 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 21/156 2024-04-25T11:14:12.575 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 22/156 2024-04-25T11:14:12.575 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 23/156 2024-04-25T11:14:12.575 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 24/156 2024-04-25T11:14:12.575 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 25/156 2024-04-25T11:14:12.576 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 26/156 2024-04-25T11:14:12.576 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 27/156 2024-04-25T11:14:12.576 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 28/156 2024-04-25T11:14:12.576 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 29/156 2024-04-25T11:14:12.576 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 30/156 2024-04-25T11:14:12.576 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 31/156 2024-04-25T11:14:12.576 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 32/156 2024-04-25T11:14:12.576 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 33/156 2024-04-25T11:14:12.576 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 34/156 2024-04-25T11:14:12.576 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 35/156 2024-04-25T11:14:12.576 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 36/156 2024-04-25T11:14:12.576 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 37/156 2024-04-25T11:14:12.576 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 38/156 2024-04-25T11:14:12.576 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 39/156 2024-04-25T11:14:12.577 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T11:14:12.577 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T11:14:12.577 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T11:14:12.577 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T11:14:12.577 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T11:14:12.577 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T11:14:12.577 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T11:14:12.577 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T11:14:12.577 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T11:14:12.578 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T11:14:12.578 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T11:14:12.578 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T11:14:12.578 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T11:14:12.578 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T11:14:12.578 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T11:14:12.578 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T11:14:12.578 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T11:14:12.578 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T11:14:12.578 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T11:14:12.578 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T11:14:12.578 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T11:14:12.578 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T11:14:12.578 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T11:14:12.579 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T11:14:12.579 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T11:14:12.579 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T11:14:12.579 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T11:14:12.579 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T11:14:12.579 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T11:14:12.579 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T11:14:12.579 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T11:14:12.579 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T11:14:12.579 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T11:14:12.579 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T11:14:12.579 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T11:14:12.579 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T11:14:12.579 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T11:14:12.580 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T11:14:12.580 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T11:14:12.580 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T11:14:12.580 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T11:14:12.580 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T11:14:12.580 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T11:14:12.580 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T11:14:12.580 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T11:14:12.580 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T11:14:12.580 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T11:14:12.580 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T11:14:12.580 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T11:14:12.580 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T11:14:12.581 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T11:14:12.581 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T11:14:12.581 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T11:14:12.581 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T11:14:12.581 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T11:14:12.581 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T11:14:12.581 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T11:14:12.581 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T11:14:12.581 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T11:14:12.581 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T11:14:12.581 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T11:14:12.581 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T11:14:12.581 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T11:14:12.582 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T11:14:12.582 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T11:14:12.582 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T11:14:12.582 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T11:14:12.582 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T11:14:12.582 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T11:14:12.582 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T11:14:12.582 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T11:14:12.582 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T11:14:12.582 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T11:14:12.582 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T11:14:12.582 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T11:14:12.582 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T11:14:12.582 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T11:14:12.583 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T11:14:12.583 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T11:14:12.583 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T11:14:12.583 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T11:14:12.583 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T11:14:12.583 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T11:14:12.583 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T11:14:12.583 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T11:14:12.583 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T11:14:12.583 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T11:14:12.583 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T11:14:12.583 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T11:14:12.583 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T11:14:12.583 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T11:14:12.584 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T11:14:12.584 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T11:14:12.584 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T11:14:12.584 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T11:14:12.584 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T11:14:12.584 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T11:14:12.584 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T11:14:12.584 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T11:14:12.584 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T11:14:12.584 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T11:14:12.584 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T11:14:12.584 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T11:14:12.584 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T11:14:12.584 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T11:14:12.584 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T11:14:12.585 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T11:14:12.585 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T11:14:12.585 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T11:14:12.585 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 149/156 2024-04-25T11:14:12.585 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T11:14:12.585 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 151/156 2024-04-25T11:14:12.585 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T11:14:12.585 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T11:14:12.585 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T11:14:12.585 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T11:14:14.647 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T11:14:14.647 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:14:14.647 INFO:teuthology.orchestra.run.smithi069.stdout:Upgraded: 2024-04-25T11:14:14.647 INFO:teuthology.orchestra.run.smithi069.stdout: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:14.647 INFO:teuthology.orchestra.run.smithi069.stdout: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:14.647 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T11:14:14.647 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T11:14:14.647 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2024-04-25T11:14:14.648 INFO:teuthology.orchestra.run.smithi069.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T11:14:14.648 INFO:teuthology.orchestra.run.smithi069.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T11:14:14.648 INFO:teuthology.orchestra.run.smithi069.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T11:14:14.648 INFO:teuthology.orchestra.run.smithi069.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T11:14:14.648 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:14.648 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:14.648 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:14.648 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:14.648 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:14:14.648 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-immutable-object-cache-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:14.648 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:14.648 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:14.648 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:14:14.649 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:14:14.649 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:14:14.649 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:14:14.649 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:14:14.649 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:14.649 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:14.649 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:14:14.649 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:14.649 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:14.649 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:14.649 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:14:14.650 INFO:teuthology.orchestra.run.smithi069.stdout: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:14:14.650 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:14.650 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:14.650 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:14:14.650 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T11:14:14.650 INFO:teuthology.orchestra.run.smithi069.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T11:14:14.650 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T11:14:14.650 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T11:14:14.650 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T11:14:14.650 INFO:teuthology.orchestra.run.smithi069.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T11:14:14.650 INFO:teuthology.orchestra.run.smithi069.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T11:14:14.650 INFO:teuthology.orchestra.run.smithi069.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T11:14:14.650 INFO:teuthology.orchestra.run.smithi069.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T11:14:14.651 INFO:teuthology.orchestra.run.smithi069.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T11:14:14.651 INFO:teuthology.orchestra.run.smithi069.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T11:14:14.651 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:14.651 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:14.651 INFO:teuthology.orchestra.run.smithi069.stdout: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:14.651 INFO:teuthology.orchestra.run.smithi069.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T11:14:14.651 INFO:teuthology.orchestra.run.smithi069.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T11:14:14.651 INFO:teuthology.orchestra.run.smithi069.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T11:14:14.651 INFO:teuthology.orchestra.run.smithi069.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T11:14:14.651 INFO:teuthology.orchestra.run.smithi069.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T11:14:14.652 INFO:teuthology.orchestra.run.smithi069.stdout: librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:14.652 INFO:teuthology.orchestra.run.smithi069.stdout: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:14.652 INFO:teuthology.orchestra.run.smithi069.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T11:14:14.652 INFO:teuthology.orchestra.run.smithi069.stdout: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:14.652 INFO:teuthology.orchestra.run.smithi069.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T11:14:14.652 INFO:teuthology.orchestra.run.smithi069.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T11:14:14.652 INFO:teuthology.orchestra.run.smithi069.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T11:14:14.652 INFO:teuthology.orchestra.run.smithi069.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T11:14:14.652 INFO:teuthology.orchestra.run.smithi069.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T11:14:14.652 INFO:teuthology.orchestra.run.smithi069.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T11:14:14.652 INFO:teuthology.orchestra.run.smithi069.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T11:14:14.652 INFO:teuthology.orchestra.run.smithi069.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T11:14:14.652 INFO:teuthology.orchestra.run.smithi069.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T11:14:14.653 INFO:teuthology.orchestra.run.smithi069.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T11:14:14.653 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T11:14:14.653 INFO:teuthology.orchestra.run.smithi069.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T11:14:14.653 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T11:14:14.653 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T11:14:14.653 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T11:14:14.653 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T11:14:14.653 INFO:teuthology.orchestra.run.smithi069.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T11:14:14.653 INFO:teuthology.orchestra.run.smithi069.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T11:14:14.653 INFO:teuthology.orchestra.run.smithi069.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T11:14:14.654 INFO:teuthology.orchestra.run.smithi069.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T11:14:14.654 INFO:teuthology.orchestra.run.smithi069.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T11:14:14.654 INFO:teuthology.orchestra.run.smithi069.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T11:14:14.654 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T11:14:14.654 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:14.654 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:14.654 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:14.654 INFO:teuthology.orchestra.run.smithi069.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T11:14:14.654 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T11:14:14.654 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T11:14:14.655 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T11:14:14.655 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T11:14:14.655 INFO:teuthology.orchestra.run.smithi069.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T11:14:14.655 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T11:14:14.655 INFO:teuthology.orchestra.run.smithi069.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T11:14:14.655 INFO:teuthology.orchestra.run.smithi069.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T11:14:14.655 INFO:teuthology.orchestra.run.smithi069.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T11:14:14.655 INFO:teuthology.orchestra.run.smithi069.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T11:14:14.655 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T11:14:14.655 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T11:14:14.655 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T11:14:14.656 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T11:14:14.656 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T11:14:14.656 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T11:14:14.656 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T11:14:14.656 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T11:14:14.656 INFO:teuthology.orchestra.run.smithi069.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T11:14:14.656 INFO:teuthology.orchestra.run.smithi069.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T11:14:14.656 INFO:teuthology.orchestra.run.smithi069.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T11:14:14.656 INFO:teuthology.orchestra.run.smithi069.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T11:14:14.656 INFO:teuthology.orchestra.run.smithi069.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T11:14:14.657 INFO:teuthology.orchestra.run.smithi069.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T11:14:14.657 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T11:14:14.657 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T11:14:14.657 INFO:teuthology.orchestra.run.smithi069.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T11:14:14.657 INFO:teuthology.orchestra.run.smithi069.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T11:14:14.657 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T11:14:14.657 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T11:14:14.657 INFO:teuthology.orchestra.run.smithi069.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T11:14:14.657 INFO:teuthology.orchestra.run.smithi069.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T11:14:14.657 INFO:teuthology.orchestra.run.smithi069.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T11:14:14.658 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T11:14:14.658 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T11:14:14.658 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T11:14:14.658 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T11:14:14.658 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T11:14:14.658 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T11:14:14.658 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:14.658 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:14.658 INFO:teuthology.orchestra.run.smithi069.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T11:14:14.658 INFO:teuthology.orchestra.run.smithi069.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T11:14:14.658 INFO:teuthology.orchestra.run.smithi069.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T11:14:14.658 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:14.658 INFO:teuthology.orchestra.run.smithi069.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T11:14:14.659 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T11:14:14.659 INFO:teuthology.orchestra.run.smithi069.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T11:14:14.659 INFO:teuthology.orchestra.run.smithi069.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T11:14:14.659 INFO:teuthology.orchestra.run.smithi069.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T11:14:14.659 INFO:teuthology.orchestra.run.smithi069.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T11:14:14.659 INFO:teuthology.orchestra.run.smithi069.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T11:14:14.659 INFO:teuthology.orchestra.run.smithi069.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T11:14:14.659 INFO:teuthology.orchestra.run.smithi069.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T11:14:14.659 INFO:teuthology.orchestra.run.smithi069.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T11:14:14.659 INFO:teuthology.orchestra.run.smithi069.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T11:14:14.659 INFO:teuthology.orchestra.run.smithi069.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T11:14:14.659 INFO:teuthology.orchestra.run.smithi069.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T11:14:14.659 INFO:teuthology.orchestra.run.smithi069.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T11:14:14.659 INFO:teuthology.orchestra.run.smithi069.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T11:14:14.660 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:14.660 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:14.660 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:14.660 INFO:teuthology.orchestra.run.smithi069.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T11:14:14.660 INFO:teuthology.orchestra.run.smithi069.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T11:14:14.660 INFO:teuthology.orchestra.run.smithi069.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T11:14:14.660 INFO:teuthology.orchestra.run.smithi069.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T11:14:14.660 INFO:teuthology.orchestra.run.smithi069.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T11:14:14.660 INFO:teuthology.orchestra.run.smithi069.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T11:14:14.661 INFO:teuthology.orchestra.run.smithi069.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T11:14:14.661 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:14:14.661 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2024-04-25T11:14:15.042 DEBUG:teuthology.parallel:result is None 2024-04-25T11:14:16.903 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T11:14:16.903 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 1/156 2024-04-25T11:14:16.903 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2/156 2024-04-25T11:14:16.903 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 3/156 2024-04-25T11:14:16.903 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 4/156 2024-04-25T11:14:16.903 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 5/156 2024-04-25T11:14:16.904 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 6/156 2024-04-25T11:14:16.904 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 7/156 2024-04-25T11:14:16.904 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T11:14:16.904 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T11:14:16.904 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T11:14:16.904 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 11/156 2024-04-25T11:14:16.904 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 12/156 2024-04-25T11:14:16.904 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T11:14:16.904 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 14/156 2024-04-25T11:14:16.904 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 15/156 2024-04-25T11:14:16.904 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 16/156 2024-04-25T11:14:16.904 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 17/156 2024-04-25T11:14:16.904 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 18/156 2024-04-25T11:14:16.904 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 19/156 2024-04-25T11:14:16.905 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 20/156 2024-04-25T11:14:16.905 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 21/156 2024-04-25T11:14:16.905 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 22/156 2024-04-25T11:14:16.905 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 23/156 2024-04-25T11:14:16.905 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 24/156 2024-04-25T11:14:16.905 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 25/156 2024-04-25T11:14:16.905 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 26/156 2024-04-25T11:14:16.905 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 27/156 2024-04-25T11:14:16.905 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 28/156 2024-04-25T11:14:16.905 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 29/156 2024-04-25T11:14:16.905 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 30/156 2024-04-25T11:14:16.905 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 31/156 2024-04-25T11:14:16.905 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 32/156 2024-04-25T11:14:16.905 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 33/156 2024-04-25T11:14:16.906 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 34/156 2024-04-25T11:14:16.906 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 35/156 2024-04-25T11:14:16.906 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 36/156 2024-04-25T11:14:16.906 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 37/156 2024-04-25T11:14:16.906 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 38/156 2024-04-25T11:14:16.906 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 39/156 2024-04-25T11:14:16.906 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T11:14:16.906 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T11:14:16.906 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T11:14:16.906 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T11:14:16.906 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T11:14:16.906 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T11:14:16.906 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T11:14:16.906 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T11:14:16.907 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T11:14:16.907 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T11:14:16.907 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T11:14:16.907 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T11:14:16.907 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T11:14:16.907 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T11:14:16.907 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T11:14:16.907 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T11:14:16.907 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T11:14:16.907 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T11:14:16.907 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T11:14:16.907 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T11:14:16.907 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T11:14:16.907 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T11:14:16.908 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T11:14:16.908 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T11:14:16.908 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T11:14:16.908 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T11:14:16.908 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T11:14:16.908 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T11:14:16.908 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T11:14:16.908 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T11:14:16.908 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T11:14:16.908 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T11:14:16.908 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T11:14:16.908 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T11:14:16.908 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T11:14:16.908 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T11:14:16.909 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T11:14:16.909 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T11:14:16.909 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T11:14:16.909 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T11:14:16.909 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T11:14:16.909 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T11:14:16.909 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T11:14:16.909 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T11:14:16.909 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T11:14:16.909 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T11:14:16.909 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T11:14:16.909 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T11:14:16.909 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T11:14:16.909 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T11:14:16.910 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T11:14:16.910 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T11:14:16.910 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T11:14:16.910 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T11:14:16.910 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T11:14:16.910 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T11:14:16.910 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T11:14:16.910 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T11:14:16.910 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T11:14:16.910 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T11:14:16.910 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T11:14:16.910 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T11:14:16.910 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T11:14:16.910 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T11:14:16.911 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T11:14:16.911 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T11:14:16.911 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T11:14:16.911 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T11:14:16.911 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T11:14:16.911 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T11:14:16.911 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T11:14:16.911 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T11:14:16.911 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T11:14:16.911 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T11:14:16.911 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T11:14:16.911 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T11:14:16.911 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T11:14:16.911 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T11:14:16.912 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T11:14:16.912 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T11:14:16.912 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T11:14:16.912 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T11:14:16.912 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T11:14:16.912 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T11:14:16.912 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T11:14:16.912 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T11:14:16.912 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T11:14:16.912 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T11:14:16.912 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T11:14:16.912 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T11:14:16.912 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T11:14:16.912 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T11:14:16.913 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T11:14:16.913 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T11:14:16.913 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T11:14:16.913 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T11:14:16.913 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T11:14:16.913 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T11:14:16.913 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T11:14:16.913 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T11:14:16.913 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T11:14:16.913 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T11:14:16.913 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T11:14:16.913 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T11:14:16.913 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T11:14:16.913 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T11:14:16.913 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T11:14:16.914 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T11:14:16.914 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T11:14:16.914 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 149/156 2024-04-25T11:14:16.914 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T11:14:16.914 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 151/156 2024-04-25T11:14:16.914 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T11:14:16.914 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T11:14:16.914 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T11:14:16.914 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T11:14:18.530 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T11:14:18.530 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 1/156 2024-04-25T11:14:18.530 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2/156 2024-04-25T11:14:18.531 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 3/156 2024-04-25T11:14:18.531 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 4/156 2024-04-25T11:14:18.531 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 5/156 2024-04-25T11:14:18.531 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 6/156 2024-04-25T11:14:18.531 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 7/156 2024-04-25T11:14:18.531 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T11:14:18.531 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T11:14:18.531 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T11:14:18.531 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 11/156 2024-04-25T11:14:18.531 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 12/156 2024-04-25T11:14:18.531 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T11:14:18.531 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 14/156 2024-04-25T11:14:18.531 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 15/156 2024-04-25T11:14:18.531 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 16/156 2024-04-25T11:14:18.532 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 17/156 2024-04-25T11:14:18.532 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 18/156 2024-04-25T11:14:18.532 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 19/156 2024-04-25T11:14:18.532 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 20/156 2024-04-25T11:14:18.532 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 21/156 2024-04-25T11:14:18.532 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 22/156 2024-04-25T11:14:18.532 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 23/156 2024-04-25T11:14:18.532 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 24/156 2024-04-25T11:14:18.532 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 25/156 2024-04-25T11:14:18.532 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 26/156 2024-04-25T11:14:18.532 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 27/156 2024-04-25T11:14:18.532 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 28/156 2024-04-25T11:14:18.544 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 29/156 2024-04-25T11:14:18.544 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 30/156 2024-04-25T11:14:18.545 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 31/156 2024-04-25T11:14:18.545 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 32/156 2024-04-25T11:14:18.545 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 33/156 2024-04-25T11:14:18.545 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 34/156 2024-04-25T11:14:18.545 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 35/156 2024-04-25T11:14:18.545 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 36/156 2024-04-25T11:14:18.545 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 37/156 2024-04-25T11:14:18.545 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 38/156 2024-04-25T11:14:18.545 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 39/156 2024-04-25T11:14:18.545 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T11:14:18.545 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T11:14:18.546 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T11:14:18.546 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T11:14:18.546 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T11:14:18.546 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T11:14:18.546 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T11:14:18.546 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T11:14:18.546 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T11:14:18.546 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T11:14:18.546 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T11:14:18.546 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T11:14:18.546 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T11:14:18.546 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T11:14:18.547 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T11:14:18.547 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T11:14:18.547 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T11:14:18.547 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T11:14:18.547 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T11:14:18.547 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T11:14:18.547 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T11:14:18.547 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T11:14:18.547 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T11:14:18.547 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T11:14:18.547 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T11:14:18.548 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T11:14:18.548 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T11:14:18.548 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T11:14:18.548 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T11:14:18.548 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T11:14:18.548 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T11:14:18.548 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T11:14:18.548 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T11:14:18.548 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T11:14:18.548 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T11:14:18.548 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T11:14:18.548 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T11:14:18.549 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T11:14:18.549 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T11:14:18.549 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T11:14:18.549 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T11:14:18.549 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T11:14:18.549 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T11:14:18.549 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T11:14:18.549 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T11:14:18.549 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T11:14:18.549 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T11:14:18.549 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T11:14:18.549 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T11:14:18.550 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T11:14:18.550 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T11:14:18.550 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T11:14:18.550 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T11:14:18.550 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T11:14:18.550 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T11:14:18.550 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T11:14:18.552 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T11:14:18.552 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T11:14:18.552 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T11:14:18.552 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T11:14:18.552 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T11:14:18.552 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T11:14:18.552 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T11:14:18.552 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T11:14:18.552 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T11:14:18.552 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T11:14:18.552 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T11:14:18.553 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T11:14:18.553 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T11:14:18.553 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T11:14:18.553 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T11:14:18.553 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T11:14:18.553 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T11:14:18.553 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T11:14:18.553 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T11:14:18.553 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T11:14:18.553 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T11:14:18.554 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T11:14:18.554 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T11:14:18.554 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T11:14:18.554 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T11:14:18.554 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T11:14:18.554 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T11:14:18.554 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T11:14:18.554 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T11:14:18.554 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T11:14:18.554 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T11:14:18.554 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T11:14:18.554 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T11:14:18.555 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T11:14:18.555 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T11:14:18.555 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T11:14:18.555 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T11:14:18.555 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T11:14:18.555 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T11:14:18.555 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T11:14:18.555 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T11:14:18.555 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T11:14:18.555 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T11:14:18.555 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T11:14:18.555 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T11:14:18.555 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T11:14:18.555 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T11:14:18.555 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T11:14:18.556 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T11:14:18.556 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T11:14:18.556 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T11:14:18.556 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T11:14:18.556 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T11:14:18.556 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 149/156 2024-04-25T11:14:18.556 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T11:14:18.556 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 151/156 2024-04-25T11:14:18.556 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T11:14:18.556 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T11:14:18.557 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T11:14:18.557 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T11:14:19.047 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T11:14:19.047 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T11:14:19.048 INFO:teuthology.orchestra.run.smithi146.stdout:Upgraded: 2024-04-25T11:14:19.048 INFO:teuthology.orchestra.run.smithi146.stdout: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:19.048 INFO:teuthology.orchestra.run.smithi146.stdout: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:19.048 INFO:teuthology.orchestra.run.smithi146.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T11:14:19.048 INFO:teuthology.orchestra.run.smithi146.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T11:14:19.048 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2024-04-25T11:14:19.048 INFO:teuthology.orchestra.run.smithi146.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T11:14:19.048 INFO:teuthology.orchestra.run.smithi146.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T11:14:19.048 INFO:teuthology.orchestra.run.smithi146.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T11:14:19.048 INFO:teuthology.orchestra.run.smithi146.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T11:14:19.048 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:19.048 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:19.048 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:19.049 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:19.049 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:14:19.049 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-immutable-object-cache-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:19.049 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:19.049 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:19.049 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:14:19.049 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:14:19.049 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:14:19.050 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:14:19.050 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:14:19.050 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:19.050 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:19.050 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:14:19.050 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:19.050 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:19.050 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:19.051 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:14:19.051 INFO:teuthology.orchestra.run.smithi146.stdout: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:14:19.051 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:19.051 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:19.051 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:14:19.052 INFO:teuthology.orchestra.run.smithi146.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T11:14:19.052 INFO:teuthology.orchestra.run.smithi146.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T11:14:19.052 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T11:14:19.052 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T11:14:19.052 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T11:14:19.052 INFO:teuthology.orchestra.run.smithi146.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T11:14:19.053 INFO:teuthology.orchestra.run.smithi146.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T11:14:19.053 INFO:teuthology.orchestra.run.smithi146.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T11:14:19.053 INFO:teuthology.orchestra.run.smithi146.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T11:14:19.053 INFO:teuthology.orchestra.run.smithi146.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T11:14:19.053 INFO:teuthology.orchestra.run.smithi146.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T11:14:19.054 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:19.054 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:19.054 INFO:teuthology.orchestra.run.smithi146.stdout: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:19.054 INFO:teuthology.orchestra.run.smithi146.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T11:14:19.054 INFO:teuthology.orchestra.run.smithi146.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T11:14:19.054 INFO:teuthology.orchestra.run.smithi146.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T11:14:19.055 INFO:teuthology.orchestra.run.smithi146.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T11:14:19.055 INFO:teuthology.orchestra.run.smithi146.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T11:14:19.055 INFO:teuthology.orchestra.run.smithi146.stdout: librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:19.055 INFO:teuthology.orchestra.run.smithi146.stdout: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:19.055 INFO:teuthology.orchestra.run.smithi146.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T11:14:19.055 INFO:teuthology.orchestra.run.smithi146.stdout: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:19.055 INFO:teuthology.orchestra.run.smithi146.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T11:14:19.055 INFO:teuthology.orchestra.run.smithi146.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T11:14:19.055 INFO:teuthology.orchestra.run.smithi146.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T11:14:19.055 INFO:teuthology.orchestra.run.smithi146.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T11:14:19.055 INFO:teuthology.orchestra.run.smithi146.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T11:14:19.055 INFO:teuthology.orchestra.run.smithi146.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T11:14:19.055 INFO:teuthology.orchestra.run.smithi146.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T11:14:19.056 INFO:teuthology.orchestra.run.smithi146.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T11:14:19.056 INFO:teuthology.orchestra.run.smithi146.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T11:14:19.056 INFO:teuthology.orchestra.run.smithi146.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T11:14:19.056 INFO:teuthology.orchestra.run.smithi146.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T11:14:19.056 INFO:teuthology.orchestra.run.smithi146.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T11:14:19.056 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T11:14:19.056 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T11:14:19.056 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T11:14:19.056 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T11:14:19.056 INFO:teuthology.orchestra.run.smithi146.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T11:14:19.056 INFO:teuthology.orchestra.run.smithi146.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T11:14:19.057 INFO:teuthology.orchestra.run.smithi146.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T11:14:19.057 INFO:teuthology.orchestra.run.smithi146.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T11:14:19.057 INFO:teuthology.orchestra.run.smithi146.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T11:14:19.057 INFO:teuthology.orchestra.run.smithi146.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T11:14:19.057 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T11:14:19.057 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:19.057 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:19.057 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:19.057 INFO:teuthology.orchestra.run.smithi146.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T11:14:19.057 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T11:14:19.057 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T11:14:19.057 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T11:14:19.058 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T11:14:19.058 INFO:teuthology.orchestra.run.smithi146.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T11:14:19.058 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T11:14:19.058 INFO:teuthology.orchestra.run.smithi146.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T11:14:19.058 INFO:teuthology.orchestra.run.smithi146.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T11:14:19.058 INFO:teuthology.orchestra.run.smithi146.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T11:14:19.058 INFO:teuthology.orchestra.run.smithi146.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T11:14:19.058 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T11:14:19.058 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T11:14:19.058 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T11:14:19.058 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T11:14:19.058 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T11:14:19.058 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T11:14:19.058 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T11:14:19.059 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T11:14:19.059 INFO:teuthology.orchestra.run.smithi146.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T11:14:19.059 INFO:teuthology.orchestra.run.smithi146.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T11:14:19.059 INFO:teuthology.orchestra.run.smithi146.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T11:14:19.059 INFO:teuthology.orchestra.run.smithi146.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T11:14:19.059 INFO:teuthology.orchestra.run.smithi146.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T11:14:19.059 INFO:teuthology.orchestra.run.smithi146.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T11:14:19.059 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T11:14:19.059 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T11:14:19.059 INFO:teuthology.orchestra.run.smithi146.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T11:14:19.059 INFO:teuthology.orchestra.run.smithi146.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T11:14:19.059 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T11:14:19.060 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T11:14:19.060 INFO:teuthology.orchestra.run.smithi146.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T11:14:19.060 INFO:teuthology.orchestra.run.smithi146.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T11:14:19.060 INFO:teuthology.orchestra.run.smithi146.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T11:14:19.060 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T11:14:19.060 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T11:14:19.060 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T11:14:19.060 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T11:14:19.061 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T11:14:19.061 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T11:14:19.061 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:19.061 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:19.061 INFO:teuthology.orchestra.run.smithi146.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T11:14:19.061 INFO:teuthology.orchestra.run.smithi146.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T11:14:19.061 INFO:teuthology.orchestra.run.smithi146.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T11:14:19.061 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:19.061 INFO:teuthology.orchestra.run.smithi146.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T11:14:19.061 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T11:14:19.061 INFO:teuthology.orchestra.run.smithi146.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T11:14:19.061 INFO:teuthology.orchestra.run.smithi146.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T11:14:19.061 INFO:teuthology.orchestra.run.smithi146.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T11:14:19.061 INFO:teuthology.orchestra.run.smithi146.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T11:14:19.062 INFO:teuthology.orchestra.run.smithi146.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T11:14:19.062 INFO:teuthology.orchestra.run.smithi146.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T11:14:19.062 INFO:teuthology.orchestra.run.smithi146.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T11:14:19.062 INFO:teuthology.orchestra.run.smithi146.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T11:14:19.062 INFO:teuthology.orchestra.run.smithi146.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T11:14:19.062 INFO:teuthology.orchestra.run.smithi146.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T11:14:19.062 INFO:teuthology.orchestra.run.smithi146.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T11:14:19.062 INFO:teuthology.orchestra.run.smithi146.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T11:14:19.062 INFO:teuthology.orchestra.run.smithi146.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T11:14:19.062 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:19.062 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:19.063 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:19.063 INFO:teuthology.orchestra.run.smithi146.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T11:14:19.063 INFO:teuthology.orchestra.run.smithi146.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T11:14:19.063 INFO:teuthology.orchestra.run.smithi146.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T11:14:19.063 INFO:teuthology.orchestra.run.smithi146.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T11:14:19.063 INFO:teuthology.orchestra.run.smithi146.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T11:14:19.063 INFO:teuthology.orchestra.run.smithi146.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T11:14:19.064 INFO:teuthology.orchestra.run.smithi146.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T11:14:19.064 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-04-25T11:14:19.064 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-04-25T11:14:19.442 DEBUG:teuthology.parallel:result is None 2024-04-25T11:14:20.572 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T11:14:20.572 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-25T11:14:20.572 INFO:teuthology.orchestra.run.smithi152.stdout:Upgraded: 2024-04-25T11:14:20.572 INFO:teuthology.orchestra.run.smithi152.stdout: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:20.572 INFO:teuthology.orchestra.run.smithi152.stdout: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:20.572 INFO:teuthology.orchestra.run.smithi152.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T11:14:20.572 INFO:teuthology.orchestra.run.smithi152.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T11:14:20.572 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2024-04-25T11:14:20.572 INFO:teuthology.orchestra.run.smithi152.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T11:14:20.573 INFO:teuthology.orchestra.run.smithi152.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T11:14:20.573 INFO:teuthology.orchestra.run.smithi152.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T11:14:20.573 INFO:teuthology.orchestra.run.smithi152.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T11:14:20.573 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:20.573 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:20.573 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:20.573 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:20.573 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:14:20.573 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-immutable-object-cache-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:20.573 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:20.573 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:20.573 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:14:20.573 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:14:20.573 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:14:20.574 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:14:20.574 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:14:20.574 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:20.574 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:20.574 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:14:20.574 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:20.574 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:20.574 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:20.574 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:14:20.574 INFO:teuthology.orchestra.run.smithi152.stdout: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:14:20.574 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:20.574 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:20.575 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:14:20.575 INFO:teuthology.orchestra.run.smithi152.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T11:14:20.575 INFO:teuthology.orchestra.run.smithi152.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T11:14:20.575 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T11:14:20.575 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T11:14:20.575 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T11:14:20.575 INFO:teuthology.orchestra.run.smithi152.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T11:14:20.575 INFO:teuthology.orchestra.run.smithi152.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T11:14:20.575 INFO:teuthology.orchestra.run.smithi152.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T11:14:20.575 INFO:teuthology.orchestra.run.smithi152.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T11:14:20.575 INFO:teuthology.orchestra.run.smithi152.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T11:14:20.575 INFO:teuthology.orchestra.run.smithi152.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T11:14:20.575 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:20.575 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:20.576 INFO:teuthology.orchestra.run.smithi152.stdout: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:20.576 INFO:teuthology.orchestra.run.smithi152.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T11:14:20.576 INFO:teuthology.orchestra.run.smithi152.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T11:14:20.576 INFO:teuthology.orchestra.run.smithi152.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T11:14:20.576 INFO:teuthology.orchestra.run.smithi152.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T11:14:20.576 INFO:teuthology.orchestra.run.smithi152.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T11:14:20.576 INFO:teuthology.orchestra.run.smithi152.stdout: librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:20.577 INFO:teuthology.orchestra.run.smithi152.stdout: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:20.577 INFO:teuthology.orchestra.run.smithi152.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T11:14:20.577 INFO:teuthology.orchestra.run.smithi152.stdout: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:20.577 INFO:teuthology.orchestra.run.smithi152.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T11:14:20.577 INFO:teuthology.orchestra.run.smithi152.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T11:14:20.577 INFO:teuthology.orchestra.run.smithi152.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T11:14:20.577 INFO:teuthology.orchestra.run.smithi152.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T11:14:20.577 INFO:teuthology.orchestra.run.smithi152.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T11:14:20.577 INFO:teuthology.orchestra.run.smithi152.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T11:14:20.577 INFO:teuthology.orchestra.run.smithi152.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T11:14:20.577 INFO:teuthology.orchestra.run.smithi152.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T11:14:20.577 INFO:teuthology.orchestra.run.smithi152.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T11:14:20.577 INFO:teuthology.orchestra.run.smithi152.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T11:14:20.578 INFO:teuthology.orchestra.run.smithi152.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T11:14:20.578 INFO:teuthology.orchestra.run.smithi152.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T11:14:20.578 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T11:14:20.578 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T11:14:20.578 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T11:14:20.578 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T11:14:20.578 INFO:teuthology.orchestra.run.smithi152.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T11:14:20.578 INFO:teuthology.orchestra.run.smithi152.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T11:14:20.578 INFO:teuthology.orchestra.run.smithi152.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T11:14:20.578 INFO:teuthology.orchestra.run.smithi152.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T11:14:20.578 INFO:teuthology.orchestra.run.smithi152.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T11:14:20.578 INFO:teuthology.orchestra.run.smithi152.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T11:14:20.578 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T11:14:20.578 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:20.579 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:20.579 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:20.579 INFO:teuthology.orchestra.run.smithi152.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T11:14:20.579 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T11:14:20.579 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T11:14:20.579 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T11:14:20.579 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T11:14:20.579 INFO:teuthology.orchestra.run.smithi152.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T11:14:20.579 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T11:14:20.579 INFO:teuthology.orchestra.run.smithi152.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T11:14:20.579 INFO:teuthology.orchestra.run.smithi152.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T11:14:20.579 INFO:teuthology.orchestra.run.smithi152.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T11:14:20.579 INFO:teuthology.orchestra.run.smithi152.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T11:14:20.579 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T11:14:20.580 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T11:14:20.580 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T11:14:20.580 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T11:14:20.580 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T11:14:20.580 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T11:14:20.580 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T11:14:20.580 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T11:14:20.580 INFO:teuthology.orchestra.run.smithi152.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T11:14:20.580 INFO:teuthology.orchestra.run.smithi152.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T11:14:20.580 INFO:teuthology.orchestra.run.smithi152.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T11:14:20.580 INFO:teuthology.orchestra.run.smithi152.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T11:14:20.580 INFO:teuthology.orchestra.run.smithi152.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T11:14:20.580 INFO:teuthology.orchestra.run.smithi152.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T11:14:20.581 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T11:14:20.581 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T11:14:20.581 INFO:teuthology.orchestra.run.smithi152.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T11:14:20.581 INFO:teuthology.orchestra.run.smithi152.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T11:14:20.581 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T11:14:20.581 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T11:14:20.581 INFO:teuthology.orchestra.run.smithi152.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T11:14:20.581 INFO:teuthology.orchestra.run.smithi152.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T11:14:20.581 INFO:teuthology.orchestra.run.smithi152.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T11:14:20.581 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T11:14:20.581 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T11:14:20.581 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T11:14:20.581 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T11:14:20.581 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T11:14:20.582 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T11:14:20.582 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:20.582 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:20.582 INFO:teuthology.orchestra.run.smithi152.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T11:14:20.582 INFO:teuthology.orchestra.run.smithi152.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T11:14:20.582 INFO:teuthology.orchestra.run.smithi152.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T11:14:20.582 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:20.582 INFO:teuthology.orchestra.run.smithi152.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T11:14:20.582 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T11:14:20.582 INFO:teuthology.orchestra.run.smithi152.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T11:14:20.582 INFO:teuthology.orchestra.run.smithi152.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T11:14:20.582 INFO:teuthology.orchestra.run.smithi152.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T11:14:20.582 INFO:teuthology.orchestra.run.smithi152.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T11:14:20.582 INFO:teuthology.orchestra.run.smithi152.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T11:14:20.583 INFO:teuthology.orchestra.run.smithi152.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T11:14:20.583 INFO:teuthology.orchestra.run.smithi152.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T11:14:20.583 INFO:teuthology.orchestra.run.smithi152.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T11:14:20.583 INFO:teuthology.orchestra.run.smithi152.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T11:14:20.583 INFO:teuthology.orchestra.run.smithi152.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T11:14:20.583 INFO:teuthology.orchestra.run.smithi152.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T11:14:20.583 INFO:teuthology.orchestra.run.smithi152.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T11:14:20.583 INFO:teuthology.orchestra.run.smithi152.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T11:14:20.583 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:20.583 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:20.583 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:14:20.583 INFO:teuthology.orchestra.run.smithi152.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T11:14:20.583 INFO:teuthology.orchestra.run.smithi152.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T11:14:20.583 INFO:teuthology.orchestra.run.smithi152.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T11:14:20.583 INFO:teuthology.orchestra.run.smithi152.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T11:14:20.584 INFO:teuthology.orchestra.run.smithi152.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T11:14:20.584 INFO:teuthology.orchestra.run.smithi152.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T11:14:20.584 INFO:teuthology.orchestra.run.smithi152.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T11:14:20.584 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-25T11:14:20.584 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-04-25T11:14:20.931 DEBUG:teuthology.parallel:result is None 2024-04-25T11:14:20.932 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T11:14:21.189 DEBUG:teuthology.orchestra.run.smithi069:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T11:14:21.214 INFO:teuthology.orchestra.run.smithi069.stdout:19.0.0-3291.g994e2f20.el9 2024-04-25T11:14:21.214 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291.g994e2f20.el9 2024-04-25T11:14:21.215 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291.g994e2f20 is installed. 2024-04-25T11:14:21.217 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T11:14:21.353 DEBUG:teuthology.orchestra.run.smithi146:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T11:14:21.375 INFO:teuthology.orchestra.run.smithi146.stdout:19.0.0-3291.g994e2f20.el9 2024-04-25T11:14:21.375 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291.g994e2f20.el9 2024-04-25T11:14:21.375 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291.g994e2f20 is installed. 2024-04-25T11:14:21.377 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T11:14:21.633 DEBUG:teuthology.orchestra.run.smithi152:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T11:14:21.654 INFO:teuthology.orchestra.run.smithi152.stdout:19.0.0-3291.g994e2f20.el9 2024-04-25T11:14:21.655 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291.g994e2f20.el9 2024-04-25T11:14:21.655 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291.g994e2f20 is installed. 2024-04-25T11:14:21.657 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T11:14:21.657 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T11:14:21.657 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T11:14:21.688 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T11:14:21.688 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T11:14:21.715 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-25T11:14:21.715 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T11:14:21.743 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T11:14:21.743 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T11:14:21.744 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T11:14:21.772 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T11:14:21.838 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T11:14:21.839 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T11:14:21.863 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T11:14:21.926 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-25T11:14:21.927 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T11:14:21.952 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T11:14:22.015 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T11:14:22.016 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T11:14:22.016 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T11:14:22.041 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T11:14:22.107 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T11:14:22.108 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T11:14:22.131 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T11:14:22.194 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-25T11:14:22.194 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T11:14:22.218 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T11:14:22.283 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T11:14:22.283 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T11:14:22.283 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T11:14:22.309 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T11:14:22.375 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T11:14:22.375 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T11:14:22.399 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T11:14:22.463 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-25T11:14:22.463 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T11:14:22.487 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T11:14:22.550 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-25T11:14:22.657 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'subvols': {'create': 2}} 2024-04-25T11:14:22.658 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T11:14:22.658 INFO:tasks.cephadm:Cluster fsid is f7ed73dc-02f4-11ef-bc93-c7b262605968 2024-04-25T11:14:22.658 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-25T11:14:22.658 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.69', 'mon.b': '172.21.15.146', 'mon.c': '172.21.15.152'} 2024-04-25T11:14:22.659 INFO:tasks.cephadm:First mon is mon.a on smithi069 2024-04-25T11:14:22.659 INFO:tasks.cephadm:First mgr is x 2024-04-25T11:14:22.659 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-25T11:14:22.659 DEBUG:teuthology.orchestra.run.smithi069:> sudo hostname $(hostname -s) 2024-04-25T11:14:22.685 DEBUG:teuthology.orchestra.run.smithi146:> sudo hostname $(hostname -s) 2024-04-25T11:14:22.708 DEBUG:teuthology.orchestra.run.smithi152:> sudo hostname $(hostname -s) 2024-04-25T11:14:22.732 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-25T11:14:22.732 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T11:14:22.982 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78872/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3291-g994e2f20', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3291.g994e2f20'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-25 04:02:18.990013', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240425.015853-debug', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-04-25T11:14:23.214 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-pdonnell-testing-20240425.015853-debug, sha1 994e2f20102a20fa9a27966c14b625cc01884df8 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T11:14:23.215 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/x86_64/flavors/default/cephadm 2024-04-25T11:14:23.216 DEBUG:teuthology.orchestra.run.smithi069:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T11:14:23.842 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-r--r--. 1 ubuntu ubuntu 781647 Apr 25 11:14 /home/ubuntu/cephtest/cephadm 2024-04-25T11:14:23.842 DEBUG:teuthology.orchestra.run.smithi146:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T11:14:24.244 INFO:teuthology.orchestra.run.smithi146.stdout:-rw-r--r--. 1 ubuntu ubuntu 781647 Apr 25 11:14 /home/ubuntu/cephtest/cephadm 2024-04-25T11:14:24.245 DEBUG:teuthology.orchestra.run.smithi152:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T11:14:24.995 INFO:teuthology.orchestra.run.smithi152.stdout:-rw-r--r--. 1 ubuntu ubuntu 781647 Apr 25 11:14 /home/ubuntu/cephtest/cephadm 2024-04-25T11:14:24.995 DEBUG:teuthology.orchestra.run.smithi069:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T11:14:25.017 DEBUG:teuthology.orchestra.run.smithi146:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T11:14:25.036 DEBUG:teuthology.orchestra.run.smithi152:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T11:14:25.074 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 on all hosts... 2024-04-25T11:14:25.074 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 pull 2024-04-25T11:14:25.077 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 pull 2024-04-25T11:14:25.079 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 pull 2024-04-25T11:14:26.579 INFO:teuthology.orchestra.run.smithi069.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8... 2024-04-25T11:14:26.602 INFO:teuthology.orchestra.run.smithi152.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8... 2024-04-25T11:14:26.617 INFO:teuthology.orchestra.run.smithi146.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8... 2024-04-25T11:14:57.774 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-25T11:14:57.775 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T11:14:57.775 INFO:teuthology.orchestra.run.smithi069.stdout: "image_id": "d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4", 2024-04-25T11:14:57.775 INFO:teuthology.orchestra.run.smithi069.stdout: "repo_digests": [ 2024-04-25T11:14:57.775 INFO:teuthology.orchestra.run.smithi069.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385" 2024-04-25T11:14:57.775 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-04-25T11:14:57.775 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-25T11:15:24.975 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-04-25T11:15:24.975 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T11:15:24.976 INFO:teuthology.orchestra.run.smithi146.stdout: "image_id": "d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4", 2024-04-25T11:15:24.976 INFO:teuthology.orchestra.run.smithi146.stdout: "repo_digests": [ 2024-04-25T11:15:24.976 INFO:teuthology.orchestra.run.smithi146.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385" 2024-04-25T11:15:24.976 INFO:teuthology.orchestra.run.smithi146.stdout: ] 2024-04-25T11:15:24.976 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-04-25T11:15:28.935 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-25T11:15:28.935 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T11:15:28.936 INFO:teuthology.orchestra.run.smithi152.stdout: "image_id": "d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4", 2024-04-25T11:15:28.936 INFO:teuthology.orchestra.run.smithi152.stdout: "repo_digests": [ 2024-04-25T11:15:28.936 INFO:teuthology.orchestra.run.smithi152.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385" 2024-04-25T11:15:28.936 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-04-25T11:15:28.936 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-25T11:15:28.960 DEBUG:teuthology.orchestra.run.smithi069:> sudo mkdir -p /etc/ceph 2024-04-25T11:15:28.993 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /etc/ceph 2024-04-25T11:15:29.026 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /etc/ceph 2024-04-25T11:15:29.052 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 777 /etc/ceph 2024-04-25T11:15:29.080 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 777 /etc/ceph 2024-04-25T11:15:29.108 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 777 /etc/ceph 2024-04-25T11:15:29.132 INFO:tasks.cephadm:Writing seed config... 2024-04-25T11:15:29.133 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-25T11:15:29.134 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-25T11:15:29.134 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-25T11:15:29.134 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-25T11:15:29.134 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-25T11:15:29.134 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-25T11:15:29.134 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-25T11:15:29.134 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-25T11:15:29.134 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-25T11:15:29.134 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-25T11:15:29.134 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-25T11:15:29.134 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-25T11:15:29.134 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-25T11:15:29.134 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-25T11:15:29.134 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-25T11:15:29.134 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-25T11:15:29.135 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-04-25T11:15:29.135 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-25T11:15:29.135 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-25T11:15:29.135 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-25T11:15:29.135 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-25T11:15:29.135 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-25T11:15:29.135 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-25T11:15:29.135 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-25T11:15:29.135 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-25T11:15:29.135 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-25T11:15:29.135 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-25T11:15:29.135 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-25T11:15:29.135 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-25T11:15:29.135 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-25T11:15:29.136 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-25T11:15:29.136 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-25T11:15:29.136 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-25T11:15:29.136 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-25T11:15:29.136 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-25T11:15:29.136 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-25T11:15:29.136 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-25T11:15:29.136 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-25T11:15:29.136 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-25T11:15:29.136 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-25T11:15:29.136 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-25T11:15:29.136 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-25T11:15:29.136 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-25T11:15:29.136 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-04-25T11:15:29.137 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T11:15:29.137 DEBUG:teuthology.orchestra.run.smithi069:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-25T11:15:29.154 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = f7ed73dc-02f4-11ef-bc93-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-25T11:15:29.155 DEBUG:teuthology.orchestra.run.smithi069:mon.a> sudo journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mon.a.service 2024-04-25T11:15:29.196 DEBUG:teuthology.orchestra.run.smithi069:mgr.x> sudo journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mgr.x.service 2024-04-25T11:15:29.240 INFO:tasks.cephadm:Bootstrapping... 2024-04-25T11:15:29.241 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 -v bootstrap --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.69 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-25T11:15:29.439 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2024-04-25T11:15:29.439 INFO:teuthology.orchestra.run.smithi069.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8', '-v', 'bootstrap', '--fsid', 'f7ed73dc-02f4-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.69', '--skip-admin-label'] 2024-04-25T11:15:29.440 INFO:teuthology.orchestra.run.smithi069.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-25T11:15:29.440 INFO:teuthology.orchestra.run.smithi069.stdout:Verifying podman|docker is present... 2024-04-25T11:15:29.466 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T11:15:29.466 INFO:teuthology.orchestra.run.smithi069.stdout:Verifying lvm2 is present... 2024-04-25T11:15:29.466 INFO:teuthology.orchestra.run.smithi069.stdout:Verifying time synchronization is in place... 2024-04-25T11:15:29.473 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T11:15:29.473 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T11:15:29.480 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T11:15:29.480 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout inactive 2024-04-25T11:15:29.486 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout enabled 2024-04-25T11:15:29.493 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout active 2024-04-25T11:15:29.493 INFO:teuthology.orchestra.run.smithi069.stdout:Unit chronyd.service is enabled and running 2024-04-25T11:15:29.493 INFO:teuthology.orchestra.run.smithi069.stdout:Repeating the final host check... 2024-04-25T11:15:29.518 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T11:15:29.518 INFO:teuthology.orchestra.run.smithi069.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-25T11:15:29.518 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl is present 2024-04-25T11:15:29.518 INFO:teuthology.orchestra.run.smithi069.stdout:lvcreate is present 2024-04-25T11:15:29.525 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T11:15:29.525 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T11:15:29.531 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T11:15:29.531 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout inactive 2024-04-25T11:15:29.539 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout enabled 2024-04-25T11:15:29.546 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout active 2024-04-25T11:15:29.546 INFO:teuthology.orchestra.run.smithi069.stdout:Unit chronyd.service is enabled and running 2024-04-25T11:15:29.546 INFO:teuthology.orchestra.run.smithi069.stdout:Host looks OK 2024-04-25T11:15:29.546 INFO:teuthology.orchestra.run.smithi069.stdout:Cluster fsid: f7ed73dc-02f4-11ef-bc93-c7b262605968 2024-04-25T11:15:29.546 INFO:teuthology.orchestra.run.smithi069.stdout:Acquiring lock 139682830666912 on /run/cephadm/f7ed73dc-02f4-11ef-bc93-c7b262605968.lock 2024-04-25T11:15:29.547 INFO:teuthology.orchestra.run.smithi069.stdout:Lock 139682830666912 acquired on /run/cephadm/f7ed73dc-02f4-11ef-bc93-c7b262605968.lock 2024-04-25T11:15:29.547 INFO:teuthology.orchestra.run.smithi069.stdout:Verifying IP 172.21.15.69 port 3300 ... 2024-04-25T11:15:29.547 INFO:teuthology.orchestra.run.smithi069.stdout:Verifying IP 172.21.15.69 port 6789 ... 2024-04-25T11:15:29.548 INFO:teuthology.orchestra.run.smithi069.stdout:Base mon IP(s) is [172.21.15.69:3300, 172.21.15.69:6789], mon addrv is [v2:172.21.15.69:3300,v1:172.21.15.69:6789] 2024-04-25T11:15:29.552 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.69 metric 100 2024-04-25T11:15:29.552 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.69 metric 100 2024-04-25T11:15:29.555 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-25T11:15:29.556 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-04-25T11:15:29.556 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-04-25T11:15:29.558 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-25T11:15:29.559 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-25T11:15:29.559 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T11:15:29.559 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-25T11:15:29.559 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout inet6 fe80::4637:f1e7:b6c7:4071/64 scope link noprefixroute 2024-04-25T11:15:29.559 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T11:15:29.559 INFO:teuthology.orchestra.run.smithi069.stdout:Mon IP `172.21.15.69` is in CIDR network `172.21.0.0/20` 2024-04-25T11:15:29.559 INFO:teuthology.orchestra.run.smithi069.stdout:Mon IP `172.21.15.69` is in CIDR network `172.21.0.0/20` 2024-04-25T11:15:29.560 INFO:teuthology.orchestra.run.smithi069.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-25T11:15:29.560 INFO:teuthology.orchestra.run.smithi069.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-25T11:15:29.561 INFO:teuthology.orchestra.run.smithi069.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8... 2024-04-25T11:15:30.109 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stdout d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 2024-04-25T11:15:30.110 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8... 2024-04-25T11:15:30.110 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stderr Getting image source signatures 2024-04-25T11:15:30.110 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stderr Copying blob sha256:3d665f4c032644be9cae748af04bee69f2bfe96cdc3616dabe0ab17899633ab5 2024-04-25T11:15:30.110 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stderr Copying blob sha256:665339d96aad80d083ba9988a62422bcef0a42210fb2b28340e33ab98589f7b8 2024-04-25T11:15:30.110 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stderr Copying config sha256:d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 2024-04-25T11:15:30.110 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-25T11:15:31.006 INFO:teuthology.orchestra.run.smithi069.stdout:ceph: stdout ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev) 2024-04-25T11:15:31.006 INFO:teuthology.orchestra.run.smithi069.stdout:Ceph version: ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev) 2024-04-25T11:15:31.006 INFO:teuthology.orchestra.run.smithi069.stdout:Extracting ceph user uid/gid from container image... 2024-04-25T11:15:31.987 INFO:teuthology.orchestra.run.smithi069.stdout:stat: stdout 167 167 2024-04-25T11:15:31.987 INFO:teuthology.orchestra.run.smithi069.stdout:Creating initial keys... 2024-04-25T11:15:32.944 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph-authtool: stdout AQBUOypmEN89FBAA3qd35eMAQ+kelocvkIuFFA== 2024-04-25T11:15:33.785 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph-authtool: stdout AQBVOypml6FaDhAARQpreUybvRhL67Et/xl2KA== 2024-04-25T11:15:34.710 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph-authtool: stdout AQBWOypmczNBBhAAlqIUaVIFlJLNBCkOAUuK1g== 2024-04-25T11:15:34.711 INFO:teuthology.orchestra.run.smithi069.stdout:Creating initial monmap... 2024-04-25T11:15:35.592 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T11:15:35.592 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-25T11:15:35.592 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to f7ed73dc-02f4-11ef-bc93-c7b262605968 2024-04-25T11:15:35.592 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T11:15:35.592 INFO:teuthology.orchestra.run.smithi069.stdout:monmaptool for a [v2:172.21.15.69:3300,v1:172.21.15.69:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T11:15:35.592 INFO:teuthology.orchestra.run.smithi069.stdout:setting min_mon_release = quincy 2024-04-25T11:15:35.592 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/monmaptool: set fsid to f7ed73dc-02f4-11ef-bc93-c7b262605968 2024-04-25T11:15:35.592 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T11:15:35.592 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:15:35.593 INFO:teuthology.orchestra.run.smithi069.stdout:Creating mon... 2024-04-25T11:15:36.531 INFO:teuthology.orchestra.run.smithi069.stdout:create mon.a on 2024-04-25T11:15:36.751 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-25T11:15:36.884 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-25T11:15:37.086 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968.target. 2024-04-25T11:15:37.086 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968.target. 2024-04-25T11:15:37.299 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mon.a 2024-04-25T11:15:37.299 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Failed to reset failed state of unit ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-25T11:15:37.458 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968.target.wants/ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@.service. 2024-04-25T11:15:37.822 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:37 smithi069 podman[28797]: 2024-04-25 11:15:37.706143643 +0000 UTC m=+0.116366988 container create 7311c612062f1a1505215c3fe1b6027670ee375f7c40eff0edd78e7b39000c8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-25T11:15:38.005 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T11:15:38.005 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout disabled 2024-04-25T11:15:38.011 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T11:15:38.012 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout inactive 2024-04-25T11:15:38.012 INFO:teuthology.orchestra.run.smithi069.stdout:firewalld.service is not enabled 2024-04-25T11:15:38.012 INFO:teuthology.orchestra.run.smithi069.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T11:15:38.012 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for mon to start... 2024-04-25T11:15:38.012 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for mon... 2024-04-25T11:15:38.137 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:37 smithi069 podman[28797]: 2024-04-25 11:15:37.908231996 +0000 UTC m=+0.318455299 container init 7311c612062f1a1505215c3fe1b6027670ee375f7c40eff0edd78e7b39000c8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux ) 2024-04-25T11:15:38.137 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:37 smithi069 podman[28797]: 2024-04-25 11:15:37.911110688 +0000 UTC m=+0.321333988 container start 7311c612062f1a1505215c3fe1b6027670ee375f7c40eff0edd78e7b39000c8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-25T11:15:38.137 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:37 smithi069 ceph-mon[28816]: mkfs f7ed73dc-02f4-11ef-bc93-c7b262605968 2024-04-25T11:15:38.137 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:37 smithi069 ceph-mon[28816]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T11:15:38.137 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:37 smithi069 bash[28797]: 7311c612062f1a1505215c3fe1b6027670ee375f7c40eff0edd78e7b39000c8f 2024-04-25T11:15:38.137 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:37 smithi069 systemd[1]: Started Ceph mon.a for f7ed73dc-02f4-11ef-bc93-c7b262605968. 2024-04-25T11:15:39.098 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:38 smithi069 ceph-mon[28816]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T11:15:39.098 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:38 smithi069 ceph-mon[28816]: monmap epoch 1 2024-04-25T11:15:39.099 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:38 smithi069 ceph-mon[28816]: fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 2024-04-25T11:15:39.099 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:38 smithi069 ceph-mon[28816]: last_changed 2024-04-25T11:15:35.052240+0000 2024-04-25T11:15:39.099 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:38 smithi069 ceph-mon[28816]: created 2024-04-25T11:15:35.052240+0000 2024-04-25T11:15:39.099 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:38 smithi069 ceph-mon[28816]: min_mon_release 19 (squid) 2024-04-25T11:15:39.099 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:38 smithi069 ceph-mon[28816]: election_strategy: 1 2024-04-25T11:15:39.099 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:38 smithi069 ceph-mon[28816]: 0: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.a 2024-04-25T11:15:39.099 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:38 smithi069 ceph-mon[28816]: fsmap 2024-04-25T11:15:39.099 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:38 smithi069 ceph-mon[28816]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T11:15:39.099 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:38 smithi069 ceph-mon[28816]: mgrmap e1: no daemons active 2024-04-25T11:15:39.099 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:38 smithi069 ceph-mon[28816]: from='client.? 172.21.15.69:0/1641877153' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-25T11:15:39.107 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout cluster: 2024-04-25T11:15:39.108 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout id: f7ed73dc-02f4-11ef-bc93-c7b262605968 2024-04-25T11:15:39.108 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-25T11:15:39.108 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-04-25T11:15:39.108 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout services: 2024-04-25T11:15:39.108 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.598053s) 2024-04-25T11:15:39.108 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-25T11:15:39.108 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-25T11:15:39.108 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-04-25T11:15:39.108 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout data: 2024-04-25T11:15:39.108 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-25T11:15:39.108 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-25T11:15:39.109 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-25T11:15:39.109 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout pgs: 2024-04-25T11:15:39.109 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-04-25T11:15:39.109 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:38.484+0000 7fdeacc9f700 1 Processor -- start 2024-04-25T11:15:39.109 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:38.484+0000 7fdeacc9f700 1 -- start start 2024-04-25T11:15:39.109 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:38.484+0000 7fdeacc9f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdea81099a0 0x7fdea8109db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:39.109 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:38.484+0000 7fdeacc9f700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdea810a3a0 con 0x7fdea81099a0 2024-04-25T11:15:39.109 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:38.484+0000 7fdea659c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdea81099a0 0x7fdea8109db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:39.109 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:38.484+0000 7fdea659c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdea81099a0 0x7fdea8109db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59676/0 (socket says 172.21.15.69:59676) 2024-04-25T11:15:39.109 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:38.484+0000 7fdea659c700 1 -- 172.21.15.69:0/2090185741 learned_addr learned my addr 172.21.15.69:0/2090185741 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:15:39.109 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:38.485+0000 7fdea659c700 1 -- 172.21.15.69:0/2090185741 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdea810aad0 con 0x7fdea81099a0 2024-04-25T11:15:39.110 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:38.485+0000 7fdea659c700 1 --2- 172.21.15.69:0/2090185741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdea81099a0 0x7fdea8109db0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fde980098a0 tx=0x7fde98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d333d943152d7f7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:39.110 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:38.485+0000 7fdea559a700 1 -- 172.21.15.69:0/2090185741 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde98004160 con 0x7fdea81099a0 2024-04-25T11:15:39.110 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:38.485+0000 7fdea559a700 1 -- 172.21.15.69:0/2090185741 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde98004300 con 0x7fdea81099a0 2024-04-25T11:15:39.110 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:38.485+0000 7fdea559a700 1 -- 172.21.15.69:0/2090185741 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde980036a0 con 0x7fdea81099a0 2024-04-25T11:15:39.110 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:38.486+0000 7fdeacc9f700 1 -- 172.21.15.69:0/2090185741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdea81099a0 msgr2=0x7fdea8109db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:39.110 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:38.486+0000 7fdeacc9f700 1 --2- 172.21.15.69:0/2090185741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdea81099a0 0x7fdea8109db0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fde980098a0 tx=0x7fde98009bb0 comp rx=0 tx=0).stop 2024-04-25T11:15:39.110 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:38.486+0000 7fdeacc9f700 1 -- 172.21.15.69:0/2090185741 shutdown_connections 2024-04-25T11:15:39.110 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:38.486+0000 7fdeacc9f700 1 --2- 172.21.15.69:0/2090185741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdea81099a0 0x7fdea8109db0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:39.110 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:38.486+0000 7fdeacc9f700 1 -- 172.21.15.69:0/2090185741 >> 172.21.15.69:0/2090185741 conn(0x7fdea8105040 msgr2=0x7fdea8107490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:39.110 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:38.486+0000 7fdeacc9f700 1 -- 172.21.15.69:0/2090185741 shutdown_connections 2024-04-25T11:15:39.111 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:38.486+0000 7fdeacc9f700 1 -- 172.21.15.69:0/2090185741 wait complete. 2024-04-25T11:15:39.111 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:38.487+0000 7fdeacc9f700 1 Processor -- start 2024-04-25T11:15:39.111 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:38.487+0000 7fdeacc9f700 1 -- start start 2024-04-25T11:15:39.111 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:38.487+0000 7fdeacc9f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdea81099a0 0x7fdea819d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:39.111 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:38.487+0000 7fdeacc9f700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdea810b000 con 0x7fdea81099a0 2024-04-25T11:15:39.111 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:38.487+0000 7fdea659c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdea81099a0 0x7fdea819d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:39.111 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:38.487+0000 7fdea659c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdea81099a0 0x7fdea819d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59688/0 (socket says 172.21.15.69:59688) 2024-04-25T11:15:39.111 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:38.487+0000 7fdea659c700 1 -- 172.21.15.69:0/1641877153 learned_addr learned my addr 172.21.15.69:0/1641877153 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:15:39.111 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:38.487+0000 7fdea659c700 1 -- 172.21.15.69:0/1641877153 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdea819d8e0 con 0x7fdea81099a0 2024-04-25T11:15:39.111 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:38.487+0000 7fdea659c700 1 --2- 172.21.15.69:0/1641877153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdea81099a0 0x7fdea819d3a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fde98000c00 tx=0x7fde98004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:39.111 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:38.488+0000 7fde977fe700 1 -- 172.21.15.69:0/1641877153 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde98003db0 con 0x7fdea81099a0 2024-04-25T11:15:39.111 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:38.488+0000 7fde977fe700 1 -- 172.21.15.69:0/1641877153 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde98011630 con 0x7fdea81099a0 2024-04-25T11:15:39.111 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:38.488+0000 7fde977fe700 1 -- 172.21.15.69:0/1641877153 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde98011950 con 0x7fdea81099a0 2024-04-25T11:15:39.112 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:38.488+0000 7fdeacc9f700 1 -- 172.21.15.69:0/1641877153 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdea819db70 con 0x7fdea81099a0 2024-04-25T11:15:39.112 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:38.488+0000 7fdeacc9f700 1 -- 172.21.15.69:0/1641877153 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdea81a0780 con 0x7fdea81099a0 2024-04-25T11:15:39.112 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:38.488+0000 7fde977fe700 1 -- 172.21.15.69:0/1641877153 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fde9801a460 con 0x7fdea81099a0 2024-04-25T11:15:39.112 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:38.488+0000 7fde977fe700 1 -- 172.21.15.69:0/1641877153 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fde9801b780 con 0x7fdea81099a0 2024-04-25T11:15:39.112 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:38.489+0000 7fdeacc9f700 1 -- 172.21.15.69:0/1641877153 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdea8064cf0 con 0x7fdea81099a0 2024-04-25T11:15:39.112 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:38.491+0000 7fde977fe700 1 -- 172.21.15.69:0/1641877153 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fde98031020 con 0x7fdea81099a0 2024-04-25T11:15:39.112 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:38.561+0000 7fdeacc9f700 1 -- 172.21.15.69:0/1641877153 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fdea802e790 con 0x7fdea81099a0 2024-04-25T11:15:39.112 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:38.561+0000 7fde977fe700 1 -- 172.21.15.69:0/1641877153 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fde9801b9d0 con 0x7fdea81099a0 2024-04-25T11:15:39.112 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:38.562+0000 7fdeacc9f700 1 -- 172.21.15.69:0/1641877153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdea81099a0 msgr2=0x7fdea819d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:39.112 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:38.562+0000 7fdeacc9f700 1 --2- 172.21.15.69:0/1641877153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdea81099a0 0x7fdea819d3a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fde98000c00 tx=0x7fde98004750 comp rx=0 tx=0).stop 2024-04-25T11:15:39.112 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:38.562+0000 7fdeacc9f700 1 -- 172.21.15.69:0/1641877153 shutdown_connections 2024-04-25T11:15:39.112 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:38.562+0000 7fdeacc9f700 1 --2- 172.21.15.69:0/1641877153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdea81099a0 0x7fdea819d3a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:39.112 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:38.562+0000 7fdeacc9f700 1 -- 172.21.15.69:0/1641877153 >> 172.21.15.69:0/1641877153 conn(0x7fdea8105040 msgr2=0x7fdea8193a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:39.113 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:38.562+0000 7fdeacc9f700 1 -- 172.21.15.69:0/1641877153 shutdown_connections 2024-04-25T11:15:39.113 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:38.562+0000 7fdeacc9f700 1 -- 172.21.15.69:0/1641877153 wait complete. 2024-04-25T11:15:39.113 INFO:teuthology.orchestra.run.smithi069.stdout:mon is available 2024-04-25T11:15:39.113 INFO:teuthology.orchestra.run.smithi069.stdout:Assimilating anything we can from ceph.conf... 2024-04-25T11:15:40.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:39 smithi069 ceph-mon[28816]: from='client.? 172.21.15.69:0/3976526067' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T11:15:40.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:39 smithi069 ceph-mon[28816]: from='client.? 172.21.15.69:0/3976526067' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-25T11:15:40.251 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-04-25T11:15:40.251 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T11:15:40.252 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout fsid = f7ed73dc-02f4-11ef-bc93-c7b262605968 2024-04-25T11:15:40.252 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T11:15:40.252 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.69:3300,v1:172.21.15.69:6789] 2024-04-25T11:15:40.252 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T11:15:40.252 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T11:15:40.252 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T11:15:40.252 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T11:15:40.252 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-04-25T11:15:40.252 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T11:15:40.252 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T11:15:40.252 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-04-25T11:15:40.252 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T11:15:40.252 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T11:15:40.253 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T11:15:40.253 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T11:15:40.253 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T11:15:40.253 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T11:15:40.253 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:39.509+0000 7fdb0cd00700 1 Processor -- start 2024-04-25T11:15:40.253 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:39.509+0000 7fdb0cd00700 1 -- start start 2024-04-25T11:15:40.253 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:39.509+0000 7fdb0cd00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb08109a00 0x7fdb08109e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:40.253 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:39.509+0000 7fdb0cd00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0810a400 con 0x7fdb08109a00 2024-04-25T11:15:40.254 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:39.509+0000 7fdb0659c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb08109a00 0x7fdb08109e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:40.254 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:39.510+0000 7fdb0659c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb08109a00 0x7fdb08109e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59700/0 (socket says 172.21.15.69:59700) 2024-04-25T11:15:40.254 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:39.510+0000 7fdb0659c700 1 -- 172.21.15.69:0/1065515354 learned_addr learned my addr 172.21.15.69:0/1065515354 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:15:40.254 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:39.510+0000 7fdb0659c700 1 -- 172.21.15.69:0/1065515354 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb0810ab30 con 0x7fdb08109a00 2024-04-25T11:15:40.254 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:39.510+0000 7fdb0659c700 1 --2- 172.21.15.69:0/1065515354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb08109a00 0x7fdb08109e10 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fdaf80098a0 tx=0x7fdaf8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b0a251a01eb64e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:40.254 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:39.510+0000 7fdb05d9b700 1 -- 172.21.15.69:0/1065515354 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdaf8004160 con 0x7fdb08109a00 2024-04-25T11:15:40.254 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:39.510+0000 7fdb05d9b700 1 -- 172.21.15.69:0/1065515354 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaf8004300 con 0x7fdb08109a00 2024-04-25T11:15:40.254 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:39.510+0000 7fdb05d9b700 1 -- 172.21.15.69:0/1065515354 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdaf80036a0 con 0x7fdb08109a00 2024-04-25T11:15:40.254 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:39.511+0000 7fdb0cd00700 1 -- 172.21.15.69:0/1065515354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb08109a00 msgr2=0x7fdb08109e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:40.254 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:39.511+0000 7fdb0cd00700 1 --2- 172.21.15.69:0/1065515354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb08109a00 0x7fdb08109e10 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fdaf80098a0 tx=0x7fdaf8009bb0 comp rx=0 tx=0).stop 2024-04-25T11:15:40.254 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:39.511+0000 7fdb0cd00700 1 -- 172.21.15.69:0/1065515354 shutdown_connections 2024-04-25T11:15:40.254 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:39.511+0000 7fdb0cd00700 1 --2- 172.21.15.69:0/1065515354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb08109a00 0x7fdb08109e10 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:40.254 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:39.511+0000 7fdb0cd00700 1 -- 172.21.15.69:0/1065515354 >> 172.21.15.69:0/1065515354 conn(0x7fdb08105080 msgr2=0x7fdb081074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:40.255 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:39.511+0000 7fdb0cd00700 1 -- 172.21.15.69:0/1065515354 shutdown_connections 2024-04-25T11:15:40.255 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:39.511+0000 7fdb0cd00700 1 -- 172.21.15.69:0/1065515354 wait complete. 2024-04-25T11:15:40.255 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:39.512+0000 7fdb0cd00700 1 Processor -- start 2024-04-25T11:15:40.255 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:39.512+0000 7fdb0cd00700 1 -- start start 2024-04-25T11:15:40.255 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:39.512+0000 7fdb0cd00700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb08109a00 0x7fdb0819d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:40.255 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:39.512+0000 7fdb0cd00700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0810b060 con 0x7fdb08109a00 2024-04-25T11:15:40.255 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:39.512+0000 7fdb0659c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb08109a00 0x7fdb0819d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:40.255 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:39.512+0000 7fdb0659c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb08109a00 0x7fdb0819d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59716/0 (socket says 172.21.15.69:59716) 2024-04-25T11:15:40.255 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:39.512+0000 7fdb0659c700 1 -- 172.21.15.69:0/3976526067 learned_addr learned my addr 172.21.15.69:0/3976526067 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:15:40.255 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:39.512+0000 7fdb0659c700 1 -- 172.21.15.69:0/3976526067 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb0819d920 con 0x7fdb08109a00 2024-04-25T11:15:40.255 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:39.513+0000 7fdb0659c700 1 --2- 172.21.15.69:0/3976526067 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb08109a00 0x7fdb0819d3e0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8000c00 tx=0x7fdaf8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:40.255 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:39.513+0000 7fdafffff700 1 -- 172.21.15.69:0/3976526067 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdaf8003db0 con 0x7fdb08109a00 2024-04-25T11:15:40.255 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:39.513+0000 7fdafffff700 1 -- 172.21.15.69:0/3976526067 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaf8011630 con 0x7fdb08109a00 2024-04-25T11:15:40.255 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:39.513+0000 7fdafffff700 1 -- 172.21.15.69:0/3976526067 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdaf8011950 con 0x7fdb08109a00 2024-04-25T11:15:40.256 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:39.513+0000 7fdb0cd00700 1 -- 172.21.15.69:0/3976526067 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb0819dbb0 con 0x7fdb08109a00 2024-04-25T11:15:40.256 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:39.513+0000 7fdb0cd00700 1 -- 172.21.15.69:0/3976526067 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb081a07c0 con 0x7fdb08109a00 2024-04-25T11:15:40.256 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:39.513+0000 7fdafffff700 1 -- 172.21.15.69:0/3976526067 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdaf801a460 con 0x7fdb08109a00 2024-04-25T11:15:40.256 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:39.513+0000 7fdafffff700 1 -- 172.21.15.69:0/3976526067 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdaf801b780 con 0x7fdb08109a00 2024-04-25T11:15:40.256 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:39.514+0000 7fdb0cd00700 1 -- 172.21.15.69:0/3976526067 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb08064cf0 con 0x7fdb08109a00 2024-04-25T11:15:40.256 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:39.516+0000 7fdafffff700 1 -- 172.21.15.69:0/3976526067 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fdaf8031020 con 0x7fdb08109a00 2024-04-25T11:15:40.256 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:39.582+0000 7fdb0cd00700 1 -- 172.21.15.69:0/3976526067 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fdb0802e790 con 0x7fdb08109a00 2024-04-25T11:15:40.256 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:39.586+0000 7fdafffff700 1 -- 172.21.15.69:0/3976526067 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+465 (secure 0 0 0) 0x7fdaf801bad0 con 0x7fdb08109a00 2024-04-25T11:15:40.256 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:39.587+0000 7fdafffff700 1 -- 172.21.15.69:0/3976526067 <== mon.0 v2:172.21.15.69:3300/0 8 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fdaf801ad50 con 0x7fdb08109a00 2024-04-25T11:15:40.256 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:39.588+0000 7fdb0cd00700 1 -- 172.21.15.69:0/3976526067 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb08109a00 msgr2=0x7fdb0819d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:40.256 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:39.588+0000 7fdb0cd00700 1 --2- 172.21.15.69:0/3976526067 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb08109a00 0x7fdb0819d3e0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8000c00 tx=0x7fdaf8004750 comp rx=0 tx=0).stop 2024-04-25T11:15:40.256 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:39.588+0000 7fdb0cd00700 1 -- 172.21.15.69:0/3976526067 shutdown_connections 2024-04-25T11:15:40.256 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:39.588+0000 7fdb0cd00700 1 --2- 172.21.15.69:0/3976526067 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb08109a00 0x7fdb0819d3e0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:40.257 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:39.588+0000 7fdb0cd00700 1 -- 172.21.15.69:0/3976526067 >> 172.21.15.69:0/3976526067 conn(0x7fdb08105080 msgr2=0x7fdb081913c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:40.257 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:39.588+0000 7fdb0cd00700 1 -- 172.21.15.69:0/3976526067 shutdown_connections 2024-04-25T11:15:40.257 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:39.588+0000 7fdb0cd00700 1 -- 172.21.15.69:0/3976526067 wait complete. 2024-04-25T11:15:40.257 INFO:teuthology.orchestra.run.smithi069.stdout:Generating new minimal ceph.conf... 2024-04-25T11:15:41.241 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:40 smithi069 ceph-mon[28816]: from='client.? 172.21.15.69:0/1015624947' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:15:41.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:40.664+0000 7fb967e13700 1 Processor -- start 2024-04-25T11:15:41.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:40.664+0000 7fb967e13700 1 -- start start 2024-04-25T11:15:41.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:40.664+0000 7fb967e13700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb96007d560 0x7fb96010c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:41.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:40.664+0000 7fb967e13700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb96010cb10 con 0x7fb96007d560 2024-04-25T11:15:41.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:40.665+0000 7fb965baf700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb96007d560 0x7fb96010c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:41.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:40.665+0000 7fb965baf700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb96007d560 0x7fb96010c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59720/0 (socket says 172.21.15.69:59720) 2024-04-25T11:15:41.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:40.665+0000 7fb965baf700 1 -- 172.21.15.69:0/4035626365 learned_addr learned my addr 172.21.15.69:0/4035626365 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:15:41.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:40.665+0000 7fb965baf700 1 -- 172.21.15.69:0/4035626365 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb96010cc90 con 0x7fb96007d560 2024-04-25T11:15:41.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:40.665+0000 7fb965baf700 1 --2- 172.21.15.69:0/4035626365 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb96007d560 0x7fb96010c520 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb95c0098a0 tx=0x7fb95c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=583fa859158d95d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:41.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:40.665+0000 7fb964bad700 1 -- 172.21.15.69:0/4035626365 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb95c004160 con 0x7fb96007d560 2024-04-25T11:15:41.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:40.665+0000 7fb964bad700 1 -- 172.21.15.69:0/4035626365 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fb95c004300 con 0x7fb96007d560 2024-04-25T11:15:41.248 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:40.665+0000 7fb964bad700 1 -- 172.21.15.69:0/4035626365 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb95c0119e0 con 0x7fb96007d560 2024-04-25T11:15:41.249 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:40.666+0000 7fb967e13700 1 -- 172.21.15.69:0/4035626365 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb96007d560 msgr2=0x7fb96010c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:41.249 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:40.666+0000 7fb967e13700 1 --2- 172.21.15.69:0/4035626365 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb96007d560 0x7fb96010c520 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb95c0098a0 tx=0x7fb95c009bb0 comp rx=0 tx=0).stop 2024-04-25T11:15:41.249 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:40.666+0000 7fb967e13700 1 -- 172.21.15.69:0/4035626365 shutdown_connections 2024-04-25T11:15:41.249 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:40.666+0000 7fb967e13700 1 --2- 172.21.15.69:0/4035626365 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb96007d560 0x7fb96010c520 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:41.249 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:40.666+0000 7fb967e13700 1 -- 172.21.15.69:0/4035626365 >> 172.21.15.69:0/4035626365 conn(0x7fb960103f90 msgr2=0x7fb9601063e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:41.249 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:40.666+0000 7fb967e13700 1 -- 172.21.15.69:0/4035626365 shutdown_connections 2024-04-25T11:15:41.249 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:40.666+0000 7fb967e13700 1 -- 172.21.15.69:0/4035626365 wait complete. 2024-04-25T11:15:41.249 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:40.667+0000 7fb967e13700 1 Processor -- start 2024-04-25T11:15:41.249 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:40.667+0000 7fb967e13700 1 -- start start 2024-04-25T11:15:41.249 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:40.667+0000 7fb967e13700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb96007d560 0x7fb96019db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:41.249 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:40.667+0000 7fb967e13700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb96010dbd0 con 0x7fb96007d560 2024-04-25T11:15:41.249 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:40.667+0000 7fb965baf700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb96007d560 0x7fb96019db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:41.249 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:40.667+0000 7fb965baf700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb96007d560 0x7fb96019db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59730/0 (socket says 172.21.15.69:59730) 2024-04-25T11:15:41.249 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:40.667+0000 7fb965baf700 1 -- 172.21.15.69:0/1015624947 learned_addr learned my addr 172.21.15.69:0/1015624947 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:15:41.250 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:40.667+0000 7fb965baf700 1 -- 172.21.15.69:0/1015624947 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb96019e080 con 0x7fb96007d560 2024-04-25T11:15:41.250 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:40.667+0000 7fb965baf700 1 --2- 172.21.15.69:0/1015624947 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb96007d560 0x7fb96019db40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb95c0116a0 tx=0x7fb95c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:41.250 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:40.667+0000 7fb956ffd700 1 -- 172.21.15.69:0/1015624947 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb95c0039f0 con 0x7fb96007d560 2024-04-25T11:15:41.250 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:40.668+0000 7fb956ffd700 1 -- 172.21.15.69:0/1015624947 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fb95c003b90 con 0x7fb96007d560 2024-04-25T11:15:41.250 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:40.668+0000 7fb967e13700 1 -- 172.21.15.69:0/1015624947 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb96019e310 con 0x7fb96007d560 2024-04-25T11:15:41.250 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:40.668+0000 7fb956ffd700 1 -- 172.21.15.69:0/1015624947 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb95c01a600 con 0x7fb96007d560 2024-04-25T11:15:41.250 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:40.668+0000 7fb967e13700 1 -- 172.21.15.69:0/1015624947 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9601a0f20 con 0x7fb96007d560 2024-04-25T11:15:41.250 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:40.668+0000 7fb956ffd700 1 -- 172.21.15.69:0/1015624947 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb95c01ac80 con 0x7fb96007d560 2024-04-25T11:15:41.251 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:40.668+0000 7fb956ffd700 1 -- 172.21.15.69:0/1015624947 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb95c019700 con 0x7fb96007d560 2024-04-25T11:15:41.251 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:40.669+0000 7fb967e13700 1 -- 172.21.15.69:0/1015624947 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb960064cf0 con 0x7fb96007d560 2024-04-25T11:15:41.251 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:40.671+0000 7fb956ffd700 1 -- 172.21.15.69:0/1015624947 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fb95c024a50 con 0x7fb96007d560 2024-04-25T11:15:41.251 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:40.737+0000 7fb967e13700 1 -- 172.21.15.69:0/1015624947 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fb9601a10f0 con 0x7fb96007d560 2024-04-25T11:15:41.251 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:40.738+0000 7fb956ffd700 1 -- 172.21.15.69:0/1015624947 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fb95c024430 con 0x7fb96007d560 2024-04-25T11:15:41.251 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:40.739+0000 7fb967e13700 1 -- 172.21.15.69:0/1015624947 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb96007d560 msgr2=0x7fb96019db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:41.252 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:40.739+0000 7fb967e13700 1 --2- 172.21.15.69:0/1015624947 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb96007d560 0x7fb96019db40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb95c0116a0 tx=0x7fb95c011a60 comp rx=0 tx=0).stop 2024-04-25T11:15:41.252 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:40.739+0000 7fb967e13700 1 -- 172.21.15.69:0/1015624947 shutdown_connections 2024-04-25T11:15:41.252 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:40.739+0000 7fb967e13700 1 --2- 172.21.15.69:0/1015624947 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb96007d560 0x7fb96019db40 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:41.252 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:40.739+0000 7fb967e13700 1 -- 172.21.15.69:0/1015624947 >> 172.21.15.69:0/1015624947 conn(0x7fb960103f90 msgr2=0x7fb960105810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:41.252 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:40.739+0000 7fb967e13700 1 -- 172.21.15.69:0/1015624947 shutdown_connections 2024-04-25T11:15:41.252 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:40.739+0000 7fb967e13700 1 -- 172.21.15.69:0/1015624947 wait complete. 2024-04-25T11:15:41.252 INFO:teuthology.orchestra.run.smithi069.stdout:Restarting the monitor... 2024-04-25T11:15:41.519 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:41 smithi069 systemd[1]: Stopping Ceph mon.a for f7ed73dc-02f4-11ef-bc93-c7b262605968... 2024-04-25T11:15:41.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:41 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mon-a[28812]: 2024-04-25T11:15:41.518+0000 7f66848d4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:15:41.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:41 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mon-a[28812]: 2024-04-25T11:15:41.518+0000 7f66848d4700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-25T11:15:41.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:41 smithi069 podman[29061]: 2024-04-25 11:15:41.744462248 +0000 UTC m=+0.338359787 container died 7311c612062f1a1505215c3fe1b6027670ee375f7c40eff0edd78e7b39000c8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424) 2024-04-25T11:15:42.255 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 podman[29061]: 2024-04-25 11:15:42.054840508 +0000 UTC m=+0.648738045 container cleanup 7311c612062f1a1505215c3fe1b6027670ee375f7c40eff0edd78e7b39000c8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T11:15:42.255 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 bash[29061]: ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mon-a 2024-04-25T11:15:42.518 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 podman[29075]: 2024-04-25 11:15:42.255740975 +0000 UTC m=+0.507392307 container remove 7311c612062f1a1505215c3fe1b6027670ee375f7c40eff0edd78e7b39000c8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS) 2024-04-25T11:15:42.519 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 systemd[1]: ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-25T11:15:42.519 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 systemd[1]: Stopped Ceph mon.a for f7ed73dc-02f4-11ef-bc93-c7b262605968. 2024-04-25T11:15:42.519 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 systemd[1]: Starting Ceph mon.a for f7ed73dc-02f4-11ef-bc93-c7b262605968... 2024-04-25T11:15:42.794 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 podman[29169]: 2024-04-25 11:15:42.59726216 +0000 UTC m=+0.018016670 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T11:15:42.794 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 podman[29169]: 2024-04-25 11:15:42.698082269 +0000 UTC m=+0.118836774 container create a2cd3fa6825851085799332cf869756d214713181871e6aa820ed8572268610b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T11:15:43.021 INFO:teuthology.orchestra.run.smithi069.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-25T11:15:43.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 podman[29169]: 2024-04-25 11:15:42.900307953 +0000 UTC m=+0.321062450 container init a2cd3fa6825851085799332cf869756d214713181871e6aa820ed8572268610b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS) 2024-04-25T11:15:43.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 podman[29169]: 2024-04-25 11:15:42.903134981 +0000 UTC m=+0.323889479 container start a2cd3fa6825851085799332cf869756d214713181871e6aa820ed8572268610b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True) 2024-04-25T11:15:43.201 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T11:15:43.201 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev), process ceph-mon, pid 2 2024-04-25T11:15:43.201 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: pidfile_write: ignore empty --pid-file 2024-04-25T11:15:43.201 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: load: jerasure load: lrc 2024-04-25T11:15:43.201 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: RocksDB version: 7.9.2 2024-04-25T11:15:43.201 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Git sha 0 2024-04-25T11:15:43.201 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Compile date 2024-04-25 02:57:34 2024-04-25T11:15:43.201 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: DB SUMMARY 2024-04-25T11:15:43.201 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: DB Session ID: HJS8137AIZ5KS0J80NF5 2024-04-25T11:15:43.201 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: CURRENT file: CURRENT 2024-04-25T11:15:43.202 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T11:15:43.202 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-25T11:15:43.202 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-25T11:15:43.202 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 103546 ; 2024-04-25T11:15:43.202 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.error_if_exists: 0 2024-04-25T11:15:43.202 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.create_if_missing: 0 2024-04-25T11:15:43.202 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.paranoid_checks: 1 2024-04-25T11:15:43.202 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T11:15:43.202 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T11:15:43.202 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T11:15:43.202 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.env: 0x555af655a500 2024-04-25T11:15:43.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T11:15:43.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.info_log: 0x555af79d7fc0 2024-04-25T11:15:43.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T11:15:43.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.statistics: (nil) 2024-04-25T11:15:43.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.use_fsync: 0 2024-04-25T11:15:43.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.max_log_file_size: 0 2024-04-25T11:15:43.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T11:15:43.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T11:15:43.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T11:15:43.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T11:15:43.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.allow_fallocate: 1 2024-04-25T11:15:43.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T11:15:43.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T11:15:43.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.use_direct_reads: 0 2024-04-25T11:15:43.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T11:15:43.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T11:15:43.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.db_log_dir: 2024-04-25T11:15:43.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.wal_dir: 2024-04-25T11:15:43.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T11:15:43.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T11:15:43.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T11:15:43.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T11:15:43.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T11:15:43.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T11:15:43.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T11:15:43.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T11:15:43.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.write_buffer_manager: 0x555af79a88c0 2024-04-25T11:15:43.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T11:15:43.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T11:15:43.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T11:15:43.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T11:15:43.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T11:15:43.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T11:15:43.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T11:15:43.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T11:15:43.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.unordered_write: 0 2024-04-25T11:15:43.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T11:15:43.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T11:15:43.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T11:15:43.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T11:15:43.206 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.row_cache: None 2024-04-25T11:15:43.206 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.wal_filter: None 2024-04-25T11:15:43.207 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T11:15:43.207 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T11:15:43.207 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.two_write_queues: 0 2024-04-25T11:15:43.207 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T11:15:43.207 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.wal_compression: 0 2024-04-25T11:15:43.207 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.atomic_flush: 0 2024-04-25T11:15:43.207 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T11:15:43.208 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T11:15:43.208 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T11:15:43.208 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.log_readahead_size: 0 2024-04-25T11:15:43.208 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T11:15:43.208 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T11:15:43.208 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T11:15:43.208 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T11:15:43.208 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T11:15:43.208 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T11:15:43.208 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T11:15:43.208 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.max_background_jobs: 2 2024-04-25T11:15:43.208 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.max_background_compactions: -1 2024-04-25T11:15:43.208 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.max_subcompactions: 1 2024-04-25T11:15:43.209 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T11:15:43.209 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T11:15:43.209 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T11:15:43.209 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T11:15:43.209 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T11:15:43.209 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T11:15:43.209 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T11:15:43.209 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T11:15:43.209 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.max_open_files: -1 2024-04-25T11:15:43.209 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T11:15:43.209 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T11:15:43.209 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T11:15:43.209 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T11:15:43.209 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.max_background_flushes: -1 2024-04-25T11:15:43.210 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Compression algorithms supported: 2024-04-25T11:15:43.210 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T11:15:43.210 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: kZSTD supported: 0 2024-04-25T11:15:43.210 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: kXpressCompression supported: 0 2024-04-25T11:15:43.210 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T11:15:43.210 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: kZlibCompression supported: 1 2024-04-25T11:15:43.210 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: kSnappyCompression supported: 1 2024-04-25T11:15:43.210 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: kLZ4Compression supported: 1 2024-04-25T11:15:43.210 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: kBZip2Compression supported: 0 2024-04-25T11:15:43.210 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T11:15:43.210 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T11:15:43.210 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-25T11:15:43.210 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T11:15:43.211 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T11:15:43.211 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.merge_operator: 2024-04-25T11:15:43.211 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.compaction_filter: None 2024-04-25T11:15:43.211 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T11:15:43.211 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T11:15:43.211 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T11:15:43.211 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T11:15:43.211 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x555af7ade1a0) 2024-04-25T11:15:43.211 INFO:journalctl@ceph.mon.a.smithi069.stdout: cache_index_and_filter_blocks: 1 2024-04-25T11:15:43.211 INFO:journalctl@ceph.mon.a.smithi069.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T11:15:43.211 INFO:journalctl@ceph.mon.a.smithi069.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T11:15:43.211 INFO:journalctl@ceph.mon.a.smithi069.stdout: pin_top_level_index_and_filter: 1 2024-04-25T11:15:43.212 INFO:journalctl@ceph.mon.a.smithi069.stdout: index_type: 0 2024-04-25T11:15:43.212 INFO:journalctl@ceph.mon.a.smithi069.stdout: data_block_index_type: 0 2024-04-25T11:15:43.212 INFO:journalctl@ceph.mon.a.smithi069.stdout: index_shortening: 1 2024-04-25T11:15:43.212 INFO:journalctl@ceph.mon.a.smithi069.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T11:15:43.212 INFO:journalctl@ceph.mon.a.smithi069.stdout: checksum: 4 2024-04-25T11:15:43.212 INFO:journalctl@ceph.mon.a.smithi069.stdout: no_block_cache: 0 2024-04-25T11:15:43.212 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_cache: 0x555af79db1f0 2024-04-25T11:15:43.212 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_cache_name: BinnedLRUCache 2024-04-25T11:15:43.212 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_cache_options: 2024-04-25T11:15:43.212 INFO:journalctl@ceph.mon.a.smithi069.stdout: capacity : 536870912 2024-04-25T11:15:43.212 INFO:journalctl@ceph.mon.a.smithi069.stdout: num_shard_bits : 4 2024-04-25T11:15:43.212 INFO:journalctl@ceph.mon.a.smithi069.stdout: strict_capacity_limit : 0 2024-04-25T11:15:43.212 INFO:journalctl@ceph.mon.a.smithi069.stdout: high_pri_pool_ratio: 0.000 2024-04-25T11:15:43.213 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_cache_compressed: (nil) 2024-04-25T11:15:43.213 INFO:journalctl@ceph.mon.a.smithi069.stdout: persistent_cache: (nil) 2024-04-25T11:15:43.213 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_size: 4096 2024-04-25T11:15:43.213 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_size_deviation: 10 2024-04-25T11:15:43.213 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_restart_interval: 16 2024-04-25T11:15:43.213 INFO:journalctl@ceph.mon.a.smithi069.stdout: index_block_restart_interval: 1 2024-04-25T11:15:43.213 INFO:journalctl@ceph.mon.a.smithi069.stdout: metadata_block_size: 4096 2024-04-25T11:15:43.213 INFO:journalctl@ceph.mon.a.smithi069.stdout: partition_filters: 0 2024-04-25T11:15:43.213 INFO:journalctl@ceph.mon.a.smithi069.stdout: use_delta_encoding: 1 2024-04-25T11:15:43.213 INFO:journalctl@ceph.mon.a.smithi069.stdout: filter_policy: bloomfilter 2024-04-25T11:15:43.213 INFO:journalctl@ceph.mon.a.smithi069.stdout: whole_key_filtering: 1 2024-04-25T11:15:43.213 INFO:journalctl@ceph.mon.a.smithi069.stdout: verify_compression: 0 2024-04-25T11:15:43.214 INFO:journalctl@ceph.mon.a.smithi069.stdout: read_amp_bytes_per_bit: 0 2024-04-25T11:15:43.214 INFO:journalctl@ceph.mon.a.smithi069.stdout: format_version: 5 2024-04-25T11:15:43.214 INFO:journalctl@ceph.mon.a.smithi069.stdout: enable_index_compression: 1 2024-04-25T11:15:43.214 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_align: 0 2024-04-25T11:15:43.214 INFO:journalctl@ceph.mon.a.smithi069.stdout: max_auto_readahead_size: 262144 2024-04-25T11:15:43.214 INFO:journalctl@ceph.mon.a.smithi069.stdout: prepopulate_block_cache: 0 2024-04-25T11:15:43.214 INFO:journalctl@ceph.mon.a.smithi069.stdout: initial_auto_readahead_size: 8192 2024-04-25T11:15:43.214 INFO:journalctl@ceph.mon.a.smithi069.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T11:15:43.214 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T11:15:43.214 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T11:15:43.214 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.compression: NoCompression 2024-04-25T11:15:43.214 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T11:15:43.214 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T11:15:43.215 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T11:15:43.215 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.num_levels: 7 2024-04-25T11:15:43.215 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T11:15:43.215 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T11:15:43.215 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T11:15:43.215 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T11:15:43.215 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T11:15:43.215 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T11:15:43.215 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T11:15:43.215 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T11:15:43.215 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T11:15:43.215 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T11:15:43.215 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T11:15:43.216 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T11:15:43.216 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T11:15:43.216 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T11:15:43.216 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T11:15:43.216 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T11:15:43.216 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T11:15:43.216 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T11:15:43.216 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T11:15:43.216 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T11:15:43.216 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T11:15:43.216 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T11:15:43.216 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T11:15:43.216 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T11:15:43.217 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T11:15:43.217 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T11:15:43.217 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T11:15:43.217 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T11:15:43.217 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T11:15:43.217 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T11:15:43.217 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T11:15:43.217 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T11:15:43.217 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T11:15:43.217 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T11:15:43.217 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T11:15:43.217 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T11:15:43.217 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T11:15:43.218 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T11:15:43.218 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T11:15:43.218 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T11:15:43.218 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T11:15:43.218 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T11:15:43.218 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T11:15:43.218 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T11:15:43.218 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T11:15:43.218 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T11:15:43.219 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T11:15:43.219 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T11:15:43.219 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T11:15:43.219 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T11:15:43.219 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T11:15:43.219 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T11:15:43.219 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T11:15:43.219 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T11:15:43.219 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.inplace_update_support: 0 2024-04-25T11:15:43.219 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T11:15:43.219 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T11:15:43.219 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T11:15:43.219 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T11:15:43.220 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.bloom_locality: 0 2024-04-25T11:15:43.220 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.max_successive_merges: 0 2024-04-25T11:15:43.220 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T11:15:43.220 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T11:15:43.220 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T11:15:43.220 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T11:15:43.220 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.ttl: 2592000 2024-04-25T11:15:43.220 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T11:15:43.220 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T11:15:43.220 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T11:15:43.221 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.enable_blob_files: false 2024-04-25T11:15:43.221 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.min_blob_size: 0 2024-04-25T11:15:43.221 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T11:15:43.221 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T11:15:43.221 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T11:15:43.221 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T11:15:43.221 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T11:15:43.221 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T11:15:43.221 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T11:15:43.221 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T11:15:43.221 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-25T11:15:43.221 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-25T11:15:43.221 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 63e62046-f310-4b61-9f0e-9aec045333ee 2024-04-25T11:15:43.221 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714043742941791, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-25T11:15:43.222 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-25T11:15:43.222 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714043742942941, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 98834, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 320, "table_properties": {"data_size": 96787, "index_size": 303, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13792, "raw_average_key_size": 49, "raw_value_size": 89229, "raw_average_value_size": 319, "num_data_blocks": 12, "num_entries": 279, "num_filter_entries": 279, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714043742, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "63e62046-f310-4b61-9f0e-9aec045333ee", "db_session_id": "HJS8137AIZ5KS0J80NF5", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-25T11:15:43.222 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714043742943025, "job": 1, "event": "recovery_finished"} 2024-04-25T11:15:43.222 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-25T11:15:43.222 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T11:15:43.222 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x555af79bfc00 2024-04-25T11:15:43.222 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: DB pointer 0x555afc7b2000 2024-04-25T11:15:43.222 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T11:15:43.222 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T11:15:43.222 INFO:journalctl@ceph.mon.a.smithi069.stdout: ** DB Stats ** 2024-04-25T11:15:43.222 INFO:journalctl@ceph.mon.a.smithi069.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T11:15:43.222 INFO:journalctl@ceph.mon.a.smithi069.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T11:15:43.222 INFO:journalctl@ceph.mon.a.smithi069.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T11:15:43.222 INFO:journalctl@ceph.mon.a.smithi069.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T11:15:43.223 INFO:journalctl@ceph.mon.a.smithi069.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T11:15:43.223 INFO:journalctl@ceph.mon.a.smithi069.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T11:15:43.223 INFO:journalctl@ceph.mon.a.smithi069.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T11:15:43.223 INFO:journalctl@ceph.mon.a.smithi069.stdout: 2024-04-25T11:15:43.223 INFO:journalctl@ceph.mon.a.smithi069.stdout: ** Compaction Stats [default] ** 2024-04-25T11:15:43.223 INFO:journalctl@ceph.mon.a.smithi069.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T11:15:43.223 INFO:journalctl@ceph.mon.a.smithi069.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T11:15:43.223 INFO:journalctl@ceph.mon.a.smithi069.stdout: L0 2/0 98.38 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 147.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T11:15:43.223 INFO:journalctl@ceph.mon.a.smithi069.stdout: Sum 2/0 98.38 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 147.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T11:15:43.223 INFO:journalctl@ceph.mon.a.smithi069.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 147.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T11:15:43.223 INFO:journalctl@ceph.mon.a.smithi069.stdout: 2024-04-25T11:15:43.223 INFO:journalctl@ceph.mon.a.smithi069.stdout: ** Compaction Stats [default] ** 2024-04-25T11:15:43.223 INFO:journalctl@ceph.mon.a.smithi069.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T11:15:43.223 INFO:journalctl@ceph.mon.a.smithi069.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T11:15:43.224 INFO:journalctl@ceph.mon.a.smithi069.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 147.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T11:15:43.224 INFO:journalctl@ceph.mon.a.smithi069.stdout: 2024-04-25T11:15:43.224 INFO:journalctl@ceph.mon.a.smithi069.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T11:15:43.224 INFO:journalctl@ceph.mon.a.smithi069.stdout: 2024-04-25T11:15:43.224 INFO:journalctl@ceph.mon.a.smithi069.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T11:15:43.224 INFO:journalctl@ceph.mon.a.smithi069.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T11:15:43.224 INFO:journalctl@ceph.mon.a.smithi069.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T11:15:43.224 INFO:journalctl@ceph.mon.a.smithi069.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T11:15:43.224 INFO:journalctl@ceph.mon.a.smithi069.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T11:15:43.224 INFO:journalctl@ceph.mon.a.smithi069.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T11:15:43.224 INFO:journalctl@ceph.mon.a.smithi069.stdout: Cumulative compaction: 0.00 GB write, 29.45 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T11:15:43.224 INFO:journalctl@ceph.mon.a.smithi069.stdout: Interval compaction: 0.00 GB write, 29.45 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T11:15:43.224 INFO:journalctl@ceph.mon.a.smithi069.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T11:15:43.224 INFO:journalctl@ceph.mon.a.smithi069.stdout: Block cache BinnedLRUCache@0x555af79db1f0#2 capacity: 512.00 MB usage: 1.41 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-25T11:15:43.225 INFO:journalctl@ceph.mon.a.smithi069.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.91 KB,0.000172853%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-25T11:15:43.225 INFO:journalctl@ceph.mon.a.smithi069.stdout: 2024-04-25T11:15:43.225 INFO:journalctl@ceph.mon.a.smithi069.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T11:15:43.225 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: starting mon.a rank 0 at public addrs [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] at bind addrs [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 2024-04-25T11:15:43.225 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: mon.a@-1(???) e1 preinit fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 2024-04-25T11:15:43.225 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: mon.a@-1(???).mds e1 new map 2024-04-25T11:15:43.225 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: mon.a@-1(???).mds e1 print_map 2024-04-25T11:15:43.225 INFO:journalctl@ceph.mon.a.smithi069.stdout: e1 2024-04-25T11:15:43.225 INFO:journalctl@ceph.mon.a.smithi069.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T11:15:43.225 INFO:journalctl@ceph.mon.a.smithi069.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T11:15:43.225 INFO:journalctl@ceph.mon.a.smithi069.stdout: legacy client fscid: -1 2024-04-25T11:15:43.225 INFO:journalctl@ceph.mon.a.smithi069.stdout: 2024-04-25T11:15:43.225 INFO:journalctl@ceph.mon.a.smithi069.stdout: No filesystems configured 2024-04-25T11:15:43.225 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T11:15:43.225 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T11:15:43.226 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T11:15:43.226 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T11:15:43.226 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:42 smithi069 ceph-mon[29188]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-25T11:15:43.226 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:43 smithi069 ceph-mon[29188]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T11:15:43.226 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:43 smithi069 ceph-mon[29188]: monmap epoch 1 2024-04-25T11:15:43.226 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:43 smithi069 ceph-mon[29188]: fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 2024-04-25T11:15:43.226 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:43 smithi069 ceph-mon[29188]: last_changed 2024-04-25T11:15:35.052240+0000 2024-04-25T11:15:43.226 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:43 smithi069 ceph-mon[29188]: created 2024-04-25T11:15:35.052240+0000 2024-04-25T11:15:43.226 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:43 smithi069 ceph-mon[29188]: min_mon_release 19 (squid) 2024-04-25T11:15:43.226 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:43 smithi069 ceph-mon[29188]: election_strategy: 1 2024-04-25T11:15:43.226 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:43 smithi069 ceph-mon[29188]: 0: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.a 2024-04-25T11:15:43.226 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:43 smithi069 ceph-mon[29188]: fsmap 2024-04-25T11:15:43.226 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:43 smithi069 ceph-mon[29188]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T11:15:43.227 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:43 smithi069 ceph-mon[29188]: mgrmap e1: no daemons active 2024-04-25T11:15:43.227 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:43 smithi069 bash[29169]: a2cd3fa6825851085799332cf869756d214713181871e6aa820ed8572268610b 2024-04-25T11:15:43.227 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:43 smithi069 systemd[1]: Started Ceph mon.a for f7ed73dc-02f4-11ef-bc93-c7b262605968. 2024-04-25T11:15:44.147 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:43.465+0000 7fa2d51c4700 1 Processor -- start 2024-04-25T11:15:44.147 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:43.465+0000 7fa2d51c4700 1 -- start start 2024-04-25T11:15:44.147 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:43.465+0000 7fa2d51c4700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2d007f640 0x7fa2d007db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:44.148 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:43.465+0000 7fa2d51c4700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d007e050 con 0x7fa2d007f640 2024-04-25T11:15:44.148 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:43.466+0000 7fa2ced9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2d007f640 0x7fa2d007db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:44.148 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:43.466+0000 7fa2ced9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2d007f640 0x7fa2d007db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53744/0 (socket says 172.21.15.69:53744) 2024-04-25T11:15:44.149 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:43.466+0000 7fa2ced9d700 1 -- 172.21.15.69:0/1871190397 learned_addr learned my addr 172.21.15.69:0/1871190397 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:15:44.149 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:43.466+0000 7fa2ced9d700 1 -- 172.21.15.69:0/1871190397 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d007e1d0 con 0x7fa2d007f640 2024-04-25T11:15:44.149 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:43.466+0000 7fa2ced9d700 1 --2- 172.21.15.69:0/1871190397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2d007f640 0x7fa2d007db10 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa2c00098a0 tx=0x7fa2c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=818eedf297163009 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:44.149 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:43.466+0000 7fa2ce59c700 1 -- 172.21.15.69:0/1871190397 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2c0004160 con 0x7fa2d007f640 2024-04-25T11:15:44.149 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:43.466+0000 7fa2ce59c700 1 -- 172.21.15.69:0/1871190397 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fa2c0004300 con 0x7fa2d007f640 2024-04-25T11:15:44.149 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:43.466+0000 7fa2ce59c700 1 -- 172.21.15.69:0/1871190397 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2c00119e0 con 0x7fa2d007f640 2024-04-25T11:15:44.150 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:43.467+0000 7fa2d51c4700 1 -- 172.21.15.69:0/1871190397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2d007f640 msgr2=0x7fa2d007db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:44.150 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:43.467+0000 7fa2d51c4700 1 --2- 172.21.15.69:0/1871190397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2d007f640 0x7fa2d007db10 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa2c00098a0 tx=0x7fa2c0009bb0 comp rx=0 tx=0).stop 2024-04-25T11:15:44.150 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:43.467+0000 7fa2d51c4700 1 -- 172.21.15.69:0/1871190397 shutdown_connections 2024-04-25T11:15:44.150 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:43.467+0000 7fa2d51c4700 1 --2- 172.21.15.69:0/1871190397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2d007f640 0x7fa2d007db10 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:44.150 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:43.467+0000 7fa2d51c4700 1 -- 172.21.15.69:0/1871190397 >> 172.21.15.69:0/1871190397 conn(0x7fa2d0104f60 msgr2=0x7fa2d0107410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:44.150 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:43.467+0000 7fa2d51c4700 1 -- 172.21.15.69:0/1871190397 shutdown_connections 2024-04-25T11:15:44.150 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:43.467+0000 7fa2d51c4700 1 -- 172.21.15.69:0/1871190397 wait complete. 2024-04-25T11:15:44.150 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:43.468+0000 7fa2d51c4700 1 Processor -- start 2024-04-25T11:15:44.150 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:43.468+0000 7fa2d51c4700 1 -- start start 2024-04-25T11:15:44.150 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:43.468+0000 7fa2d51c4700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2d007f640 0x7fa2d019fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:44.150 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:43.468+0000 7fa2d51c4700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d010fdb0 con 0x7fa2d007f640 2024-04-25T11:15:44.150 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:43.468+0000 7fa2ced9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2d007f640 0x7fa2d019fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:44.150 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:43.468+0000 7fa2ced9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2d007f640 0x7fa2d019fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53746/0 (socket says 172.21.15.69:53746) 2024-04-25T11:15:44.151 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:43.468+0000 7fa2ced9d700 1 -- 172.21.15.69:0/1173860025 learned_addr learned my addr 172.21.15.69:0/1173860025 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:15:44.151 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:43.469+0000 7fa2ced9d700 1 -- 172.21.15.69:0/1173860025 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d01a01c0 con 0x7fa2d007f640 2024-04-25T11:15:44.151 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:43.469+0000 7fa2ced9d700 1 --2- 172.21.15.69:0/1173860025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2d007f640 0x7fa2d019fc80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0003f20 tx=0x7fa2c0011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:44.151 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:43.469+0000 7fa2ccd99700 1 -- 172.21.15.69:0/1173860025 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2c00038b0 con 0x7fa2d007f640 2024-04-25T11:15:44.151 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:43.469+0000 7fa2d51c4700 1 -- 172.21.15.69:0/1173860025 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2d01a0450 con 0x7fa2d007f640 2024-04-25T11:15:44.151 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:43.469+0000 7fa2d51c4700 1 -- 172.21.15.69:0/1173860025 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2d01a3060 con 0x7fa2d007f640 2024-04-25T11:15:44.151 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:43.469+0000 7fa2ccd99700 1 -- 172.21.15.69:0/1173860025 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fa2c0003a50 con 0x7fa2d007f640 2024-04-25T11:15:44.151 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:43.469+0000 7fa2ccd99700 1 -- 172.21.15.69:0/1173860025 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2c0019550 con 0x7fa2d007f640 2024-04-25T11:15:44.151 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:43.469+0000 7fa2ccd99700 1 -- 172.21.15.69:0/1173860025 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa2c0019bf0 con 0x7fa2d007f640 2024-04-25T11:15:44.151 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:43.469+0000 7fa2ccd99700 1 -- 172.21.15.69:0/1173860025 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa2c0024730 con 0x7fa2d007f640 2024-04-25T11:15:44.151 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:43.470+0000 7fa2d51c4700 1 -- 172.21.15.69:0/1173860025 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2b0005360 con 0x7fa2d007f640 2024-04-25T11:15:44.151 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:43.472+0000 7fa2ccd99700 1 -- 172.21.15.69:0/1173860025 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fa2c0043e50 con 0x7fa2d007f640 2024-04-25T11:15:44.151 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:43.542+0000 7fa2d51c4700 1 -- 172.21.15.69:0/1173860025 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fa2b0005650 con 0x7fa2d007f640 2024-04-25T11:15:44.151 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:43.545+0000 7fa2ccd99700 1 -- 172.21.15.69:0/1173860025 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fa2c0032030 con 0x7fa2d007f640 2024-04-25T11:15:44.151 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:43.546+0000 7fa2d51c4700 1 -- 172.21.15.69:0/1173860025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2d007f640 msgr2=0x7fa2d019fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:44.152 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:43.546+0000 7fa2d51c4700 1 --2- 172.21.15.69:0/1173860025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2d007f640 0x7fa2d019fc80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0003f20 tx=0x7fa2c0011a60 comp rx=0 tx=0).stop 2024-04-25T11:15:44.152 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:43.546+0000 7fa2d51c4700 1 -- 172.21.15.69:0/1173860025 shutdown_connections 2024-04-25T11:15:44.152 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:43.546+0000 7fa2d51c4700 1 --2- 172.21.15.69:0/1173860025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2d007f640 0x7fa2d019fc80 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:44.152 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:43.546+0000 7fa2d51c4700 1 -- 172.21.15.69:0/1173860025 >> 172.21.15.69:0/1173860025 conn(0x7fa2d0104f60 msgr2=0x7fa2d01057a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:44.152 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:43.546+0000 7fa2d51c4700 1 -- 172.21.15.69:0/1173860025 shutdown_connections 2024-04-25T11:15:44.152 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:43.546+0000 7fa2d51c4700 1 -- 172.21.15.69:0/1173860025 wait complete. 2024-04-25T11:15:44.152 INFO:teuthology.orchestra.run.smithi069.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-25T11:15:44.172 INFO:teuthology.orchestra.run.smithi069.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-25T11:15:44.173 INFO:teuthology.orchestra.run.smithi069.stdout:Creating mgr... 2024-04-25T11:15:44.173 INFO:teuthology.orchestra.run.smithi069.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-25T11:15:44.173 INFO:teuthology.orchestra.run.smithi069.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-25T11:15:44.417 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mgr.x 2024-04-25T11:15:44.418 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Failed to reset failed state of unit ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-25T11:15:44.562 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968.target.wants/ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@.service. 2024-04-25T11:15:44.700 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:44 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/1173860025' entity='client.admin' 2024-04-25T11:15:44.700 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:15:44 smithi069 systemd[1]: Starting Ceph mgr.x for f7ed73dc-02f4-11ef-bc93-c7b262605968... 2024-04-25T11:15:45.048 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T11:15:45.049 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout disabled 2024-04-25T11:15:45.056 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T11:15:45.056 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout inactive 2024-04-25T11:15:45.056 INFO:teuthology.orchestra.run.smithi069.stdout:firewalld.service is not enabled 2024-04-25T11:15:45.056 INFO:teuthology.orchestra.run.smithi069.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T11:15:45.064 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T11:15:45.064 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout disabled 2024-04-25T11:15:45.071 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T11:15:45.071 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout inactive 2024-04-25T11:15:45.071 INFO:teuthology.orchestra.run.smithi069.stdout:firewalld.service is not enabled 2024-04-25T11:15:45.071 INFO:teuthology.orchestra.run.smithi069.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-25T11:15:45.071 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for mgr to start... 2024-04-25T11:15:45.071 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for mgr... 2024-04-25T11:15:45.147 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:15:44 smithi069 podman[29403]: 2024-04-25 11:15:44.699527241 +0000 UTC m=+0.017968962 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T11:15:45.147 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:15:44 smithi069 podman[29403]: 2024-04-25 11:15:44.800429581 +0000 UTC m=+0.118871285 container create 447f3c9a8cd80fcf7f87c60baededff0c15a2ac69f348c2b27a39408a6957124 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0) 2024-04-25T11:15:45.147 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:15:44 smithi069 podman[29403]: 2024-04-25 11:15:44.969450201 +0000 UTC m=+0.287891905 container init 447f3c9a8cd80fcf7f87c60baededff0c15a2ac69f348c2b27a39408a6957124 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T11:15:45.147 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:15:44 smithi069 podman[29403]: 2024-04-25 11:15:44.971836063 +0000 UTC m=+0.290277768 container start 447f3c9a8cd80fcf7f87c60baededff0c15a2ac69f348c2b27a39408a6957124 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T11:15:45.147 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:15:45 smithi069 bash[29403]: 447f3c9a8cd80fcf7f87c60baededff0c15a2ac69f348c2b27a39408a6957124 2024-04-25T11:15:45.147 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:15:45 smithi069 systemd[1]: Started Ceph mgr.x for f7ed73dc-02f4-11ef-bc93-c7b262605968. 2024-04-25T11:15:45.147 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:15:45 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:15:45.146+0000 7f3477cbd200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T11:15:45.440 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:15:45 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:15:45.335+0000 7f3477cbd200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:15:45.441 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:15:45 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:15:45.440+0000 7f3477cbd200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T11:15:45.699 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:45 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2361717010' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T11:15:46.125 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:15:45 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:15:45.764+0000 7f3477cbd200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T11:15:46.125 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:15:45 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:15:45.851+0000 7f3477cbd200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T11:15:46.128 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-04-25T11:15:46.139 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout { 2024-04-25T11:15:46.139 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "fsid": "f7ed73dc-02f4-11ef-bc93-c7b262605968", 2024-04-25T11:15:46.139 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T11:15:46.139 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T11:15:46.139 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T11:15:46.139 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T11:15:46.139 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:15:46.139 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T11:15:46.139 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T11:15:46.140 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 0 2024-04-25T11:15:46.140 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-04-25T11:15:46.140 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T11:15:46.140 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T11:15:46.140 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-04-25T11:15:46.140 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-25T11:15:46.140 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T11:15:46.140 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:15:46.140 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T11:15:46.140 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T11:15:46.140 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:15:46.140 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T11:15:46.140 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:15:46.140 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T11:15:46.141 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T11:15:46.141 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T11:15:46.141 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T11:15:46.141 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T11:15:46.141 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T11:15:46.141 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:15:46.141 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T11:15:46.141 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T11:15:46.141 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T11:15:46.141 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T11:15:46.141 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T11:15:46.141 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T11:15:46.141 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T11:15:46.141 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T11:15:46.142 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T11:15:46.142 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:15:46.142 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T11:15:46.142 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:15:46.142 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T11:15:46.142 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T11:15:46.142 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:15:46.142 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T11:15:46.142 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T11:15:46.142 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T11:15:46.142 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T11:15:46.142 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T11:15:46.142 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T11:15:46.143 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T11:15:46.143 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-04-25T11:15:46.143 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T11:15:46.143 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:15:46.143 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T11:15:46.143 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:15:46.143 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T11:15:37.965874+0000", 2024-04-25T11:15:46.143 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T11:15:46.143 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:15:46.143 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T11:15:46.143 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout } 2024-04-25T11:15:46.143 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:45.483+0000 7fdcf70ed700 1 Processor -- start 2024-04-25T11:15:46.143 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:45.483+0000 7fdcf70ed700 1 -- start start 2024-04-25T11:15:46.143 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:45.483+0000 7fdcf70ed700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcf007ccd0 0x7fdcf010e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:46.144 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:45.483+0000 7fdcf70ed700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf010ed70 con 0x7fdcf007ccd0 2024-04-25T11:15:46.144 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:45.484+0000 7fdcf4e89700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcf007ccd0 0x7fdcf010e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:46.144 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:45.484+0000 7fdcf4e89700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcf007ccd0 0x7fdcf010e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53776/0 (socket says 172.21.15.69:53776) 2024-04-25T11:15:46.144 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:45.484+0000 7fdcf4e89700 1 -- 172.21.15.69:0/2661800916 learned_addr learned my addr 172.21.15.69:0/2661800916 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:15:46.144 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:45.484+0000 7fdcf4e89700 1 -- 172.21.15.69:0/2661800916 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf010eef0 con 0x7fdcf007ccd0 2024-04-25T11:15:46.144 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:45.484+0000 7fdcf4e89700 1 --2- 172.21.15.69:0/2661800916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcf007ccd0 0x7fdcf010e830 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdce40098a0 tx=0x7fdce4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d56614dcec3bc5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:46.144 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:45.484+0000 7fdcef7fe700 1 -- 172.21.15.69:0/2661800916 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdce4004160 con 0x7fdcf007ccd0 2024-04-25T11:15:46.144 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:45.484+0000 7fdcef7fe700 1 -- 172.21.15.69:0/2661800916 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fdce4004300 con 0x7fdcf007ccd0 2024-04-25T11:15:46.144 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:45.484+0000 7fdcef7fe700 1 -- 172.21.15.69:0/2661800916 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdce40119e0 con 0x7fdcf007ccd0 2024-04-25T11:15:46.144 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:45.486+0000 7fdcf70ed700 1 -- 172.21.15.69:0/2661800916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcf007ccd0 msgr2=0x7fdcf010e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:46.144 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:45.486+0000 7fdcf70ed700 1 --2- 172.21.15.69:0/2661800916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcf007ccd0 0x7fdcf010e830 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdce40098a0 tx=0x7fdce4009bb0 comp rx=0 tx=0).stop 2024-04-25T11:15:46.144 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:45.486+0000 7fdcf70ed700 1 -- 172.21.15.69:0/2661800916 shutdown_connections 2024-04-25T11:15:46.144 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:45.486+0000 7fdcf70ed700 1 --2- 172.21.15.69:0/2661800916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcf007ccd0 0x7fdcf010e830 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:46.145 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:45.486+0000 7fdcf70ed700 1 -- 172.21.15.69:0/2661800916 >> 172.21.15.69:0/2661800916 conn(0x7fdcf0103ee0 msgr2=0x7fdcf0106330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:46.145 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:45.486+0000 7fdcf70ed700 1 -- 172.21.15.69:0/2661800916 shutdown_connections 2024-04-25T11:15:46.145 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:45.486+0000 7fdcf70ed700 1 -- 172.21.15.69:0/2661800916 wait complete. 2024-04-25T11:15:46.145 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:45.486+0000 7fdcf70ed700 1 Processor -- start 2024-04-25T11:15:46.145 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:45.486+0000 7fdcf70ed700 1 -- start start 2024-04-25T11:15:46.145 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:45.487+0000 7fdcf70ed700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcf007ccd0 0x7fdcf019fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:46.145 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:45.487+0000 7fdcf70ed700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf010fd80 con 0x7fdcf007ccd0 2024-04-25T11:15:46.145 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:45.487+0000 7fdcf4e89700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcf007ccd0 0x7fdcf019fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:46.145 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:45.487+0000 7fdcf4e89700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcf007ccd0 0x7fdcf019fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53778/0 (socket says 172.21.15.69:53778) 2024-04-25T11:15:46.145 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:45.487+0000 7fdcf4e89700 1 -- 172.21.15.69:0/2361717010 learned_addr learned my addr 172.21.15.69:0/2361717010 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:15:46.145 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:45.487+0000 7fdcf4e89700 1 -- 172.21.15.69:0/2361717010 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf01a0230 con 0x7fdcf007ccd0 2024-04-25T11:15:46.145 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:45.487+0000 7fdcf4e89700 1 --2- 172.21.15.69:0/2361717010 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcf007ccd0 0x7fdcf019fcf0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdce40116a0 tx=0x7fdce4011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:46.145 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:45.487+0000 7fdcedffb700 1 -- 172.21.15.69:0/2361717010 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdce40039f0 con 0x7fdcf007ccd0 2024-04-25T11:15:46.145 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:45.487+0000 7fdcedffb700 1 -- 172.21.15.69:0/2361717010 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fdce4003b90 con 0x7fdcf007ccd0 2024-04-25T11:15:46.146 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:45.487+0000 7fdcf70ed700 1 -- 172.21.15.69:0/2361717010 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcf01a04c0 con 0x7fdcf007ccd0 2024-04-25T11:15:46.146 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:45.487+0000 7fdcedffb700 1 -- 172.21.15.69:0/2361717010 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdce401a600 con 0x7fdcf007ccd0 2024-04-25T11:15:46.146 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:45.488+0000 7fdcf70ed700 1 -- 172.21.15.69:0/2361717010 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcf01a30d0 con 0x7fdcf007ccd0 2024-04-25T11:15:46.146 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:45.488+0000 7fdcedffb700 1 -- 172.21.15.69:0/2361717010 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdce401ac60 con 0x7fdcf007ccd0 2024-04-25T11:15:46.146 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:45.488+0000 7fdcedffb700 1 -- 172.21.15.69:0/2361717010 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdce4019a50 con 0x7fdcf007ccd0 2024-04-25T11:15:46.146 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:45.489+0000 7fdcf70ed700 1 -- 172.21.15.69:0/2361717010 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcf0064cf0 con 0x7fdcf007ccd0 2024-04-25T11:15:46.146 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:45.493+0000 7fdcedffb700 1 -- 172.21.15.69:0/2361717010 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fdce4019cf0 con 0x7fdcf007ccd0 2024-04-25T11:15:46.146 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:45.564+0000 7fdcf70ed700 1 -- 172.21.15.69:0/2361717010 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fdcf01a32a0 con 0x7fdcf007ccd0 2024-04-25T11:15:46.146 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:45.564+0000 7fdcedffb700 1 -- 172.21.15.69:0/2361717010 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fdce4024a50 con 0x7fdcf007ccd0 2024-04-25T11:15:46.146 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:45.565+0000 7fdcf70ed700 1 -- 172.21.15.69:0/2361717010 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcf007ccd0 msgr2=0x7fdcf019fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:46.146 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:45.565+0000 7fdcf70ed700 1 --2- 172.21.15.69:0/2361717010 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcf007ccd0 0x7fdcf019fcf0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdce40116a0 tx=0x7fdce4011a60 comp rx=0 tx=0).stop 2024-04-25T11:15:46.146 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:45.565+0000 7fdcf70ed700 1 -- 172.21.15.69:0/2361717010 shutdown_connections 2024-04-25T11:15:46.146 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:45.565+0000 7fdcf70ed700 1 --2- 172.21.15.69:0/2361717010 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcf007ccd0 0x7fdcf019fcf0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:46.147 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:45.565+0000 7fdcf70ed700 1 -- 172.21.15.69:0/2361717010 >> 172.21.15.69:0/2361717010 conn(0x7fdcf0103ee0 msgr2=0x7fdcf0105860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:46.147 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:45.566+0000 7fdcf70ed700 1 -- 172.21.15.69:0/2361717010 shutdown_connections 2024-04-25T11:15:46.147 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:45.566+0000 7fdcf70ed700 1 -- 172.21.15.69:0/2361717010 wait complete. 2024-04-25T11:15:46.147 INFO:teuthology.orchestra.run.smithi069.stdout:mgr not available, waiting (1/15)... 2024-04-25T11:15:47.384 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:15:47 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:15:47.036+0000 7f3477cbd200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T11:15:47.384 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:15:47 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:15:47.214+0000 7f3477cbd200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T11:15:47.384 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:15:47 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:15:47.383+0000 7f3477cbd200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T11:15:47.949 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:15:47 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:15:47.669+0000 7f3477cbd200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T11:15:47.950 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:15:47 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:15:47.761+0000 7f3477cbd200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T11:15:48.258 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:15:47 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:15:47.963+0000 7f3477cbd200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T11:15:48.259 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:15:48 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:15:48.153+0000 7f3477cbd200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:15:48.949 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:48 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/978322772' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T11:15:48.950 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:15:48 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:15:48.771+0000 7f3477cbd200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T11:15:49.321 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-04-25T11:15:49.321 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout { 2024-04-25T11:15:49.321 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "fsid": "f7ed73dc-02f4-11ef-bc93-c7b262605968", 2024-04-25T11:15:49.321 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T11:15:49.321 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T11:15:49.321 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T11:15:49.321 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T11:15:49.322 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:15:49.322 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T11:15:49.322 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T11:15:49.322 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 0 2024-04-25T11:15:49.322 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-04-25T11:15:49.322 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T11:15:49.322 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T11:15:49.322 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-04-25T11:15:49.322 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-04-25T11:15:49.322 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T11:15:49.323 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:15:49.323 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T11:15:49.323 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T11:15:49.323 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:15:49.323 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T11:15:49.323 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:15:49.323 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T11:15:49.323 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T11:15:49.323 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T11:15:49.333 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T11:15:49.333 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T11:15:49.333 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T11:15:49.333 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:15:49.333 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T11:15:49.333 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T11:15:49.333 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T11:15:49.334 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T11:15:49.334 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T11:15:49.334 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T11:15:49.334 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T11:15:49.334 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T11:15:49.334 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T11:15:49.334 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:15:49.334 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T11:15:49.334 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:15:49.335 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T11:15:49.335 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T11:15:49.335 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:15:49.335 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T11:15:49.335 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T11:15:49.335 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T11:15:49.335 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T11:15:49.335 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T11:15:49.335 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T11:15:49.335 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T11:15:49.335 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-04-25T11:15:49.336 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T11:15:49.336 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:15:49.336 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T11:15:49.336 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:15:49.336 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T11:15:37.965874+0000", 2024-04-25T11:15:49.336 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T11:15:49.336 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:15:49.336 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T11:15:49.336 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout } 2024-04-25T11:15:49.336 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:48.578+0000 7fa8cb8f1700 1 Processor -- start 2024-04-25T11:15:49.336 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:48.578+0000 7fa8cb8f1700 1 -- start start 2024-04-25T11:15:49.336 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:48.578+0000 7fa8cb8f1700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8c4108820 0x7fa8c4108c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:49.337 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:48.578+0000 7fa8cb8f1700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c4109220 con 0x7fa8c4108820 2024-04-25T11:15:49.337 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:48.579+0000 7fa8c968d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8c4108820 0x7fa8c4108c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:49.337 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:48.579+0000 7fa8c968d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8c4108820 0x7fa8c4108c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53790/0 (socket says 172.21.15.69:53790) 2024-04-25T11:15:49.337 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:48.579+0000 7fa8c968d700 1 -- 172.21.15.69:0/3225838125 learned_addr learned my addr 172.21.15.69:0/3225838125 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:15:49.337 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:48.579+0000 7fa8c968d700 1 -- 172.21.15.69:0/3225838125 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8c4109950 con 0x7fa8c4108820 2024-04-25T11:15:49.337 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:48.579+0000 7fa8c968d700 1 --2- 172.21.15.69:0/3225838125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8c4108820 0x7fa8c4108c30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa8b40098a0 tx=0x7fa8b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52216d9058bbf81d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:49.337 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:48.579+0000 7fa8bbfff700 1 -- 172.21.15.69:0/3225838125 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8b4004160 con 0x7fa8c4108820 2024-04-25T11:15:49.337 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:48.579+0000 7fa8bbfff700 1 -- 172.21.15.69:0/3225838125 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fa8b4004300 con 0x7fa8c4108820 2024-04-25T11:15:49.337 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:48.579+0000 7fa8bbfff700 1 -- 172.21.15.69:0/3225838125 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8b40119e0 con 0x7fa8c4108820 2024-04-25T11:15:49.337 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:48.580+0000 7fa8cb8f1700 1 -- 172.21.15.69:0/3225838125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8c4108820 msgr2=0x7fa8c4108c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:49.337 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:48.580+0000 7fa8cb8f1700 1 --2- 172.21.15.69:0/3225838125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8c4108820 0x7fa8c4108c30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa8b40098a0 tx=0x7fa8b4009bb0 comp rx=0 tx=0).stop 2024-04-25T11:15:49.338 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:48.580+0000 7fa8cb8f1700 1 -- 172.21.15.69:0/3225838125 shutdown_connections 2024-04-25T11:15:49.338 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:48.580+0000 7fa8cb8f1700 1 --2- 172.21.15.69:0/3225838125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8c4108820 0x7fa8c4108c30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:49.338 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:48.580+0000 7fa8cb8f1700 1 -- 172.21.15.69:0/3225838125 >> 172.21.15.69:0/3225838125 conn(0x7fa8c4103f20 msgr2=0x7fa8c4106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:49.338 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:48.580+0000 7fa8cb8f1700 1 -- 172.21.15.69:0/3225838125 shutdown_connections 2024-04-25T11:15:49.338 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:48.580+0000 7fa8cb8f1700 1 -- 172.21.15.69:0/3225838125 wait complete. 2024-04-25T11:15:49.338 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:48.581+0000 7fa8cb8f1700 1 Processor -- start 2024-04-25T11:15:49.338 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:48.581+0000 7fa8cb8f1700 1 -- start start 2024-04-25T11:15:49.338 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:48.581+0000 7fa8cb8f1700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8c4108820 0x7fa8c419db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:49.338 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:48.581+0000 7fa8cb8f1700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c407bff0 con 0x7fa8c4108820 2024-04-25T11:15:49.338 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:48.581+0000 7fa8c968d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8c4108820 0x7fa8c419db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:49.338 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:48.581+0000 7fa8c968d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8c4108820 0x7fa8c419db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53800/0 (socket says 172.21.15.69:53800) 2024-04-25T11:15:49.338 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:48.581+0000 7fa8c968d700 1 -- 172.21.15.69:0/978322772 learned_addr learned my addr 172.21.15.69:0/978322772 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:15:49.339 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:48.581+0000 7fa8c968d700 1 -- 172.21.15.69:0/978322772 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8c419e0b0 con 0x7fa8c4108820 2024-04-25T11:15:49.339 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:48.582+0000 7fa8c968d700 1 --2- 172.21.15.69:0/978322772 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8c4108820 0x7fa8c419db70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa8b40116a0 tx=0x7fa8b4011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:49.339 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:48.582+0000 7fa8ba7fc700 1 -- 172.21.15.69:0/978322772 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8b40038b0 con 0x7fa8c4108820 2024-04-25T11:15:49.339 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:48.582+0000 7fa8ba7fc700 1 -- 172.21.15.69:0/978322772 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fa8b4003a50 con 0x7fa8c4108820 2024-04-25T11:15:49.339 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:48.582+0000 7fa8cb8f1700 1 -- 172.21.15.69:0/978322772 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8c419e340 con 0x7fa8c4108820 2024-04-25T11:15:49.339 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:48.582+0000 7fa8ba7fc700 1 -- 172.21.15.69:0/978322772 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8b4019550 con 0x7fa8c4108820 2024-04-25T11:15:49.339 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:48.582+0000 7fa8cb8f1700 1 -- 172.21.15.69:0/978322772 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8c41a0f50 con 0x7fa8c4108820 2024-04-25T11:15:49.339 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:48.582+0000 7fa8ba7fc700 1 -- 172.21.15.69:0/978322772 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa8b4019bb0 con 0x7fa8c4108820 2024-04-25T11:15:49.339 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:48.582+0000 7fa8ba7fc700 1 -- 172.21.15.69:0/978322772 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa8b4024730 con 0x7fa8c4108820 2024-04-25T11:15:49.339 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:48.583+0000 7fa8cb8f1700 1 -- 172.21.15.69:0/978322772 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8c4064cf0 con 0x7fa8c4108820 2024-04-25T11:15:49.339 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:48.585+0000 7fa8ba7fc700 1 -- 172.21.15.69:0/978322772 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fa8b4043e50 con 0x7fa8c4108820 2024-04-25T11:15:49.340 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:48.658+0000 7fa8cb8f1700 1 -- 172.21.15.69:0/978322772 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa8c402e810 con 0x7fa8c4108820 2024-04-25T11:15:49.340 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:48.659+0000 7fa8ba7fc700 1 -- 172.21.15.69:0/978322772 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fa8b4032020 con 0x7fa8c4108820 2024-04-25T11:15:49.340 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:48.660+0000 7fa8cb8f1700 1 -- 172.21.15.69:0/978322772 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8c4108820 msgr2=0x7fa8c419db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:49.340 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:48.660+0000 7fa8cb8f1700 1 --2- 172.21.15.69:0/978322772 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8c4108820 0x7fa8c419db70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa8b40116a0 tx=0x7fa8b4011a60 comp rx=0 tx=0).stop 2024-04-25T11:15:49.340 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:48.660+0000 7fa8cb8f1700 1 -- 172.21.15.69:0/978322772 shutdown_connections 2024-04-25T11:15:49.340 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:48.660+0000 7fa8cb8f1700 1 --2- 172.21.15.69:0/978322772 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8c4108820 0x7fa8c419db70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:49.340 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:48.660+0000 7fa8cb8f1700 1 -- 172.21.15.69:0/978322772 >> 172.21.15.69:0/978322772 conn(0x7fa8c4103f20 msgr2=0x7fa8c4106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:49.340 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:48.660+0000 7fa8cb8f1700 1 -- 172.21.15.69:0/978322772 shutdown_connections 2024-04-25T11:15:49.340 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:48.660+0000 7fa8cb8f1700 1 -- 172.21.15.69:0/978322772 wait complete. 2024-04-25T11:15:49.340 INFO:teuthology.orchestra.run.smithi069.stdout:mgr not available, waiting (2/15)... 2024-04-25T11:15:49.449 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:15:49 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:15:49.195+0000 7f3477cbd200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T11:15:49.450 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:15:49 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:15:49.292+0000 7f3477cbd200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T11:15:49.949 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:15:49 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:15:49.494+0000 7f3477cbd200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T11:15:50.349 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:15:50 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:15:50.007+0000 7f3477cbd200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T11:15:50.349 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:15:50 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:15:50.101+0000 7f3477cbd200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T11:15:50.698 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:15:50 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:15:50.349+0000 7f3477cbd200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T11:15:50.698 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:15:50 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:15:50.484+0000 7f3477cbd200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T11:15:50.698 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:15:50 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:15:50.580+0000 7f3477cbd200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T11:15:50.698 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:15:50 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:15:50.697+0000 7f3477cbd200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T11:15:51.445 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:15:51 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:15:51.017+0000 7f3477cbd200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T11:15:51.699 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:15:51 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:15:51.628+0000 7f3477cbd200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T11:15:52.200 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:15:51 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:15:51.855+0000 7f3477cbd200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T11:15:52.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:51 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2970329823' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T11:15:52.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:51 smithi069 ceph-mon[29188]: Activating manager daemon x 2024-04-25T11:15:52.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:51 smithi069 ceph-mon[29188]: mgrmap e2: x(active, starting, since 0.00372118s) 2024-04-25T11:15:52.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:51 smithi069 ceph-mon[29188]: from='mgr.14100 172.21.15.69:0/4178546609' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T11:15:52.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:51 smithi069 ceph-mon[29188]: from='mgr.14100 172.21.15.69:0/4178546609' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T11:15:52.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:51 smithi069 ceph-mon[29188]: from='mgr.14100 172.21.15.69:0/4178546609' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T11:15:52.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:51 smithi069 ceph-mon[29188]: from='mgr.14100 172.21.15.69:0/4178546609' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T11:15:52.201 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:51 smithi069 ceph-mon[29188]: from='mgr.14100 172.21.15.69:0/4178546609' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T11:15:52.201 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:51 smithi069 ceph-mon[29188]: Manager daemon x is now available 2024-04-25T11:15:52.201 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:51 smithi069 ceph-mon[29188]: from='mgr.14100 172.21.15.69:0/4178546609' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T11:15:52.201 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:51 smithi069 ceph-mon[29188]: from='mgr.14100 172.21.15.69:0/4178546609' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T11:15:52.201 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:51 smithi069 ceph-mon[29188]: from='mgr.14100 172.21.15.69:0/4178546609' entity='mgr.x' 2024-04-25T11:15:52.201 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:51 smithi069 ceph-mon[29188]: from='mgr.14100 172.21.15.69:0/4178546609' entity='mgr.x' 2024-04-25T11:15:52.453 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-04-25T11:15:52.453 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout { 2024-04-25T11:15:52.453 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "fsid": "f7ed73dc-02f4-11ef-bc93-c7b262605968", 2024-04-25T11:15:52.453 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T11:15:52.453 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T11:15:52.453 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T11:15:52.453 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T11:15:52.453 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:15:52.454 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T11:15:52.454 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T11:15:52.454 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 0 2024-04-25T11:15:52.454 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-04-25T11:15:52.454 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T11:15:52.454 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T11:15:52.454 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-04-25T11:15:52.454 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-04-25T11:15:52.454 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T11:15:52.454 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:15:52.454 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T11:15:52.455 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T11:15:52.455 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:15:52.455 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T11:15:52.455 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:15:52.455 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T11:15:52.455 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T11:15:52.455 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T11:15:52.455 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T11:15:52.455 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T11:15:52.455 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T11:15:52.455 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:15:52.455 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T11:15:52.455 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T11:15:52.455 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T11:15:52.456 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T11:15:52.456 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T11:15:52.456 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T11:15:52.456 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T11:15:52.456 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T11:15:52.456 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T11:15:52.456 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:15:52.456 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T11:15:52.456 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:15:52.456 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T11:15:52.456 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T11:15:52.456 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:15:52.456 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T11:15:52.457 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T11:15:52.457 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T11:15:52.457 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T11:15:52.457 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T11:15:52.457 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T11:15:52.457 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T11:15:52.457 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-04-25T11:15:52.457 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T11:15:52.457 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:15:52.457 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T11:15:52.457 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:15:52.457 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T11:15:37.965874+0000", 2024-04-25T11:15:52.457 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T11:15:52.458 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:15:52.458 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T11:15:52.458 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout } 2024-04-25T11:15:52.458 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:51.768+0000 7f2ed7805700 1 Processor -- start 2024-04-25T11:15:52.458 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:51.768+0000 7f2ed7805700 1 -- start start 2024-04-25T11:15:52.458 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:51.768+0000 7f2ed7805700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ed0109dc0 0x7f2ed010a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:52.458 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:51.768+0000 7f2ed7805700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed010a7a0 con 0x7f2ed0109dc0 2024-04-25T11:15:52.458 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:51.768+0000 7f2ed55a1700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ed0109dc0 0x7f2ed010a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:52.458 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:51.768+0000 7f2ed55a1700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ed0109dc0 0x7f2ed010a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53808/0 (socket says 172.21.15.69:53808) 2024-04-25T11:15:52.458 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:51.768+0000 7f2ed55a1700 1 -- 172.21.15.69:0/1424701949 learned_addr learned my addr 172.21.15.69:0/1424701949 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:15:52.458 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:51.769+0000 7f2ed55a1700 1 -- 172.21.15.69:0/1424701949 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed010a920 con 0x7f2ed0109dc0 2024-04-25T11:15:52.458 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:51.769+0000 7f2ed55a1700 1 --2- 172.21.15.69:0/1424701949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ed0109dc0 0x7f2ed010a1d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc0098a0 tx=0x7f2ecc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2deb3eda272a6b6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:52.458 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:51.769+0000 7f2ec7fff700 1 -- 172.21.15.69:0/1424701949 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ecc004160 con 0x7f2ed0109dc0 2024-04-25T11:15:52.458 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:51.769+0000 7f2ec7fff700 1 -- 172.21.15.69:0/1424701949 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f2ecc004300 con 0x7f2ed0109dc0 2024-04-25T11:15:52.459 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:51.769+0000 7f2ec7fff700 1 -- 172.21.15.69:0/1424701949 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ecc0119e0 con 0x7f2ed0109dc0 2024-04-25T11:15:52.459 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:51.770+0000 7f2ed7805700 1 -- 172.21.15.69:0/1424701949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ed0109dc0 msgr2=0x7f2ed010a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:52.459 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:51.770+0000 7f2ed7805700 1 --2- 172.21.15.69:0/1424701949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ed0109dc0 0x7f2ed010a1d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc0098a0 tx=0x7f2ecc009bb0 comp rx=0 tx=0).stop 2024-04-25T11:15:52.459 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:51.770+0000 7f2ed7805700 1 -- 172.21.15.69:0/1424701949 shutdown_connections 2024-04-25T11:15:52.459 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:51.770+0000 7f2ed7805700 1 --2- 172.21.15.69:0/1424701949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ed0109dc0 0x7f2ed010a1d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:52.459 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:51.770+0000 7f2ed7805700 1 -- 172.21.15.69:0/1424701949 >> 172.21.15.69:0/1424701949 conn(0x7f2ed01035f0 msgr2=0x7f2ed0105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:52.459 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:51.770+0000 7f2ed7805700 1 -- 172.21.15.69:0/1424701949 shutdown_connections 2024-04-25T11:15:52.459 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:51.770+0000 7f2ed7805700 1 -- 172.21.15.69:0/1424701949 wait complete. 2024-04-25T11:15:52.459 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:51.771+0000 7f2ed7805700 1 Processor -- start 2024-04-25T11:15:52.459 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:51.771+0000 7f2ed7805700 1 -- start start 2024-04-25T11:15:52.459 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:51.771+0000 7f2ed7805700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ed0109dc0 0x7f2ed019fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:52.459 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:51.771+0000 7f2ed7805700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed010fda0 con 0x7f2ed0109dc0 2024-04-25T11:15:52.459 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:51.771+0000 7f2ed55a1700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ed0109dc0 0x7f2ed019fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:52.459 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:51.771+0000 7f2ed55a1700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ed0109dc0 0x7f2ed019fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53818/0 (socket says 172.21.15.69:53818) 2024-04-25T11:15:52.460 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:51.771+0000 7f2ed55a1700 1 -- 172.21.15.69:0/2970329823 learned_addr learned my addr 172.21.15.69:0/2970329823 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:15:52.460 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:51.771+0000 7f2ed55a1700 1 -- 172.21.15.69:0/2970329823 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed01a0280 con 0x7f2ed0109dc0 2024-04-25T11:15:52.460 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:51.772+0000 7f2ed55a1700 1 --2- 172.21.15.69:0/2970329823 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ed0109dc0 0x7f2ed019fd40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc0116a0 tx=0x7f2ecc011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:52.460 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:51.772+0000 7f2ec67fc700 1 -- 172.21.15.69:0/2970329823 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ecc0038b0 con 0x7f2ed0109dc0 2024-04-25T11:15:52.460 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:51.772+0000 7f2ec67fc700 1 -- 172.21.15.69:0/2970329823 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f2ecc003a50 con 0x7f2ed0109dc0 2024-04-25T11:15:52.460 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:51.772+0000 7f2ed7805700 1 -- 172.21.15.69:0/2970329823 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ed01a0510 con 0x7f2ed0109dc0 2024-04-25T11:15:52.460 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:51.772+0000 7f2ec67fc700 1 -- 172.21.15.69:0/2970329823 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ecc019550 con 0x7f2ed0109dc0 2024-04-25T11:15:52.460 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:51.772+0000 7f2ed7805700 1 -- 172.21.15.69:0/2970329823 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ed01a3110 con 0x7f2ed0109dc0 2024-04-25T11:15:52.460 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:51.772+0000 7f2ec67fc700 1 -- 172.21.15.69:0/2970329823 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2ecc019bb0 con 0x7f2ed0109dc0 2024-04-25T11:15:52.460 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:51.772+0000 7f2ec67fc700 1 -- 172.21.15.69:0/2970329823 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2ecc024730 con 0x7f2ed0109dc0 2024-04-25T11:15:52.460 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:51.773+0000 7f2ed7805700 1 -- 172.21.15.69:0/2970329823 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ed0064cf0 con 0x7f2ed0109dc0 2024-04-25T11:15:52.460 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:51.775+0000 7f2ec67fc700 1 -- 172.21.15.69:0/2970329823 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f2ecc043e50 con 0x7f2ed0109dc0 2024-04-25T11:15:52.461 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:51.848+0000 7f2ed7805700 1 -- 172.21.15.69:0/2970329823 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f2ed01a32e0 con 0x7f2ed0109dc0 2024-04-25T11:15:52.461 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:51.848+0000 7f2ec67fc700 1 -- 172.21.15.69:0/2970329823 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f2ecc032020 con 0x7f2ed0109dc0 2024-04-25T11:15:52.461 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:51.850+0000 7f2ed7805700 1 -- 172.21.15.69:0/2970329823 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ed0109dc0 msgr2=0x7f2ed019fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:52.461 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:51.850+0000 7f2ed7805700 1 --2- 172.21.15.69:0/2970329823 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ed0109dc0 0x7f2ed019fd40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc0116a0 tx=0x7f2ecc011a60 comp rx=0 tx=0).stop 2024-04-25T11:15:52.461 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:51.850+0000 7f2ed7805700 1 -- 172.21.15.69:0/2970329823 shutdown_connections 2024-04-25T11:15:52.461 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:51.850+0000 7f2ed7805700 1 --2- 172.21.15.69:0/2970329823 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2ed0109dc0 0x7f2ed019fd40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:52.461 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:51.850+0000 7f2ed7805700 1 -- 172.21.15.69:0/2970329823 >> 172.21.15.69:0/2970329823 conn(0x7f2ed01035f0 msgr2=0x7f2ed0103ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:52.461 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:51.850+0000 7f2ed7805700 1 -- 172.21.15.69:0/2970329823 shutdown_connections 2024-04-25T11:15:52.461 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:51.850+0000 7f2ed7805700 1 -- 172.21.15.69:0/2970329823 wait complete. 2024-04-25T11:15:52.461 INFO:teuthology.orchestra.run.smithi069.stdout:mgr not available, waiting (3/15)... 2024-04-25T11:15:53.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:52 smithi069 ceph-mon[29188]: from='mgr.14100 172.21.15.69:0/4178546609' entity='mgr.x' 2024-04-25T11:15:53.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:52 smithi069 ceph-mon[29188]: mgrmap e3: x(active, since 1.00801s) 2024-04-25T11:15:55.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:54 smithi069 ceph-mon[29188]: mgrmap e4: x(active, since 2s) 2024-04-25T11:15:55.724 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-04-25T11:15:55.724 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout { 2024-04-25T11:15:55.724 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "fsid": "f7ed73dc-02f4-11ef-bc93-c7b262605968", 2024-04-25T11:15:55.724 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T11:15:55.725 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T11:15:55.725 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T11:15:55.725 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T11:15:55.725 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:15:55.725 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T11:15:55.725 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T11:15:55.725 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 0 2024-04-25T11:15:55.725 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-04-25T11:15:55.725 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T11:15:55.725 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T11:15:55.725 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-04-25T11:15:55.725 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-04-25T11:15:55.726 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T11:15:55.726 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:15:55.726 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T11:15:55.726 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T11:15:55.726 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:15:55.726 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T11:15:55.726 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:15:55.726 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T11:15:55.726 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T11:15:55.726 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T11:15:55.735 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T11:15:55.735 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T11:15:55.735 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T11:15:55.736 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:15:55.736 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T11:15:55.736 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T11:15:55.736 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T11:15:55.736 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T11:15:55.736 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T11:15:55.736 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T11:15:55.736 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T11:15:55.736 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T11:15:55.736 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T11:15:55.736 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:15:55.736 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T11:15:55.736 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:15:55.737 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T11:15:55.737 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T11:15:55.737 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:15:55.737 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T11:15:55.737 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T11:15:55.737 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T11:15:55.737 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T11:15:55.737 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T11:15:55.737 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T11:15:55.737 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T11:15:55.737 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-04-25T11:15:55.737 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T11:15:55.737 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:15:55.738 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T11:15:55.738 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:15:55.738 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T11:15:37.965874+0000", 2024-04-25T11:15:55.738 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T11:15:55.738 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:15:55.738 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T11:15:55.738 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout } 2024-04-25T11:15:55.738 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:54.918+0000 7fdeae00d700 1 Processor -- start 2024-04-25T11:15:55.738 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:54.918+0000 7fdeae00d700 1 -- start start 2024-04-25T11:15:55.738 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:54.918+0000 7fdeae00d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdea807d540 0x7fdea810c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:55.738 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:54.918+0000 7fdeae00d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdea810cb40 con 0x7fdea807d540 2024-04-25T11:15:55.738 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:54.918+0000 7fdea77fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdea807d540 0x7fdea810c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:55.739 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:54.918+0000 7fdea77fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdea807d540 0x7fdea810c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50124/0 (socket says 172.21.15.69:50124) 2024-04-25T11:15:55.739 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:54.918+0000 7fdea77fe700 1 -- 172.21.15.69:0/1983194283 learned_addr learned my addr 172.21.15.69:0/1983194283 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:15:55.739 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:54.919+0000 7fdea77fe700 1 -- 172.21.15.69:0/1983194283 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdea810ccc0 con 0x7fdea807d540 2024-04-25T11:15:55.739 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:54.919+0000 7fdea77fe700 1 --2- 172.21.15.69:0/1983194283 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdea807d540 0x7fdea810c570 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fde980098a0 tx=0x7fde98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5922a6ee7fa2a96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:55.739 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:54.919+0000 7fdea67fc700 1 -- 172.21.15.69:0/1983194283 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde98004160 con 0x7fdea807d540 2024-04-25T11:15:55.739 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:54.919+0000 7fdea67fc700 1 -- 172.21.15.69:0/1983194283 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fde98004300 con 0x7fdea807d540 2024-04-25T11:15:55.739 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:54.919+0000 7fdea67fc700 1 -- 172.21.15.69:0/1983194283 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde980119e0 con 0x7fdea807d540 2024-04-25T11:15:55.739 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:54.920+0000 7fdeae00d700 1 -- 172.21.15.69:0/1983194283 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdea807d540 msgr2=0x7fdea810c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:55.739 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:54.920+0000 7fdeae00d700 1 --2- 172.21.15.69:0/1983194283 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdea807d540 0x7fdea810c570 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fde980098a0 tx=0x7fde98009bb0 comp rx=0 tx=0).stop 2024-04-25T11:15:55.739 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:54.920+0000 7fdeae00d700 1 -- 172.21.15.69:0/1983194283 shutdown_connections 2024-04-25T11:15:55.739 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:54.920+0000 7fdeae00d700 1 --2- 172.21.15.69:0/1983194283 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdea807d540 0x7fdea810c570 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:55.739 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:54.920+0000 7fdeae00d700 1 -- 172.21.15.69:0/1983194283 >> 172.21.15.69:0/1983194283 conn(0x7fdea8103fe0 msgr2=0x7fdea8106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:55.739 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:54.920+0000 7fdeae00d700 1 -- 172.21.15.69:0/1983194283 shutdown_connections 2024-04-25T11:15:55.739 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:54.920+0000 7fdeae00d700 1 -- 172.21.15.69:0/1983194283 wait complete. 2024-04-25T11:15:55.740 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:54.921+0000 7fdeae00d700 1 Processor -- start 2024-04-25T11:15:55.740 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:54.921+0000 7fdeae00d700 1 -- start start 2024-04-25T11:15:55.740 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:54.921+0000 7fdeae00d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdea807d540 0x7fdea819db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:55.740 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:54.921+0000 7fdeae00d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdea810dc00 con 0x7fdea807d540 2024-04-25T11:15:55.740 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:54.921+0000 7fdea77fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdea807d540 0x7fdea819db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:55.740 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:54.921+0000 7fdea77fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdea807d540 0x7fdea819db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50140/0 (socket says 172.21.15.69:50140) 2024-04-25T11:15:55.740 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:54.921+0000 7fdea77fe700 1 -- 172.21.15.69:0/3538904824 learned_addr learned my addr 172.21.15.69:0/3538904824 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:15:55.740 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:54.921+0000 7fdea77fe700 1 -- 172.21.15.69:0/3538904824 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdea819e050 con 0x7fdea807d540 2024-04-25T11:15:55.740 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:54.921+0000 7fdea77fe700 1 --2- 172.21.15.69:0/3538904824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdea807d540 0x7fdea819db10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fde980119b0 tx=0x7fde98011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:55.740 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:54.922+0000 7fdea4ff9700 1 -- 172.21.15.69:0/3538904824 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde980039f0 con 0x7fdea807d540 2024-04-25T11:15:55.740 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:54.922+0000 7fdea4ff9700 1 -- 172.21.15.69:0/3538904824 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fde98003b90 con 0x7fdea807d540 2024-04-25T11:15:55.740 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:54.922+0000 7fdeae00d700 1 -- 172.21.15.69:0/3538904824 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdea819e2e0 con 0x7fdea807d540 2024-04-25T11:15:55.741 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:54.922+0000 7fdea4ff9700 1 -- 172.21.15.69:0/3538904824 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde9801a620 con 0x7fdea807d540 2024-04-25T11:15:55.741 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:54.922+0000 7fdeae00d700 1 -- 172.21.15.69:0/3538904824 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdea81a0ef0 con 0x7fdea807d540 2024-04-25T11:15:55.741 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:54.923+0000 7fdea4ff9700 1 -- 172.21.15.69:0/3538904824 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 4) ==== 49505+0+0 (secure 0 0 0) 0x7fde9801a920 con 0x7fdea807d540 2024-04-25T11:15:55.741 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:54.923+0000 7fdeae00d700 1 -- 172.21.15.69:0/3538904824 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdea8064cf0 con 0x7fdea807d540 2024-04-25T11:15:55.741 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:54.923+0000 7fdea4ff9700 1 --2- 172.21.15.69:0/3538904824 >> [v2:172.21.15.69:6800/1508198215,v1:172.21.15.69:6801/1508198215] conn(0x7fde9003d590 0x7fde9003faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:55.741 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:54.923+0000 7fdea4ff9700 1 -- 172.21.15.69:0/3538904824 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fde980529d0 con 0x7fdea807d540 2024-04-25T11:15:55.741 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:54.923+0000 7fdea6ffd700 1 --2- 172.21.15.69:0/3538904824 >> [v2:172.21.15.69:6800/1508198215,v1:172.21.15.69:6801/1508198215] conn(0x7fde9003d590 0x7fde9003faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:55.741 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:54.924+0000 7fdea6ffd700 1 --2- 172.21.15.69:0/3538904824 >> [v2:172.21.15.69:6800/1508198215,v1:172.21.15.69:6801/1508198215] conn(0x7fde9003d590 0x7fde9003faa0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fde9c006fd0 tx=0x7fde9c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:55.741 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:54.929+0000 7fdea4ff9700 1 -- 172.21.15.69:0/3538904824 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fde9802c350 con 0x7fdea807d540 2024-04-25T11:15:55.741 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:55.207+0000 7fdeae00d700 1 -- 172.21.15.69:0/3538904824 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fdea81a1110 con 0x7fdea807d540 2024-04-25T11:15:55.741 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:55.208+0000 7fdea4ff9700 1 -- 172.21.15.69:0/3538904824 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fdea81a1110 con 0x7fdea807d540 2024-04-25T11:15:55.741 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:55.209+0000 7fdeae00d700 1 -- 172.21.15.69:0/3538904824 >> [v2:172.21.15.69:6800/1508198215,v1:172.21.15.69:6801/1508198215] conn(0x7fde9003d590 msgr2=0x7fde9003faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:55.741 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:55.209+0000 7fdeae00d700 1 --2- 172.21.15.69:0/3538904824 >> [v2:172.21.15.69:6800/1508198215,v1:172.21.15.69:6801/1508198215] conn(0x7fde9003d590 0x7fde9003faa0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fde9c006fd0 tx=0x7fde9c006e40 comp rx=0 tx=0).stop 2024-04-25T11:15:55.742 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:55.209+0000 7fdeae00d700 1 -- 172.21.15.69:0/3538904824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdea807d540 msgr2=0x7fdea819db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:55.742 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:55.209+0000 7fdeae00d700 1 --2- 172.21.15.69:0/3538904824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdea807d540 0x7fdea819db10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fde980119b0 tx=0x7fde98011a90 comp rx=0 tx=0).stop 2024-04-25T11:15:55.742 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:55.210+0000 7fdeae00d700 1 -- 172.21.15.69:0/3538904824 shutdown_connections 2024-04-25T11:15:55.742 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:55.210+0000 7fdeae00d700 1 --2- 172.21.15.69:0/3538904824 >> [v2:172.21.15.69:6800/1508198215,v1:172.21.15.69:6801/1508198215] conn(0x7fde9003d590 0x7fde9003faa0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:55.742 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:55.210+0000 7fdeae00d700 1 --2- 172.21.15.69:0/3538904824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdea807d540 0x7fdea819db10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:55.742 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:55.210+0000 7fdeae00d700 1 -- 172.21.15.69:0/3538904824 >> 172.21.15.69:0/3538904824 conn(0x7fdea8103fe0 msgr2=0x7fdea8105880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:55.742 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:55.210+0000 7fdeae00d700 1 -- 172.21.15.69:0/3538904824 shutdown_connections 2024-04-25T11:15:55.742 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:55.210+0000 7fdeae00d700 1 -- 172.21.15.69:0/3538904824 wait complete. 2024-04-25T11:15:55.742 INFO:teuthology.orchestra.run.smithi069.stdout:mgr is available 2024-04-25T11:15:56.147 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:55 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/3538904824' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T11:15:56.949 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:56 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/1889487422' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T11:15:56.969 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-04-25T11:15:56.969 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T11:15:56.969 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout fsid = f7ed73dc-02f4-11ef-bc93-c7b262605968 2024-04-25T11:15:56.969 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T11:15:56.969 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T11:15:56.969 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T11:15:56.969 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T11:15:56.970 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T11:15:56.970 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-04-25T11:15:56.970 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T11:15:56.970 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T11:15:56.970 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-04-25T11:15:56.970 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T11:15:56.970 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T11:15:56.970 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T11:15:56.970 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T11:15:56.970 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T11:15:56.970 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T11:15:56.970 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.170+0000 7f11f4195700 1 Processor -- start 2024-04-25T11:15:56.970 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.170+0000 7f11f4195700 1 -- start start 2024-04-25T11:15:56.971 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.170+0000 7f11f4195700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11ec07d560 0x7f11ec10e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:56.971 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.170+0000 7f11f4195700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f11ec10ee60 con 0x7f11ec07d560 2024-04-25T11:15:56.971 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.170+0000 7f11f1f31700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11ec07d560 0x7f11ec10e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:56.971 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.170+0000 7f11f1f31700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11ec07d560 0x7f11ec10e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50144/0 (socket says 172.21.15.69:50144) 2024-04-25T11:15:56.971 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.170+0000 7f11f1f31700 1 -- 172.21.15.69:0/1180356803 learned_addr learned my addr 172.21.15.69:0/1180356803 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:15:56.971 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.171+0000 7f11f1f31700 1 -- 172.21.15.69:0/1180356803 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11ec10efe0 con 0x7f11ec07d560 2024-04-25T11:15:56.971 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.171+0000 7f11f1f31700 1 --2- 172.21.15.69:0/1180356803 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11ec07d560 0x7f11ec10e920 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f11e80098a0 tx=0x7f11e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f36921bd2a37fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:56.971 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.171+0000 7f11f0f2f700 1 -- 172.21.15.69:0/1180356803 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11e8004160 con 0x7f11ec07d560 2024-04-25T11:15:56.971 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.171+0000 7f11f0f2f700 1 -- 172.21.15.69:0/1180356803 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f11e8004300 con 0x7f11ec07d560 2024-04-25T11:15:56.971 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.171+0000 7f11f0f2f700 1 -- 172.21.15.69:0/1180356803 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11e80119e0 con 0x7f11ec07d560 2024-04-25T11:15:56.971 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.172+0000 7f11f4195700 1 -- 172.21.15.69:0/1180356803 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11ec07d560 msgr2=0x7f11ec10e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:56.971 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.172+0000 7f11f4195700 1 --2- 172.21.15.69:0/1180356803 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11ec07d560 0x7f11ec10e920 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f11e80098a0 tx=0x7f11e8009bb0 comp rx=0 tx=0).stop 2024-04-25T11:15:56.972 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.172+0000 7f11f4195700 1 -- 172.21.15.69:0/1180356803 shutdown_connections 2024-04-25T11:15:56.972 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.172+0000 7f11f4195700 1 --2- 172.21.15.69:0/1180356803 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11ec07d560 0x7f11ec10e920 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:56.972 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.172+0000 7f11f4195700 1 -- 172.21.15.69:0/1180356803 >> 172.21.15.69:0/1180356803 conn(0x7f11ec103f90 msgr2=0x7f11ec1063e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:56.972 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.172+0000 7f11f4195700 1 -- 172.21.15.69:0/1180356803 shutdown_connections 2024-04-25T11:15:56.972 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.172+0000 7f11f4195700 1 -- 172.21.15.69:0/1180356803 wait complete. 2024-04-25T11:15:56.972 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.173+0000 7f11f4195700 1 Processor -- start 2024-04-25T11:15:56.972 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.173+0000 7f11f4195700 1 -- start start 2024-04-25T11:15:56.972 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.173+0000 7f11f4195700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11ec07d560 0x7f11ec19fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:56.972 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.173+0000 7f11f4195700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f11ec10fe00 con 0x7f11ec07d560 2024-04-25T11:15:56.972 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.173+0000 7f11f1f31700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11ec07d560 0x7f11ec19fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:56.973 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.173+0000 7f11f1f31700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11ec07d560 0x7f11ec19fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50152/0 (socket says 172.21.15.69:50152) 2024-04-25T11:15:56.973 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.173+0000 7f11f1f31700 1 -- 172.21.15.69:0/1889487422 learned_addr learned my addr 172.21.15.69:0/1889487422 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:15:56.973 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.173+0000 7f11f1f31700 1 -- 172.21.15.69:0/1889487422 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11ec1a0280 con 0x7f11ec07d560 2024-04-25T11:15:56.973 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.173+0000 7f11f1f31700 1 --2- 172.21.15.69:0/1889487422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11ec07d560 0x7f11ec19fd40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f11e80119b0 tx=0x7f11e8011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:56.973 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.173+0000 7f11e2ffd700 1 -- 172.21.15.69:0/1889487422 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11e8003af0 con 0x7f11ec07d560 2024-04-25T11:15:56.973 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.173+0000 7f11e2ffd700 1 -- 172.21.15.69:0/1889487422 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f11e801a8a0 con 0x7f11ec07d560 2024-04-25T11:15:56.973 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.173+0000 7f11f4195700 1 -- 172.21.15.69:0/1889487422 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11ec1a0510 con 0x7f11ec07d560 2024-04-25T11:15:56.973 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.173+0000 7f11e2ffd700 1 -- 172.21.15.69:0/1889487422 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11e8024830 con 0x7f11ec07d560 2024-04-25T11:15:56.973 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.174+0000 7f11f4195700 1 -- 172.21.15.69:0/1889487422 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11ec1a3110 con 0x7f11ec07d560 2024-04-25T11:15:56.973 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.174+0000 7f11e2ffd700 1 -- 172.21.15.69:0/1889487422 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 4) ==== 49505+0+0 (secure 0 0 0) 0x7f11e8022020 con 0x7f11ec07d560 2024-04-25T11:15:56.973 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.174+0000 7f11f4195700 1 -- 172.21.15.69:0/1889487422 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11ec064cf0 con 0x7f11ec07d560 2024-04-25T11:15:56.973 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.175+0000 7f11e2ffd700 1 --2- 172.21.15.69:0/1889487422 >> [v2:172.21.15.69:6800/1508198215,v1:172.21.15.69:6801/1508198215] conn(0x7f11d803d630 0x7f11d803fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:56.973 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.175+0000 7f11e2ffd700 1 -- 172.21.15.69:0/1889487422 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f11e8052350 con 0x7f11ec07d560 2024-04-25T11:15:56.974 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.175+0000 7f11f1730700 1 --2- 172.21.15.69:0/1889487422 >> [v2:172.21.15.69:6800/1508198215,v1:172.21.15.69:6801/1508198215] conn(0x7f11d803d630 0x7f11d803fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:56.974 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.175+0000 7f11f1730700 1 --2- 172.21.15.69:0/1889487422 >> [v2:172.21.15.69:6800/1508198215,v1:172.21.15.69:6801/1508198215] conn(0x7f11d803d630 0x7f11d803fb40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f11dc006fd0 tx=0x7f11dc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:56.974 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.180+0000 7f11e2ffd700 1 -- 172.21.15.69:0/1889487422 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f11e800e260 con 0x7f11ec07d560 2024-04-25T11:15:56.974 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.376+0000 7f11f4195700 1 -- 172.21.15.69:0/1889487422 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f11ec1a4910 con 0x7f11ec07d560 2024-04-25T11:15:56.974 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.377+0000 7f11e2ffd700 1 -- 172.21.15.69:0/1889487422 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7f11e80520b0 con 0x7f11ec07d560 2024-04-25T11:15:56.974 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.380+0000 7f11f4195700 1 -- 172.21.15.69:0/1889487422 >> [v2:172.21.15.69:6800/1508198215,v1:172.21.15.69:6801/1508198215] conn(0x7f11d803d630 msgr2=0x7f11d803fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:56.974 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.380+0000 7f11f4195700 1 --2- 172.21.15.69:0/1889487422 >> [v2:172.21.15.69:6800/1508198215,v1:172.21.15.69:6801/1508198215] conn(0x7f11d803d630 0x7f11d803fb40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f11dc006fd0 tx=0x7f11dc006e40 comp rx=0 tx=0).stop 2024-04-25T11:15:56.974 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.380+0000 7f11f4195700 1 -- 172.21.15.69:0/1889487422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11ec07d560 msgr2=0x7f11ec19fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:56.974 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.380+0000 7f11f4195700 1 --2- 172.21.15.69:0/1889487422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11ec07d560 0x7f11ec19fd40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f11e80119b0 tx=0x7f11e8011a90 comp rx=0 tx=0).stop 2024-04-25T11:15:56.974 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.380+0000 7f11f4195700 1 -- 172.21.15.69:0/1889487422 shutdown_connections 2024-04-25T11:15:56.974 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.380+0000 7f11f4195700 1 --2- 172.21.15.69:0/1889487422 >> [v2:172.21.15.69:6800/1508198215,v1:172.21.15.69:6801/1508198215] conn(0x7f11d803d630 0x7f11d803fb40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:56.974 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.380+0000 7f11f4195700 1 --2- 172.21.15.69:0/1889487422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11ec07d560 0x7f11ec19fd40 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:56.974 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.380+0000 7f11f4195700 1 -- 172.21.15.69:0/1889487422 >> 172.21.15.69:0/1889487422 conn(0x7f11ec103f90 msgr2=0x7f11ec105810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:56.975 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.380+0000 7f11f4195700 1 -- 172.21.15.69:0/1889487422 shutdown_connections 2024-04-25T11:15:56.975 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:56.380+0000 7f11f4195700 1 -- 172.21.15.69:0/1889487422 wait complete. 2024-04-25T11:15:56.975 INFO:teuthology.orchestra.run.smithi069.stdout:Enabling cephadm module... 2024-04-25T11:15:58.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:57 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/3330098985' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-25T11:15:58.200 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:15:57 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: ignoring --setuser ceph since I am not root 2024-04-25T11:15:58.200 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:15:57 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: ignoring --setgroup ceph since I am not root 2024-04-25T11:15:58.200 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:15:58 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:15:58.106+0000 7ff35df82200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T11:15:58.531 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:15:58 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:15:58.294+0000 7ff35df82200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:15:58.532 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:15:58 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:15:58.398+0000 7ff35df82200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T11:15:58.539 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.390+0000 7f2fbc095700 1 Processor -- start 2024-04-25T11:15:58.539 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.390+0000 7f2fbc095700 1 -- start start 2024-04-25T11:15:58.539 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.390+0000 7f2fbc095700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fb4107d60 0x7f2fb41081b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:58.539 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.390+0000 7f2fbc095700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb41087c0 con 0x7f2fb4107d60 2024-04-25T11:15:58.539 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.391+0000 7f2fb9e31700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fb4107d60 0x7f2fb41081b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:58.539 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.391+0000 7f2fb9e31700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fb4107d60 0x7f2fb41081b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50162/0 (socket says 172.21.15.69:50162) 2024-04-25T11:15:58.539 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.391+0000 7f2fb9e31700 1 -- 172.21.15.69:0/2458475023 learned_addr learned my addr 172.21.15.69:0/2458475023 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:15:58.540 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.391+0000 7f2fb9e31700 1 -- 172.21.15.69:0/2458475023 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb410ce50 con 0x7f2fb4107d60 2024-04-25T11:15:58.540 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.391+0000 7f2fb9e31700 1 --2- 172.21.15.69:0/2458475023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fb4107d60 0x7f2fb41081b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2fb00098a0 tx=0x7f2fb0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97ce6f128484fa1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:58.540 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.391+0000 7f2fb8e2f700 1 -- 172.21.15.69:0/2458475023 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fb0004160 con 0x7f2fb4107d60 2024-04-25T11:15:58.540 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.391+0000 7f2fb8e2f700 1 -- 172.21.15.69:0/2458475023 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f2fb0004300 con 0x7f2fb4107d60 2024-04-25T11:15:58.540 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.391+0000 7f2fb8e2f700 1 -- 172.21.15.69:0/2458475023 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fb00119e0 con 0x7f2fb4107d60 2024-04-25T11:15:58.540 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.392+0000 7f2fbc095700 1 -- 172.21.15.69:0/2458475023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fb4107d60 msgr2=0x7f2fb41081b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:58.540 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.392+0000 7f2fbc095700 1 --2- 172.21.15.69:0/2458475023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fb4107d60 0x7f2fb41081b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2fb00098a0 tx=0x7f2fb0009bb0 comp rx=0 tx=0).stop 2024-04-25T11:15:58.540 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.392+0000 7f2fbc095700 1 -- 172.21.15.69:0/2458475023 shutdown_connections 2024-04-25T11:15:58.540 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.392+0000 7f2fbc095700 1 --2- 172.21.15.69:0/2458475023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fb4107d60 0x7f2fb41081b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:58.540 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.392+0000 7f2fbc095700 1 -- 172.21.15.69:0/2458475023 >> 172.21.15.69:0/2458475023 conn(0x7f2fb4103760 msgr2=0x7f2fb4105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:58.540 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.392+0000 7f2fbc095700 1 -- 172.21.15.69:0/2458475023 shutdown_connections 2024-04-25T11:15:58.540 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.392+0000 7f2fbc095700 1 -- 172.21.15.69:0/2458475023 wait complete. 2024-04-25T11:15:58.540 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.393+0000 7f2fbc095700 1 Processor -- start 2024-04-25T11:15:58.541 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.393+0000 7f2fbc095700 1 -- start start 2024-04-25T11:15:58.541 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.393+0000 7f2fbc095700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fb4107d60 0x7f2fb419db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:58.541 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.393+0000 7f2fbc095700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb410dc10 con 0x7f2fb4107d60 2024-04-25T11:15:58.541 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.393+0000 7f2fb9e31700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fb4107d60 0x7f2fb419db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:58.541 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.393+0000 7f2fb9e31700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fb4107d60 0x7f2fb419db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50168/0 (socket says 172.21.15.69:50168) 2024-04-25T11:15:58.541 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.393+0000 7f2fb9e31700 1 -- 172.21.15.69:0/3330098985 learned_addr learned my addr 172.21.15.69:0/3330098985 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:15:58.541 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.393+0000 7f2fb9e31700 1 -- 172.21.15.69:0/3330098985 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb419e060 con 0x7f2fb4107d60 2024-04-25T11:15:58.541 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.394+0000 7f2fb9e31700 1 --2- 172.21.15.69:0/3330098985 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fb4107d60 0x7f2fb419db20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2fb00119b0 tx=0x7f2fb0011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:58.541 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.394+0000 7f2faaffd700 1 -- 172.21.15.69:0/3330098985 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fb00039f0 con 0x7f2fb4107d60 2024-04-25T11:15:58.541 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.394+0000 7f2faaffd700 1 -- 172.21.15.69:0/3330098985 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f2fb0003b90 con 0x7f2fb4107d60 2024-04-25T11:15:58.541 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.394+0000 7f2fbc095700 1 -- 172.21.15.69:0/3330098985 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fb419e2f0 con 0x7f2fb4107d60 2024-04-25T11:15:58.541 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.394+0000 7f2faaffd700 1 -- 172.21.15.69:0/3330098985 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fb001a620 con 0x7f2fb4107d60 2024-04-25T11:15:58.541 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.394+0000 7f2fbc095700 1 -- 172.21.15.69:0/3330098985 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fb41a0f00 con 0x7f2fb4107d60 2024-04-25T11:15:58.542 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.395+0000 7f2faaffd700 1 -- 172.21.15.69:0/3330098985 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 4) ==== 49505+0+0 (secure 0 0 0) 0x7f2fb001a8d0 con 0x7f2fb4107d60 2024-04-25T11:15:58.542 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.395+0000 7f2faaffd700 1 --2- 172.21.15.69:0/3330098985 >> [v2:172.21.15.69:6800/1508198215,v1:172.21.15.69:6801/1508198215] conn(0x7f2fa003d5e0 0x7f2fa003faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:58.542 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.395+0000 7f2faaffd700 1 -- 172.21.15.69:0/3330098985 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2fb00537f0 con 0x7f2fb4107d60 2024-04-25T11:15:58.542 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.395+0000 7f2fbc095700 1 -- 172.21.15.69:0/3330098985 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fb4064cf0 con 0x7f2fb4107d60 2024-04-25T11:15:58.542 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.395+0000 7f2fb9630700 1 --2- 172.21.15.69:0/3330098985 >> [v2:172.21.15.69:6800/1508198215,v1:172.21.15.69:6801/1508198215] conn(0x7f2fa003d5e0 0x7f2fa003faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:58.542 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.395+0000 7f2fb9630700 1 --2- 172.21.15.69:0/3330098985 >> [v2:172.21.15.69:6800/1508198215,v1:172.21.15.69:6801/1508198215] conn(0x7f2fa003d5e0 0x7f2fa003faf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4006fd0 tx=0x7f2fa4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:58.542 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.400+0000 7f2faaffd700 1 -- 172.21.15.69:0/3330098985 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2fb002c350 con 0x7f2fb4107d60 2024-04-25T11:15:58.542 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.650+0000 7f2fbc095700 1 -- 172.21.15.69:0/3330098985 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f2fb41a14b0 con 0x7f2fb4107d60 2024-04-25T11:15:58.543 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.914+0000 7f2faaffd700 1 -- 172.21.15.69:0/3330098985 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f2fb000e260 con 0x7f2fb4107d60 2024-04-25T11:15:58.543 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.916+0000 7f2faaffd700 1 -- 172.21.15.69:0/3330098985 <== mon.0 v2:172.21.15.69:3300/0 8 ==== mgrmap(e 5) ==== 49516+0+0 (secure 0 0 0) 0x7f2fb0018350 con 0x7f2fb4107d60 2024-04-25T11:15:58.543 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.916+0000 7f2fbc095700 1 -- 172.21.15.69:0/3330098985 >> [v2:172.21.15.69:6800/1508198215,v1:172.21.15.69:6801/1508198215] conn(0x7f2fa003d5e0 msgr2=0x7f2fa003faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:58.543 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.916+0000 7f2fbc095700 1 --2- 172.21.15.69:0/3330098985 >> [v2:172.21.15.69:6800/1508198215,v1:172.21.15.69:6801/1508198215] conn(0x7f2fa003d5e0 0x7f2fa003faf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4006fd0 tx=0x7f2fa4006e40 comp rx=0 tx=0).stop 2024-04-25T11:15:58.543 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.917+0000 7f2fbc095700 1 -- 172.21.15.69:0/3330098985 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fb4107d60 msgr2=0x7f2fb419db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:58.543 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.917+0000 7f2fbc095700 1 --2- 172.21.15.69:0/3330098985 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fb4107d60 0x7f2fb419db20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2fb00119b0 tx=0x7f2fb0011a90 comp rx=0 tx=0).stop 2024-04-25T11:15:58.543 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.917+0000 7f2fbc095700 1 -- 172.21.15.69:0/3330098985 shutdown_connections 2024-04-25T11:15:58.543 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.917+0000 7f2fbc095700 1 --2- 172.21.15.69:0/3330098985 >> [v2:172.21.15.69:6800/1508198215,v1:172.21.15.69:6801/1508198215] conn(0x7f2fa003d5e0 0x7f2fa003faf0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:58.543 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.917+0000 7f2fbc095700 1 --2- 172.21.15.69:0/3330098985 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fb4107d60 0x7f2fb419db20 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:58.543 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.917+0000 7f2fbc095700 1 -- 172.21.15.69:0/3330098985 >> 172.21.15.69:0/3330098985 conn(0x7f2fb4103760 msgr2=0x7f2fb41050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:58.543 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.917+0000 7f2fbc095700 1 -- 172.21.15.69:0/3330098985 shutdown_connections 2024-04-25T11:15:58.543 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:57.917+0000 7f2fbc095700 1 -- 172.21.15.69:0/3330098985 wait complete. 2024-04-25T11:15:58.802 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:15:58 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:15:58.713+0000 7ff35df82200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T11:15:58.802 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:15:58 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:15:58.801+0000 7ff35df82200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T11:15:59.168 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:58 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/3330098985' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-25T11:15:59.168 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:58 smithi069 ceph-mon[29188]: mgrmap e5: x(active, since 6s) 2024-04-25T11:15:59.848 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout { 2024-04-25T11:15:59.848 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-25T11:15:59.849 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T11:15:59.849 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T11:15:59.849 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T11:15:59.849 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout } 2024-04-25T11:15:59.849 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:58.961+0000 7f6cf087d700 1 Processor -- start 2024-04-25T11:15:59.849 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:58.961+0000 7f6cf087d700 1 -- start start 2024-04-25T11:15:59.849 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:58.962+0000 7f6cf087d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6cec10ae70 0x7f6cec10b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:59.849 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:58.962+0000 7f6cf087d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6cec10b850 con 0x7f6cec10ae70 2024-04-25T11:15:59.849 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:58.962+0000 7f6cea59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6cec10ae70 0x7f6cec10b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:59.849 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:58.962+0000 7f6cea59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6cec10ae70 0x7f6cec10b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50180/0 (socket says 172.21.15.69:50180) 2024-04-25T11:15:59.849 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:58.962+0000 7f6cea59c700 1 -- 172.21.15.69:0/3400079159 learned_addr learned my addr 172.21.15.69:0/3400079159 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:15:59.849 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:58.962+0000 7f6cea59c700 1 -- 172.21.15.69:0/3400079159 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cec10b9d0 con 0x7f6cec10ae70 2024-04-25T11:15:59.850 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:58.962+0000 7f6cea59c700 1 --2- 172.21.15.69:0/3400079159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6cec10ae70 0x7f6cec10b280 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc0098a0 tx=0x7f6cdc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a7ccec065e19826 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:59.850 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:58.963+0000 7f6ce959a700 1 -- 172.21.15.69:0/3400079159 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6cdc004160 con 0x7f6cec10ae70 2024-04-25T11:15:59.850 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:58.963+0000 7f6ce959a700 1 -- 172.21.15.69:0/3400079159 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f6cdc004300 con 0x7f6cec10ae70 2024-04-25T11:15:59.850 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:58.963+0000 7f6ce959a700 1 -- 172.21.15.69:0/3400079159 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6cdc0119e0 con 0x7f6cec10ae70 2024-04-25T11:15:59.850 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:58.964+0000 7f6cf087d700 1 -- 172.21.15.69:0/3400079159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6cec10ae70 msgr2=0x7f6cec10b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:59.850 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:58.964+0000 7f6cf087d700 1 --2- 172.21.15.69:0/3400079159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6cec10ae70 0x7f6cec10b280 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc0098a0 tx=0x7f6cdc009bb0 comp rx=0 tx=0).stop 2024-04-25T11:15:59.850 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:58.964+0000 7f6cf087d700 1 -- 172.21.15.69:0/3400079159 shutdown_connections 2024-04-25T11:15:59.850 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:58.964+0000 7f6cf087d700 1 --2- 172.21.15.69:0/3400079159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6cec10ae70 0x7f6cec10b280 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:59.850 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:58.964+0000 7f6cf087d700 1 -- 172.21.15.69:0/3400079159 >> 172.21.15.69:0/3400079159 conn(0x7f6cec1035f0 msgr2=0x7f6cec105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:59.850 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:58.964+0000 7f6cf087d700 1 -- 172.21.15.69:0/3400079159 shutdown_connections 2024-04-25T11:15:59.850 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:58.964+0000 7f6cf087d700 1 -- 172.21.15.69:0/3400079159 wait complete. 2024-04-25T11:15:59.850 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:58.965+0000 7f6cf087d700 1 Processor -- start 2024-04-25T11:15:59.850 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:58.965+0000 7f6cf087d700 1 -- start start 2024-04-25T11:15:59.851 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:58.965+0000 7f6cf087d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6cec10ae70 0x7f6cec10a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:59.851 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:58.965+0000 7f6cf087d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6cec108650 con 0x7f6cec10ae70 2024-04-25T11:15:59.851 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:58.965+0000 7f6cea59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6cec10ae70 0x7f6cec10a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:59.851 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:58.965+0000 7f6cea59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6cec10ae70 0x7f6cec10a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50186/0 (socket says 172.21.15.69:50186) 2024-04-25T11:15:59.851 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:58.965+0000 7f6cea59c700 1 -- 172.21.15.69:0/4109428929 learned_addr learned my addr 172.21.15.69:0/4109428929 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:15:59.851 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:58.965+0000 7f6cea59c700 1 -- 172.21.15.69:0/4109428929 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cec10a8a0 con 0x7f6cec10ae70 2024-04-25T11:15:59.851 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:58.965+0000 7f6cea59c700 1 --2- 172.21.15.69:0/4109428929 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6cec10ae70 0x7f6cec10a360 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc0119b0 tx=0x7f6cdc011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:59.851 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:58.966+0000 7f6cdb7fe700 1 -- 172.21.15.69:0/4109428929 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6cdc003af0 con 0x7f6cec10ae70 2024-04-25T11:15:59.851 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:58.966+0000 7f6cdb7fe700 1 -- 172.21.15.69:0/4109428929 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f6cdc01a8a0 con 0x7f6cec10ae70 2024-04-25T11:15:59.851 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:58.966+0000 7f6cf087d700 1 -- 172.21.15.69:0/4109428929 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cec108af0 con 0x7f6cec10ae70 2024-04-25T11:15:59.851 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:58.966+0000 7f6cdb7fe700 1 -- 172.21.15.69:0/4109428929 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6cdc024830 con 0x7f6cec10ae70 2024-04-25T11:15:59.851 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:58.966+0000 7f6cf087d700 1 -- 172.21.15.69:0/4109428929 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cec108fd0 con 0x7f6cec10ae70 2024-04-25T11:15:59.851 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:58.966+0000 7f6cdb7fe700 1 -- 172.21.15.69:0/4109428929 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 49516+0+0 (secure 0 0 0) 0x7f6cdc022020 con 0x7f6cec10ae70 2024-04-25T11:15:59.852 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:58.967+0000 7f6cdb7fe700 1 --2- 172.21.15.69:0/4109428929 >> [v2:172.21.15.69:6800/1508198215,v1:172.21.15.69:6801/1508198215] conn(0x7f6cd403d680 0x7f6cd403fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:59.852 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:58.967+0000 7f6cdb7fe700 1 -- 172.21.15.69:0/4109428929 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6cdc0531d0 con 0x7f6cec10ae70 2024-04-25T11:15:59.852 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:58.967+0000 7f6ce9d9b700 1 -- 172.21.15.69:0/4109428929 >> [v2:172.21.15.69:6800/1508198215,v1:172.21.15.69:6801/1508198215] conn(0x7f6cd403d680 msgr2=0x7f6cd403fb90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/1508198215 2024-04-25T11:15:59.852 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:58.967+0000 7f6ce9d9b700 1 --2- 172.21.15.69:0/4109428929 >> [v2:172.21.15.69:6800/1508198215,v1:172.21.15.69:6801/1508198215] conn(0x7f6cd403d680 0x7f6cd403fb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T11:15:59.852 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:58.967+0000 7f6cf087d700 1 -- 172.21.15.69:0/4109428929 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cec064cf0 con 0x7f6cec10ae70 2024-04-25T11:15:59.852 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:58.972+0000 7f6cdb7fe700 1 -- 172.21.15.69:0/4109428929 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6cdc00e260 con 0x7f6cec10ae70 2024-04-25T11:15:59.852 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:59.168+0000 7f6ce9d9b700 1 -- 172.21.15.69:0/4109428929 >> [v2:172.21.15.69:6800/1508198215,v1:172.21.15.69:6801/1508198215] conn(0x7f6cd403d680 msgr2=0x7f6cd403fb90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/1508198215 2024-04-25T11:15:59.852 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:59.168+0000 7f6ce9d9b700 1 --2- 172.21.15.69:0/4109428929 >> [v2:172.21.15.69:6800/1508198215,v1:172.21.15.69:6801/1508198215] conn(0x7f6cd403d680 0x7f6cd403fb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T11:15:59.852 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:59.266+0000 7f6cf087d700 1 -- 172.21.15.69:0/4109428929 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f6cec1092b0 con 0x7f6cec10ae70 2024-04-25T11:15:59.852 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:59.266+0000 7f6cdb7fe700 1 -- 172.21.15.69:0/4109428929 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f6cec1092b0 con 0x7f6cec10ae70 2024-04-25T11:15:59.852 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:59.269+0000 7f6cf087d700 1 -- 172.21.15.69:0/4109428929 >> [v2:172.21.15.69:6800/1508198215,v1:172.21.15.69:6801/1508198215] conn(0x7f6cd403d680 msgr2=0x7f6cd403fb90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:15:59.852 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:59.269+0000 7f6cf087d700 1 --2- 172.21.15.69:0/4109428929 >> [v2:172.21.15.69:6800/1508198215,v1:172.21.15.69:6801/1508198215] conn(0x7f6cd403d680 0x7f6cd403fb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:59.852 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:59.269+0000 7f6cf087d700 1 -- 172.21.15.69:0/4109428929 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6cec10ae70 msgr2=0x7f6cec10a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:59.852 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:59.269+0000 7f6cf087d700 1 --2- 172.21.15.69:0/4109428929 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6cec10ae70 0x7f6cec10a360 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc0119b0 tx=0x7f6cdc011a90 comp rx=0 tx=0).stop 2024-04-25T11:15:59.853 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:59.269+0000 7f6cf087d700 1 -- 172.21.15.69:0/4109428929 shutdown_connections 2024-04-25T11:15:59.853 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:59.269+0000 7f6cf087d700 1 --2- 172.21.15.69:0/4109428929 >> [v2:172.21.15.69:6800/1508198215,v1:172.21.15.69:6801/1508198215] conn(0x7f6cd403d680 0x7f6cd403fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:59.853 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:59.269+0000 7f6cf087d700 1 --2- 172.21.15.69:0/4109428929 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6cec10ae70 0x7f6cec10a360 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:59.853 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:59.269+0000 7f6cf087d700 1 -- 172.21.15.69:0/4109428929 >> 172.21.15.69:0/4109428929 conn(0x7f6cec1035f0 msgr2=0x7f6cec103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:59.853 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:59.269+0000 7f6cf087d700 1 -- 172.21.15.69:0/4109428929 shutdown_connections 2024-04-25T11:15:59.853 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:15:59.269+0000 7f6cf087d700 1 -- 172.21.15.69:0/4109428929 wait complete. 2024-04-25T11:15:59.853 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for the mgr to restart... 2024-04-25T11:15:59.853 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for mgr epoch 5... 2024-04-25T11:16:00.094 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:15:59 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/4109428929' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T11:16:00.095 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:00 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:16:00.075+0000 7ff35df82200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T11:16:00.438 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:00 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:16:00.260+0000 7ff35df82200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T11:16:00.438 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:00 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:16:00.437+0000 7ff35df82200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T11:16:01.030 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:00 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:16:00.735+0000 7ff35df82200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T11:16:01.031 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:00 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:16:00.828+0000 7ff35df82200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T11:16:01.031 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:01 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:16:01.030+0000 7ff35df82200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T11:16:01.449 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:01 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:16:01.223+0000 7ff35df82200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:16:02.199 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:01 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:16:01.823+0000 7ff35df82200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T11:16:02.550 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:02 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:16:02.249+0000 7ff35df82200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T11:16:02.550 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:02 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:16:02.346+0000 7ff35df82200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T11:16:02.550 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:02 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:16:02.549+0000 7ff35df82200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T11:16:03.323 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:03 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:16:03.065+0000 7ff35df82200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T11:16:03.323 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:03 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:16:03.158+0000 7ff35df82200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T11:16:03.628 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:03 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:16:03.401+0000 7ff35df82200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T11:16:03.629 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:03 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:16:03.533+0000 7ff35df82200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T11:16:03.629 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:03 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:16:03.628+0000 7ff35df82200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T11:16:03.949 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:03 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:16:03.744+0000 7ff35df82200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T11:16:04.449 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:04 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:16:04.045+0000 7ff35df82200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T11:16:04.911 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:04 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:16:04.640+0000 7ff35df82200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T11:16:04.912 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:04 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:16:04.857+0000 7ff35df82200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T11:16:05.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:04 smithi069 ceph-mon[29188]: Active manager daemon x restarted 2024-04-25T11:16:05.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:04 smithi069 ceph-mon[29188]: Activating manager daemon x 2024-04-25T11:16:05.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:04 smithi069 ceph-mon[29188]: osdmap e2: 0 total, 0 up, 0 in 2024-04-25T11:16:05.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:04 smithi069 ceph-mon[29188]: mgrmap e6: x(active, starting, since 0.00359006s) 2024-04-25T11:16:05.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:04 smithi069 ceph-mon[29188]: from='mgr.14120 172.21.15.69:0/157690041' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T11:16:05.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:04 smithi069 ceph-mon[29188]: from='mgr.14120 172.21.15.69:0/157690041' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T11:16:05.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:04 smithi069 ceph-mon[29188]: from='mgr.14120 172.21.15.69:0/157690041' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T11:16:05.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:04 smithi069 ceph-mon[29188]: from='mgr.14120 172.21.15.69:0/157690041' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T11:16:05.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:04 smithi069 ceph-mon[29188]: from='mgr.14120 172.21.15.69:0/157690041' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T11:16:05.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:04 smithi069 ceph-mon[29188]: Manager daemon x is now available 2024-04-25T11:16:05.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:04 smithi069 ceph-mon[29188]: from='mgr.14120 172.21.15.69:0/157690041' entity='mgr.x' 2024-04-25T11:16:05.201 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:04 smithi069 ceph-mon[29188]: from='mgr.14120 172.21.15.69:0/157690041' entity='mgr.x' 2024-04-25T11:16:05.201 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:04 smithi069 ceph-mon[29188]: from='mgr.14120 172.21.15.69:0/157690041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:16:05.201 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:04 smithi069 ceph-mon[29188]: from='mgr.14120 172.21.15.69:0/157690041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:16:05.201 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:04 smithi069 ceph-mon[29188]: from='mgr.14120 172.21.15.69:0/157690041' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T11:16:06.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:05 smithi069 ceph-mon[29188]: Found migration_current of "None". Setting to last migration. 2024-04-25T11:16:06.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:05 smithi069 ceph-mon[29188]: from='mgr.14120 172.21.15.69:0/157690041' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T11:16:06.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:05 smithi069 ceph-mon[29188]: from='mgr.14120 172.21.15.69:0/157690041' entity='mgr.x' 2024-04-25T11:16:06.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:05 smithi069 ceph-mon[29188]: from='mgr.14120 172.21.15.69:0/157690041' entity='mgr.x' 2024-04-25T11:16:06.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:05 smithi069 ceph-mon[29188]: mgrmap e7: x(active, since 1.00654s) 2024-04-25T11:16:06.479 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout { 2024-04-25T11:16:06.479 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-25T11:16:06.480 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T11:16:06.480 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout } 2024-04-25T11:16:06.480 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:00.312+0000 7f59c43f3700 1 Processor -- start 2024-04-25T11:16:06.480 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:00.313+0000 7f59c43f3700 1 -- start start 2024-04-25T11:16:06.480 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:00.313+0000 7f59c43f3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59bc07d560 0x7f59bc10c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:06.480 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:00.313+0000 7f59c43f3700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc10ca70 con 0x7f59bc07d560 2024-04-25T11:16:06.480 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:00.313+0000 7f59c218f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59bc07d560 0x7f59bc10c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:06.480 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:00.313+0000 7f59c218f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59bc07d560 0x7f59bc10c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50190/0 (socket says 172.21.15.69:50190) 2024-04-25T11:16:06.480 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:00.313+0000 7f59c218f700 1 -- 172.21.15.69:0/3674708017 learned_addr learned my addr 172.21.15.69:0/3674708017 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:16:06.481 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:00.314+0000 7f59c218f700 1 -- 172.21.15.69:0/3674708017 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59bc10cbf0 con 0x7f59bc07d560 2024-04-25T11:16:06.481 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:00.314+0000 7f59c218f700 1 --2- 172.21.15.69:0/3674708017 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59bc07d560 0x7f59bc10c480 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f59b80098a0 tx=0x7f59b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5ff876dde2d37ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:06.481 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:00.314+0000 7f59c118d700 1 -- 172.21.15.69:0/3674708017 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59b8004160 con 0x7f59bc07d560 2024-04-25T11:16:06.481 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:00.314+0000 7f59c118d700 1 -- 172.21.15.69:0/3674708017 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f59b8004300 con 0x7f59bc07d560 2024-04-25T11:16:06.481 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:00.314+0000 7f59c118d700 1 -- 172.21.15.69:0/3674708017 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59b80119e0 con 0x7f59bc07d560 2024-04-25T11:16:06.481 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:00.315+0000 7f59c43f3700 1 -- 172.21.15.69:0/3674708017 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59bc07d560 msgr2=0x7f59bc10c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:06.482 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:00.315+0000 7f59c43f3700 1 --2- 172.21.15.69:0/3674708017 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59bc07d560 0x7f59bc10c480 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f59b80098a0 tx=0x7f59b8009bb0 comp rx=0 tx=0).stop 2024-04-25T11:16:06.482 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:00.315+0000 7f59c43f3700 1 -- 172.21.15.69:0/3674708017 shutdown_connections 2024-04-25T11:16:06.482 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:00.315+0000 7f59c43f3700 1 --2- 172.21.15.69:0/3674708017 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59bc07d560 0x7f59bc10c480 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:06.482 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:00.315+0000 7f59c43f3700 1 -- 172.21.15.69:0/3674708017 >> 172.21.15.69:0/3674708017 conn(0x7f59bc103eb0 msgr2=0x7f59bc106320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:06.482 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:00.315+0000 7f59c43f3700 1 -- 172.21.15.69:0/3674708017 shutdown_connections 2024-04-25T11:16:06.482 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:00.315+0000 7f59c43f3700 1 -- 172.21.15.69:0/3674708017 wait complete. 2024-04-25T11:16:06.483 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:00.316+0000 7f59c43f3700 1 Processor -- start 2024-04-25T11:16:06.483 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:00.316+0000 7f59c43f3700 1 -- start start 2024-04-25T11:16:06.483 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:00.316+0000 7f59c43f3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59bc07d560 0x7f59bc109bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:06.483 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:00.316+0000 7f59c43f3700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc10dba0 con 0x7f59bc07d560 2024-04-25T11:16:06.483 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:00.316+0000 7f59c218f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59bc07d560 0x7f59bc109bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:06.483 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:00.317+0000 7f59c218f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59bc07d560 0x7f59bc109bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50204/0 (socket says 172.21.15.69:50204) 2024-04-25T11:16:06.483 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:00.317+0000 7f59c218f700 1 -- 172.21.15.69:0/3964894966 learned_addr learned my addr 172.21.15.69:0/3964894966 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:16:06.484 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:00.317+0000 7f59c218f700 1 -- 172.21.15.69:0/3964894966 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59bc10a130 con 0x7f59bc07d560 2024-04-25T11:16:06.484 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:00.317+0000 7f59c218f700 1 --2- 172.21.15.69:0/3964894966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59bc07d560 0x7f59bc109bf0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f59b80119b0 tx=0x7f59b8011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:06.484 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:00.317+0000 7f59b37fe700 1 -- 172.21.15.69:0/3964894966 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59b8003af0 con 0x7f59bc07d560 2024-04-25T11:16:06.484 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:00.317+0000 7f59b37fe700 1 -- 172.21.15.69:0/3964894966 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f59b801a8a0 con 0x7f59bc07d560 2024-04-25T11:16:06.484 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:00.317+0000 7f59b37fe700 1 -- 172.21.15.69:0/3964894966 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59b8024830 con 0x7f59bc07d560 2024-04-25T11:16:06.484 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:00.317+0000 7f59c43f3700 1 -- 172.21.15.69:0/3964894966 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59bc108380 con 0x7f59bc07d560 2024-04-25T11:16:06.484 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:00.317+0000 7f59c43f3700 1 -- 172.21.15.69:0/3964894966 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59bc108860 con 0x7f59bc07d560 2024-04-25T11:16:06.485 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:00.318+0000 7f59b37fe700 1 -- 172.21.15.69:0/3964894966 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 49516+0+0 (secure 0 0 0) 0x7f59b8022020 con 0x7f59bc07d560 2024-04-25T11:16:06.485 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:00.319+0000 7f59b37fe700 1 --2- 172.21.15.69:0/3964894966 >> [v2:172.21.15.69:6800/1508198215,v1:172.21.15.69:6801/1508198215] conn(0x7f59a803d680 0x7f59a803fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:06.485 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:00.319+0000 7f59b37fe700 1 -- 172.21.15.69:0/3964894966 --> [v2:172.21.15.69:6800/1508198215,v1:172.21.15.69:6801/1508198215] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f59a80402a0 con 0x7f59a803d680 2024-04-25T11:16:06.485 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:00.319+0000 7f59b37fe700 1 -- 172.21.15.69:0/3964894966 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f59b8052350 con 0x7f59bc07d560 2024-04-25T11:16:06.485 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:00.319+0000 7f59c198e700 1 -- 172.21.15.69:0/3964894966 >> [v2:172.21.15.69:6800/1508198215,v1:172.21.15.69:6801/1508198215] conn(0x7f59a803d680 msgr2=0x7f59a803fb90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/1508198215 2024-04-25T11:16:06.485 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:00.319+0000 7f59c198e700 1 --2- 172.21.15.69:0/3964894966 >> [v2:172.21.15.69:6800/1508198215,v1:172.21.15.69:6801/1508198215] conn(0x7f59a803d680 0x7f59a803fb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T11:16:06.486 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:00.520+0000 7f59c198e700 1 -- 172.21.15.69:0/3964894966 >> [v2:172.21.15.69:6800/1508198215,v1:172.21.15.69:6801/1508198215] conn(0x7f59a803d680 msgr2=0x7f59a803fb90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/1508198215 2024-04-25T11:16:06.486 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:00.520+0000 7f59c198e700 1 --2- 172.21.15.69:0/3964894966 >> [v2:172.21.15.69:6800/1508198215,v1:172.21.15.69:6801/1508198215] conn(0x7f59a803d680 0x7f59a803fb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T11:16:06.486 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:00.921+0000 7f59c198e700 1 -- 172.21.15.69:0/3964894966 >> [v2:172.21.15.69:6800/1508198215,v1:172.21.15.69:6801/1508198215] conn(0x7f59a803d680 msgr2=0x7f59a803fb90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/1508198215 2024-04-25T11:16:06.486 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:00.921+0000 7f59c198e700 1 --2- 172.21.15.69:0/3964894966 >> [v2:172.21.15.69:6800/1508198215,v1:172.21.15.69:6801/1508198215] conn(0x7f59a803d680 0x7f59a803fb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T11:16:06.486 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:01.722+0000 7f59c198e700 1 -- 172.21.15.69:0/3964894966 >> [v2:172.21.15.69:6800/1508198215,v1:172.21.15.69:6801/1508198215] conn(0x7f59a803d680 msgr2=0x7f59a803fb90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/1508198215 2024-04-25T11:16:06.486 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:01.722+0000 7f59c198e700 1 --2- 172.21.15.69:0/3964894966 >> [v2:172.21.15.69:6800/1508198215,v1:172.21.15.69:6801/1508198215] conn(0x7f59a803d680 0x7f59a803fb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T11:16:06.486 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:03.323+0000 7f59c198e700 1 -- 172.21.15.69:0/3964894966 >> [v2:172.21.15.69:6800/1508198215,v1:172.21.15.69:6801/1508198215] conn(0x7f59a803d680 msgr2=0x7f59a803fb90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/1508198215 2024-04-25T11:16:06.486 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:03.323+0000 7f59c198e700 1 --2- 172.21.15.69:0/3964894966 >> [v2:172.21.15.69:6800/1508198215,v1:172.21.15.69:6801/1508198215] conn(0x7f59a803d680 0x7f59a803fb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T11:16:06.486 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:04.861+0000 7f59b37fe700 1 -- 172.21.15.69:0/3964894966 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mgrmap(e 6) ==== 49283+0+0 (secure 0 0 0) 0x7f59b802d530 con 0x7f59bc07d560 2024-04-25T11:16:06.486 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:04.861+0000 7f59b37fe700 1 -- 172.21.15.69:0/3964894966 >> [v2:172.21.15.69:6800/1508198215,v1:172.21.15.69:6801/1508198215] conn(0x7f59a803d680 msgr2=0x7f59a803fb90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:16:06.486 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:04.861+0000 7f59b37fe700 1 --2- 172.21.15.69:0/3964894966 >> [v2:172.21.15.69:6800/1508198215,v1:172.21.15.69:6801/1508198215] conn(0x7f59a803d680 0x7f59a803fb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:06.486 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:05.862+0000 7f59b37fe700 1 -- 172.21.15.69:0/3964894966 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mgrmap(e 7) ==== 49410+0+0 (secure 0 0 0) 0x7f59b8054300 con 0x7f59bc07d560 2024-04-25T11:16:06.486 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:05.862+0000 7f59b37fe700 1 --2- 172.21.15.69:0/3964894966 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f59a8040fc0 0x7f59a80433e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:06.487 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:05.862+0000 7f59b37fe700 1 -- 172.21.15.69:0/3964894966 --> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f59a80402a0 con 0x7f59a8040fc0 2024-04-25T11:16:06.487 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:05.863+0000 7f59c198e700 1 --2- 172.21.15.69:0/3964894966 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f59a8040fc0 0x7f59a80433e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:06.487 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:05.864+0000 7f59c198e700 1 --2- 172.21.15.69:0/3964894966 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f59a8040fc0 0x7f59a80433e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f59ac003a10 tx=0x7f59ac0092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:06.487 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:05.864+0000 7f59b37fe700 1 -- 172.21.15.69:0/3964894966 <== mgr.14120 v2:172.21.15.69:6800/1097124842 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f59a80402a0 con 0x7f59a8040fc0 2024-04-25T11:16:06.487 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:05.872+0000 7f59c43f3700 1 -- 172.21.15.69:0/3964894966 --> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f59bc064cf0 con 0x7f59a8040fc0 2024-04-25T11:16:06.487 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:05.873+0000 7f59b37fe700 1 -- 172.21.15.69:0/3964894966 <== mgr.14120 v2:172.21.15.69:6800/1097124842 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f59bc064cf0 con 0x7f59a8040fc0 2024-04-25T11:16:06.487 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:05.873+0000 7f59c43f3700 1 -- 172.21.15.69:0/3964894966 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f59a8040fc0 msgr2=0x7f59a80433e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:06.487 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:05.873+0000 7f59c43f3700 1 --2- 172.21.15.69:0/3964894966 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f59a8040fc0 0x7f59a80433e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f59ac003a10 tx=0x7f59ac0092b0 comp rx=0 tx=0).stop 2024-04-25T11:16:06.487 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:05.873+0000 7f59c43f3700 1 -- 172.21.15.69:0/3964894966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59bc07d560 msgr2=0x7f59bc109bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:06.487 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:05.873+0000 7f59c43f3700 1 --2- 172.21.15.69:0/3964894966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59bc07d560 0x7f59bc109bf0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f59b80119b0 tx=0x7f59b8011a90 comp rx=0 tx=0).stop 2024-04-25T11:16:06.487 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:05.873+0000 7f59c43f3700 1 -- 172.21.15.69:0/3964894966 shutdown_connections 2024-04-25T11:16:06.487 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:05.873+0000 7f59c43f3700 1 --2- 172.21.15.69:0/3964894966 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f59a8040fc0 0x7f59a80433e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:06.487 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:05.873+0000 7f59c43f3700 1 --2- 172.21.15.69:0/3964894966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59bc07d560 0x7f59bc109bf0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:06.487 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:05.873+0000 7f59c43f3700 1 --2- 172.21.15.69:0/3964894966 >> [v2:172.21.15.69:6800/1508198215,v1:172.21.15.69:6801/1508198215] conn(0x7f59a803d680 0x7f59a803fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:06.488 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:05.873+0000 7f59c43f3700 1 -- 172.21.15.69:0/3964894966 >> 172.21.15.69:0/3964894966 conn(0x7f59bc103eb0 msgr2=0x7f59bc104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:06.488 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:05.873+0000 7f59c43f3700 1 -- 172.21.15.69:0/3964894966 shutdown_connections 2024-04-25T11:16:06.488 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:05.873+0000 7f59c43f3700 1 -- 172.21.15.69:0/3964894966 wait complete. 2024-04-25T11:16:06.488 INFO:teuthology.orchestra.run.smithi069.stdout:mgr epoch 5 is available 2024-04-25T11:16:06.488 INFO:teuthology.orchestra.run.smithi069.stdout:Setting orchestrator backend to cephadm... 2024-04-25T11:16:07.131 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:06 smithi069 ceph-mon[29188]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T11:16:07.131 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:06 smithi069 ceph-mon[29188]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T11:16:07.131 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:06 smithi069 ceph-mon[29188]: from='mgr.14120 172.21.15.69:0/157690041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:16:07.708 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:06.905+0000 7f68fc933700 1 Processor -- start 2024-04-25T11:16:07.708 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:06.905+0000 7f68fc933700 1 -- start start 2024-04-25T11:16:07.708 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:06.905+0000 7f68fc933700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68f807ccd0 0x7f68f810c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:07.708 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:06.905+0000 7f68fc933700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f68f810ca20 con 0x7f68f807ccd0 2024-04-25T11:16:07.708 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:06.905+0000 7f68f659c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68f807ccd0 0x7f68f810c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:07.708 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:06.906+0000 7f68f659c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68f807ccd0 0x7f68f810c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35388/0 (socket says 172.21.15.69:35388) 2024-04-25T11:16:07.708 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:06.906+0000 7f68f659c700 1 -- 172.21.15.69:0/1709291512 learned_addr learned my addr 172.21.15.69:0/1709291512 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:16:07.708 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:06.906+0000 7f68f659c700 1 -- 172.21.15.69:0/1709291512 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68f810cba0 con 0x7f68f807ccd0 2024-04-25T11:16:07.708 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:06.906+0000 7f68f659c700 1 --2- 172.21.15.69:0/1709291512 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68f807ccd0 0x7f68f810c430 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f68e8009e80 tx=0x7f68e800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52a8814c1ee1a60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:07.709 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:06.906+0000 7f68f5d9b700 1 -- 172.21.15.69:0/1709291512 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f68e8017070 con 0x7f68f807ccd0 2024-04-25T11:16:07.709 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:06.906+0000 7f68f5d9b700 1 -- 172.21.15.69:0/1709291512 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f68e800b8c0 con 0x7f68f807ccd0 2024-04-25T11:16:07.709 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:06.906+0000 7f68f5d9b700 1 -- 172.21.15.69:0/1709291512 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f68e8004420 con 0x7f68f807ccd0 2024-04-25T11:16:07.709 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:06.907+0000 7f68fc933700 1 -- 172.21.15.69:0/1709291512 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68f807ccd0 msgr2=0x7f68f810c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:07.709 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:06.907+0000 7f68fc933700 1 --2- 172.21.15.69:0/1709291512 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68f807ccd0 0x7f68f810c430 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f68e8009e80 tx=0x7f68e800b330 comp rx=0 tx=0).stop 2024-04-25T11:16:07.709 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:06.907+0000 7f68fc933700 1 -- 172.21.15.69:0/1709291512 shutdown_connections 2024-04-25T11:16:07.709 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:06.907+0000 7f68fc933700 1 --2- 172.21.15.69:0/1709291512 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68f807ccd0 0x7f68f810c430 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:07.709 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:06.907+0000 7f68fc933700 1 -- 172.21.15.69:0/1709291512 >> 172.21.15.69:0/1709291512 conn(0x7f68f8103ee0 msgr2=0x7f68f8106330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:07.709 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:06.907+0000 7f68fc933700 1 -- 172.21.15.69:0/1709291512 shutdown_connections 2024-04-25T11:16:07.709 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:06.907+0000 7f68fc933700 1 -- 172.21.15.69:0/1709291512 wait complete. 2024-04-25T11:16:07.709 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:06.908+0000 7f68fc933700 1 Processor -- start 2024-04-25T11:16:07.709 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:06.908+0000 7f68fc933700 1 -- start start 2024-04-25T11:16:07.709 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:06.908+0000 7f68fc933700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68f807ccd0 0x7f68f819db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:07.709 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:06.908+0000 7f68fc933700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f68f810db50 con 0x7f68f807ccd0 2024-04-25T11:16:07.710 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:06.908+0000 7f68f659c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68f807ccd0 0x7f68f819db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:07.710 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:06.908+0000 7f68f659c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68f807ccd0 0x7f68f819db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35402/0 (socket says 172.21.15.69:35402) 2024-04-25T11:16:07.710 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:06.908+0000 7f68f659c700 1 -- 172.21.15.69:0/2922206093 learned_addr learned my addr 172.21.15.69:0/2922206093 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:16:07.710 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:06.909+0000 7f68f659c700 1 -- 172.21.15.69:0/2922206093 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68f819e050 con 0x7f68f807ccd0 2024-04-25T11:16:07.710 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:06.909+0000 7f68f659c700 1 --2- 172.21.15.69:0/2922206093 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68f807ccd0 0x7f68f819db10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f68e8009e50 tx=0x7f68e800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:07.710 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:06.909+0000 7f68effff700 1 -- 172.21.15.69:0/2922206093 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f68e8020020 con 0x7f68f807ccd0 2024-04-25T11:16:07.710 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:06.909+0000 7f68effff700 1 -- 172.21.15.69:0/2922206093 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f68e8012430 con 0x7f68f807ccd0 2024-04-25T11:16:07.710 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:06.909+0000 7f68effff700 1 -- 172.21.15.69:0/2922206093 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f68e8017040 con 0x7f68f807ccd0 2024-04-25T11:16:07.710 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:06.909+0000 7f68fc933700 1 -- 172.21.15.69:0/2922206093 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68f819e2e0 con 0x7f68f807ccd0 2024-04-25T11:16:07.710 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:06.909+0000 7f68fc933700 1 -- 172.21.15.69:0/2922206093 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68f81a0ef0 con 0x7f68f807ccd0 2024-04-25T11:16:07.710 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:06.910+0000 7f68effff700 1 -- 172.21.15.69:0/2922206093 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 7) ==== 49410+0+0 (secure 0 0 0) 0x7f68e8023020 con 0x7f68f807ccd0 2024-04-25T11:16:07.710 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:06.910+0000 7f68fc933700 1 -- 172.21.15.69:0/2922206093 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68f8064cf0 con 0x7f68f807ccd0 2024-04-25T11:16:07.710 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:06.910+0000 7f68effff700 1 --2- 172.21.15.69:0/2922206093 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f68e403d4b0 0x7f68e403f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:07.711 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:06.910+0000 7f68effff700 1 -- 172.21.15.69:0/2922206093 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f68e800f050 con 0x7f68f807ccd0 2024-04-25T11:16:07.711 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:06.910+0000 7f68edd9b700 1 --2- 172.21.15.69:0/2922206093 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f68e403d4b0 0x7f68e403f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:07.711 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:06.911+0000 7f68edd9b700 1 --2- 172.21.15.69:0/2922206093 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f68e403d4b0 0x7f68e403f9c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f68e0006fd0 tx=0x7f68e0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:07.711 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:06.916+0000 7f68effff700 1 -- 172.21.15.69:0/2922206093 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f68e802a020 con 0x7f68f807ccd0 2024-04-25T11:16:07.711 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:06.921+0000 7f68effff700 1 -- 172.21.15.69:0/2922206093 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f68e80125e0 con 0x7f68f807ccd0 2024-04-25T11:16:07.711 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:07.129+0000 7f68fc933700 1 -- 172.21.15.69:0/2922206093 --> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f68f8105f70 con 0x7f68e403d4b0 2024-04-25T11:16:07.711 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:07.135+0000 7f68effff700 1 -- 172.21.15.69:0/2922206093 <== mgr.14120 v2:172.21.15.69:6800/1097124842 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f68f8105f70 con 0x7f68e403d4b0 2024-04-25T11:16:07.711 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:07.137+0000 7f68fc933700 1 -- 172.21.15.69:0/2922206093 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f68e403d4b0 msgr2=0x7f68e403f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:07.711 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:07.137+0000 7f68fc933700 1 --2- 172.21.15.69:0/2922206093 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f68e403d4b0 0x7f68e403f9c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f68e0006fd0 tx=0x7f68e0006e40 comp rx=0 tx=0).stop 2024-04-25T11:16:07.711 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:07.137+0000 7f68fc933700 1 -- 172.21.15.69:0/2922206093 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68f807ccd0 msgr2=0x7f68f819db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:07.711 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:07.137+0000 7f68fc933700 1 --2- 172.21.15.69:0/2922206093 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68f807ccd0 0x7f68f819db10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f68e8009e50 tx=0x7f68e800bd50 comp rx=0 tx=0).stop 2024-04-25T11:16:07.711 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:07.137+0000 7f68fc933700 1 -- 172.21.15.69:0/2922206093 shutdown_connections 2024-04-25T11:16:07.712 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:07.137+0000 7f68fc933700 1 --2- 172.21.15.69:0/2922206093 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f68e403d4b0 0x7f68e403f9c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:07.712 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:07.137+0000 7f68fc933700 1 --2- 172.21.15.69:0/2922206093 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68f807ccd0 0x7f68f819db10 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:07.712 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:07.137+0000 7f68fc933700 1 -- 172.21.15.69:0/2922206093 >> 172.21.15.69:0/2922206093 conn(0x7f68f8103ee0 msgr2=0x7f68f8105860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:07.712 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:07.137+0000 7f68fc933700 1 -- 172.21.15.69:0/2922206093 shutdown_connections 2024-04-25T11:16:07.712 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:07.137+0000 7f68fc933700 1 -- 172.21.15.69:0/2922206093 wait complete. 2024-04-25T11:16:07.961 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:07 smithi069 ceph-mon[29188]: [25/Apr/2024:11:16:06] ENGINE Bus STARTING 2024-04-25T11:16:07.961 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:07 smithi069 ceph-mon[29188]: [25/Apr/2024:11:16:06] ENGINE Serving on http://172.21.15.69:8765 2024-04-25T11:16:07.961 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:07 smithi069 ceph-mon[29188]: [25/Apr/2024:11:16:06] ENGINE Serving on https://172.21.15.69:7150 2024-04-25T11:16:07.962 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:07 smithi069 ceph-mon[29188]: [25/Apr/2024:11:16:06] ENGINE Bus STARTED 2024-04-25T11:16:07.962 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:07 smithi069 ceph-mon[29188]: mgrmap e8: x(active, since 2s) 2024-04-25T11:16:07.962 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:07 smithi069 ceph-mon[29188]: from='mgr.14120 172.21.15.69:0/157690041' entity='mgr.x' 2024-04-25T11:16:07.962 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:07 smithi069 ceph-mon[29188]: from='mgr.14120 172.21.15.69:0/157690041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:16:08.889 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-25T11:16:08.889 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.071+0000 7f3bfa97c700 1 Processor -- start 2024-04-25T11:16:08.889 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.072+0000 7f3bfa97c700 1 -- start start 2024-04-25T11:16:08.889 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.072+0000 7f3bfa97c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3bf407d580 0x7f3bf410c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:08.889 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.072+0000 7f3bfa97c700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf410cb40 con 0x7f3bf407d580 2024-04-25T11:16:08.889 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.072+0000 7f3bf3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3bf407d580 0x7f3bf410c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:08.890 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.072+0000 7f3bf3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3bf407d580 0x7f3bf410c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35412/0 (socket says 172.21.15.69:35412) 2024-04-25T11:16:08.890 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.072+0000 7f3bf3fff700 1 -- 172.21.15.69:0/2573833837 learned_addr learned my addr 172.21.15.69:0/2573833837 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:16:08.890 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.072+0000 7f3bf3fff700 1 -- 172.21.15.69:0/2573833837 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bf410ccc0 con 0x7f3bf407d580 2024-04-25T11:16:08.890 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.072+0000 7f3bf3fff700 1 --2- 172.21.15.69:0/2573833837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3bf407d580 0x7f3bf410c530 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3be40098a0 tx=0x7f3be4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=793a0d823fb373d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:08.890 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.073+0000 7f3bf2ffd700 1 -- 172.21.15.69:0/2573833837 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3be4004160 con 0x7f3bf407d580 2024-04-25T11:16:08.890 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.073+0000 7f3bf2ffd700 1 -- 172.21.15.69:0/2573833837 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f3be4004300 con 0x7f3bf407d580 2024-04-25T11:16:08.890 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.073+0000 7f3bf2ffd700 1 -- 172.21.15.69:0/2573833837 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3be40119e0 con 0x7f3bf407d580 2024-04-25T11:16:08.890 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.073+0000 7f3bfa97c700 1 -- 172.21.15.69:0/2573833837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3bf407d580 msgr2=0x7f3bf410c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:08.890 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.073+0000 7f3bfa97c700 1 --2- 172.21.15.69:0/2573833837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3bf407d580 0x7f3bf410c530 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3be40098a0 tx=0x7f3be4009bb0 comp rx=0 tx=0).stop 2024-04-25T11:16:08.890 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.073+0000 7f3bfa97c700 1 -- 172.21.15.69:0/2573833837 shutdown_connections 2024-04-25T11:16:08.890 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.073+0000 7f3bfa97c700 1 --2- 172.21.15.69:0/2573833837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3bf407d580 0x7f3bf410c530 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:08.890 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.073+0000 7f3bfa97c700 1 -- 172.21.15.69:0/2573833837 >> 172.21.15.69:0/2573833837 conn(0x7f3bf4103fe0 msgr2=0x7f3bf4106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:08.891 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.073+0000 7f3bfa97c700 1 -- 172.21.15.69:0/2573833837 shutdown_connections 2024-04-25T11:16:08.891 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.073+0000 7f3bfa97c700 1 -- 172.21.15.69:0/2573833837 wait complete. 2024-04-25T11:16:08.891 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.074+0000 7f3bfa97c700 1 Processor -- start 2024-04-25T11:16:08.891 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.074+0000 7f3bfa97c700 1 -- start start 2024-04-25T11:16:08.891 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.074+0000 7f3bfa97c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3bf407d580 0x7f3bf419db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:08.891 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.074+0000 7f3bfa97c700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf410dc00 con 0x7f3bf407d580 2024-04-25T11:16:08.891 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.075+0000 7f3bf3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3bf407d580 0x7f3bf419db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:08.891 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.075+0000 7f3bf3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3bf407d580 0x7f3bf419db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35416/0 (socket says 172.21.15.69:35416) 2024-04-25T11:16:08.891 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.075+0000 7f3bf3fff700 1 -- 172.21.15.69:0/320472649 learned_addr learned my addr 172.21.15.69:0/320472649 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:16:08.891 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.075+0000 7f3bf3fff700 1 -- 172.21.15.69:0/320472649 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bf419e070 con 0x7f3bf407d580 2024-04-25T11:16:08.891 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.075+0000 7f3bf3fff700 1 --2- 172.21.15.69:0/320472649 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3bf407d580 0x7f3bf419db30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3be40119b0 tx=0x7f3be4011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:08.891 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.075+0000 7f3bf17fa700 1 -- 172.21.15.69:0/320472649 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3be4003af0 con 0x7f3bf407d580 2024-04-25T11:16:08.892 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.075+0000 7f3bfa97c700 1 -- 172.21.15.69:0/320472649 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bf419e300 con 0x7f3bf407d580 2024-04-25T11:16:08.892 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.075+0000 7f3bf17fa700 1 -- 172.21.15.69:0/320472649 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f3be401a8a0 con 0x7f3bf407d580 2024-04-25T11:16:08.892 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.075+0000 7f3bf17fa700 1 -- 172.21.15.69:0/320472649 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3be4024830 con 0x7f3bf407d580 2024-04-25T11:16:08.892 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.075+0000 7f3bfa97c700 1 -- 172.21.15.69:0/320472649 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bf41a0f10 con 0x7f3bf407d580 2024-04-25T11:16:08.892 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.076+0000 7f3bf17fa700 1 -- 172.21.15.69:0/320472649 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f3be4022020 con 0x7f3bf407d580 2024-04-25T11:16:08.892 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.076+0000 7f3bfa97c700 1 -- 172.21.15.69:0/320472649 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bf4064cf0 con 0x7f3bf407d580 2024-04-25T11:16:08.892 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.076+0000 7f3bf17fa700 1 --2- 172.21.15.69:0/320472649 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f3bdc03d630 0x7f3bdc03fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:08.892 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.076+0000 7f3bf17fa700 1 -- 172.21.15.69:0/320472649 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3be40527a0 con 0x7f3bf407d580 2024-04-25T11:16:08.892 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.076+0000 7f3bf37fe700 1 --2- 172.21.15.69:0/320472649 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f3bdc03d630 0x7f3bdc03fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:08.892 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.077+0000 7f3bf37fe700 1 --2- 172.21.15.69:0/320472649 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f3bdc03d630 0x7f3bdc03fb40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3be8006fd0 tx=0x7f3be8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:08.892 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.082+0000 7f3bf17fa700 1 -- 172.21.15.69:0/320472649 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3be400e260 con 0x7f3bf407d580 2024-04-25T11:16:08.892 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.278+0000 7f3bfa97c700 1 -- 172.21.15.69:0/320472649 --> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f3bf4105ff0 con 0x7f3bdc03d630 2024-04-25T11:16:08.892 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.278+0000 7f3bf17fa700 1 -- 172.21.15.69:0/320472649 <== mgr.14120 v2:172.21.15.69:6800/1097124842 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f3bf4105ff0 con 0x7f3bdc03d630 2024-04-25T11:16:08.892 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.281+0000 7f3bfa97c700 1 -- 172.21.15.69:0/320472649 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f3bdc03d630 msgr2=0x7f3bdc03fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:08.893 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.281+0000 7f3bfa97c700 1 --2- 172.21.15.69:0/320472649 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f3bdc03d630 0x7f3bdc03fb40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3be8006fd0 tx=0x7f3be8006e40 comp rx=0 tx=0).stop 2024-04-25T11:16:08.893 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.281+0000 7f3bfa97c700 1 -- 172.21.15.69:0/320472649 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3bf407d580 msgr2=0x7f3bf419db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:08.893 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.281+0000 7f3bfa97c700 1 --2- 172.21.15.69:0/320472649 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3bf407d580 0x7f3bf419db30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3be40119b0 tx=0x7f3be4011a90 comp rx=0 tx=0).stop 2024-04-25T11:16:08.893 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.281+0000 7f3bfa97c700 1 -- 172.21.15.69:0/320472649 shutdown_connections 2024-04-25T11:16:08.893 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.281+0000 7f3bfa97c700 1 --2- 172.21.15.69:0/320472649 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f3bdc03d630 0x7f3bdc03fb40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:08.893 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.281+0000 7f3bfa97c700 1 --2- 172.21.15.69:0/320472649 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3bf407d580 0x7f3bf419db30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:08.893 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.281+0000 7f3bfa97c700 1 -- 172.21.15.69:0/320472649 >> 172.21.15.69:0/320472649 conn(0x7f3bf4103fe0 msgr2=0x7f3bf41058e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:08.893 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.281+0000 7f3bfa97c700 1 -- 172.21.15.69:0/320472649 shutdown_connections 2024-04-25T11:16:08.893 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:08.281+0000 7f3bfa97c700 1 -- 172.21.15.69:0/320472649 wait complete. 2024-04-25T11:16:08.893 INFO:teuthology.orchestra.run.smithi069.stdout:Generating ssh key... 2024-04-25T11:16:09.192 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:08 smithi069 ceph-mon[29188]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:16:10.154 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:09 smithi069 ceph-mon[29188]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:16:10.154 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:09 smithi069 ceph-mon[29188]: from='mgr.14120 172.21.15.69:0/157690041' entity='mgr.x' 2024-04-25T11:16:10.154 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:09 smithi069 ceph-mon[29188]: from='mgr.14120 172.21.15.69:0/157690041' entity='mgr.x' 2024-04-25T11:16:10.154 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:09 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: Generating public/private rsa key pair. 2024-04-25T11:16:10.155 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:09 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: Your identification has been saved in /tmp/tmptpyh50mj/key. 2024-04-25T11:16:10.155 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:09 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: Your public key has been saved in /tmp/tmptpyh50mj/key.pub. 2024-04-25T11:16:10.155 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:09 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: The key fingerprint is: 2024-04-25T11:16:10.155 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:09 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: SHA256:V9zzcokrRVpInQrwB3R8WCm2+TEpsPnyT/PFVRN4KdA ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968 2024-04-25T11:16:10.155 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:09 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: The key's randomart image is: 2024-04-25T11:16:10.155 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:09 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: +---[RSA 3072]----+ 2024-04-25T11:16:10.155 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:09 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: | .oo.+*.+..| 2024-04-25T11:16:10.155 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:09 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: | .o+*.E o.| 2024-04-25T11:16:10.155 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:09 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: | .*oOo*..| 2024-04-25T11:16:10.155 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:09 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: | o.*++.o+| 2024-04-25T11:16:10.155 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:09 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: | S o.oo+.+| 2024-04-25T11:16:10.155 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:09 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: | o ....+.| 2024-04-25T11:16:10.155 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:09 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: | o. + o| 2024-04-25T11:16:10.155 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:09 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: | .o o .| 2024-04-25T11:16:10.155 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:09 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: | .. . | 2024-04-25T11:16:10.156 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:09 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: +----[SHA256]-----+ 2024-04-25T11:16:10.463 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.306+0000 7fa72e10f700 1 Processor -- start 2024-04-25T11:16:10.463 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.306+0000 7fa72e10f700 1 -- start start 2024-04-25T11:16:10.463 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.306+0000 7fa72e10f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa728109980 0x7fa728109d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:10.463 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.306+0000 7fa72e10f700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa72810a380 con 0x7fa728109980 2024-04-25T11:16:10.463 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.306+0000 7fa7277fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa728109980 0x7fa728109d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:10.463 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.306+0000 7fa7277fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa728109980 0x7fa728109d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35420/0 (socket says 172.21.15.69:35420) 2024-04-25T11:16:10.463 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.306+0000 7fa7277fe700 1 -- 172.21.15.69:0/1408143000 learned_addr learned my addr 172.21.15.69:0/1408143000 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:16:10.463 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.307+0000 7fa7277fe700 1 -- 172.21.15.69:0/1408143000 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa72810aab0 con 0x7fa728109980 2024-04-25T11:16:10.463 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.307+0000 7fa7277fe700 1 --2- 172.21.15.69:0/1408143000 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa728109980 0x7fa728109d90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa7180098a0 tx=0x7fa718009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd495b730103bbb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:10.463 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.307+0000 7fa7267fc700 1 -- 172.21.15.69:0/1408143000 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa718004160 con 0x7fa728109980 2024-04-25T11:16:10.463 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.307+0000 7fa7267fc700 1 -- 172.21.15.69:0/1408143000 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fa718004300 con 0x7fa728109980 2024-04-25T11:16:10.464 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.307+0000 7fa7267fc700 1 -- 172.21.15.69:0/1408143000 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7180119e0 con 0x7fa728109980 2024-04-25T11:16:10.464 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.308+0000 7fa72e10f700 1 -- 172.21.15.69:0/1408143000 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa728109980 msgr2=0x7fa728109d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:10.464 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.308+0000 7fa72e10f700 1 --2- 172.21.15.69:0/1408143000 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa728109980 0x7fa728109d90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa7180098a0 tx=0x7fa718009bb0 comp rx=0 tx=0).stop 2024-04-25T11:16:10.464 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.308+0000 7fa72e10f700 1 -- 172.21.15.69:0/1408143000 shutdown_connections 2024-04-25T11:16:10.464 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.308+0000 7fa72e10f700 1 --2- 172.21.15.69:0/1408143000 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa728109980 0x7fa728109d90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:10.464 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.308+0000 7fa72e10f700 1 -- 172.21.15.69:0/1408143000 >> 172.21.15.69:0/1408143000 conn(0x7fa728105080 msgr2=0x7fa7281074d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:10.464 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.308+0000 7fa72e10f700 1 -- 172.21.15.69:0/1408143000 shutdown_connections 2024-04-25T11:16:10.464 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.308+0000 7fa72e10f700 1 -- 172.21.15.69:0/1408143000 wait complete. 2024-04-25T11:16:10.464 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.308+0000 7fa72e10f700 1 Processor -- start 2024-04-25T11:16:10.464 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.309+0000 7fa72e10f700 1 -- start start 2024-04-25T11:16:10.464 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.309+0000 7fa72e10f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa728109980 0x7fa72819dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:10.464 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.309+0000 7fa72e10f700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa72810dce0 con 0x7fa728109980 2024-04-25T11:16:10.464 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.309+0000 7fa7277fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa728109980 0x7fa72819dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:10.465 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.309+0000 7fa7277fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa728109980 0x7fa72819dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35432/0 (socket says 172.21.15.69:35432) 2024-04-25T11:16:10.465 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.309+0000 7fa7277fe700 1 -- 172.21.15.69:0/4032117105 learned_addr learned my addr 172.21.15.69:0/4032117105 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:16:10.465 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.309+0000 7fa7277fe700 1 -- 172.21.15.69:0/4032117105 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa72819e100 con 0x7fa728109980 2024-04-25T11:16:10.465 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.309+0000 7fa7277fe700 1 --2- 172.21.15.69:0/4032117105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa728109980 0x7fa72819dbc0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa718009fb0 tx=0x7fa718011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:10.465 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.309+0000 7fa724ff9700 1 -- 172.21.15.69:0/4032117105 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7180038b0 con 0x7fa728109980 2024-04-25T11:16:10.465 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.310+0000 7fa724ff9700 1 -- 172.21.15.69:0/4032117105 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fa718003a50 con 0x7fa728109980 2024-04-25T11:16:10.465 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.310+0000 7fa72e10f700 1 -- 172.21.15.69:0/4032117105 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa72819e390 con 0x7fa728109980 2024-04-25T11:16:10.465 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.310+0000 7fa724ff9700 1 -- 172.21.15.69:0/4032117105 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa718019550 con 0x7fa728109980 2024-04-25T11:16:10.465 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.310+0000 7fa72e10f700 1 -- 172.21.15.69:0/4032117105 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7281a0fa0 con 0x7fa728109980 2024-04-25T11:16:10.465 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.310+0000 7fa72e10f700 1 -- 172.21.15.69:0/4032117105 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa728064cf0 con 0x7fa728109980 2024-04-25T11:16:10.465 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.310+0000 7fa724ff9700 1 -- 172.21.15.69:0/4032117105 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7fa718019800 con 0x7fa728109980 2024-04-25T11:16:10.465 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.311+0000 7fa724ff9700 1 --2- 172.21.15.69:0/4032117105 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7fa71003d630 0x7fa71003fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:10.465 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.311+0000 7fa724ff9700 1 -- 172.21.15.69:0/4032117105 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa718052920 con 0x7fa728109980 2024-04-25T11:16:10.465 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.311+0000 7fa726ffd700 1 --2- 172.21.15.69:0/4032117105 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7fa71003d630 0x7fa71003fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:10.465 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.311+0000 7fa726ffd700 1 --2- 172.21.15.69:0/4032117105 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7fa71003d630 0x7fa71003fb40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa71c006fd0 tx=0x7fa71c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:10.466 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.316+0000 7fa724ff9700 1 -- 172.21.15.69:0/4032117105 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa718017e90 con 0x7fa728109980 2024-04-25T11:16:10.466 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.513+0000 7fa72e10f700 1 -- 172.21.15.69:0/4032117105 --> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fa728107040 con 0x7fa71003d630 2024-04-25T11:16:10.466 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.798+0000 7fa724ff9700 1 -- 172.21.15.69:0/4032117105 <== mgr.14120 v2:172.21.15.69:6800/1097124842 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa728107040 con 0x7fa71003d630 2024-04-25T11:16:10.466 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.800+0000 7fa72e10f700 1 -- 172.21.15.69:0/4032117105 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7fa71003d630 msgr2=0x7fa71003fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:10.466 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.800+0000 7fa72e10f700 1 --2- 172.21.15.69:0/4032117105 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7fa71003d630 0x7fa71003fb40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa71c006fd0 tx=0x7fa71c006e40 comp rx=0 tx=0).stop 2024-04-25T11:16:10.466 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.800+0000 7fa72e10f700 1 -- 172.21.15.69:0/4032117105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa728109980 msgr2=0x7fa72819dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:10.466 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.800+0000 7fa72e10f700 1 --2- 172.21.15.69:0/4032117105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa728109980 0x7fa72819dbc0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa718009fb0 tx=0x7fa718011a60 comp rx=0 tx=0).stop 2024-04-25T11:16:10.466 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.800+0000 7fa72e10f700 1 -- 172.21.15.69:0/4032117105 shutdown_connections 2024-04-25T11:16:10.466 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.800+0000 7fa72e10f700 1 --2- 172.21.15.69:0/4032117105 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7fa71003d630 0x7fa71003fb40 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:10.466 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.800+0000 7fa72e10f700 1 --2- 172.21.15.69:0/4032117105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa728109980 0x7fa72819dbc0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:10.466 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.800+0000 7fa72e10f700 1 -- 172.21.15.69:0/4032117105 >> 172.21.15.69:0/4032117105 conn(0x7fa728105080 msgr2=0x7fa728106930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:10.466 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.800+0000 7fa72e10f700 1 -- 172.21.15.69:0/4032117105 shutdown_connections 2024-04-25T11:16:10.466 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:09.800+0000 7fa72e10f700 1 -- 172.21.15.69:0/4032117105 wait complete. 2024-04-25T11:16:11.131 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:10 smithi069 ceph-mon[29188]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:16:11.131 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:10 smithi069 ceph-mon[29188]: Generating ssh key... 2024-04-25T11:16:11.678 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCgJuPV2zxWIMKKqp4AJzUGITwFGTOK0sf/5Co4CWi/CbQOkM3oIl4igo3w/hWgAwHHyUVgSQEAoBMhDeLfLOZM9RSGL1Jiq1oFjdWv0hCGhSDQa6j3obZp5sk9HwNRz0HrQLxu/KAjZSddFt/NDfi13cSLBVPLrZIVq7Ep0DARQkxBnlhRlezB3OaIRVRPOiOiwjzeXRBja85nnwbt8QQf9P7imPMusSd8ef0fSDlFJJ8tc/SWO1/ZhsGMNnCdHAcOuhDd0Bu9Je+UJjEcFzOwxTPE8Bf/8l+Sw/FXt/bHsspDjDAIRzyzNpqaVyLYErHTfHa8wwGKtofmMuFS6n0FGuAh5wrnFkTS6oQhC+2cZkhjlbMIwcnyBel1LfyUpkZ1zJCDvRIRM1XGj5RFUDxfkt2sce6D3kBt4VX43EthJ5BHQQ3PsNchDk7UVlyuEzgOZqTfCVrRdWwsY+knUVRZbLDi+m/QRn0KmRV9KICLQsv8Of1PnBPcmXj8TRBVMuU= ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968 2024-04-25T11:16:11.678 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:10.923+0000 7f780359e700 1 Processor -- start 2024-04-25T11:16:11.678 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:10.923+0000 7f780359e700 1 -- start start 2024-04-25T11:16:11.678 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:10.923+0000 7f780359e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78041086a0 0x7f7804108ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:11.678 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:10.923+0000 7f780359e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f78041090a0 con 0x7f78041086a0 2024-04-25T11:16:11.678 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:10.923+0000 7f780259c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78041086a0 0x7f7804108ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:11.678 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:10.923+0000 7f780259c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78041086a0 0x7f7804108ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35442/0 (socket says 172.21.15.69:35442) 2024-04-25T11:16:11.678 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:10.923+0000 7f780259c700 1 -- 172.21.15.69:0/3289124370 learned_addr learned my addr 172.21.15.69:0/3289124370 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:16:11.678 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:10.924+0000 7f780259c700 1 -- 172.21.15.69:0/3289124370 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78041097d0 con 0x7f78041086a0 2024-04-25T11:16:11.678 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:10.924+0000 7f780259c700 1 --2- 172.21.15.69:0/3289124370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78041086a0 0x7f7804108ab0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f77f40098a0 tx=0x7f77f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e26a111db99228f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:11.678 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:10.926+0000 7f780159a700 1 -- 172.21.15.69:0/3289124370 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77f4004160 con 0x7f78041086a0 2024-04-25T11:16:11.678 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:10.926+0000 7f780159a700 1 -- 172.21.15.69:0/3289124370 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f77f4004300 con 0x7f78041086a0 2024-04-25T11:16:11.679 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:10.926+0000 7f780159a700 1 -- 172.21.15.69:0/3289124370 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77f40119e0 con 0x7f78041086a0 2024-04-25T11:16:11.679 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:10.926+0000 7f780359e700 1 -- 172.21.15.69:0/3289124370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78041086a0 msgr2=0x7f7804108ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:11.679 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:10.926+0000 7f780359e700 1 --2- 172.21.15.69:0/3289124370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78041086a0 0x7f7804108ab0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f77f40098a0 tx=0x7f77f4009bb0 comp rx=0 tx=0).stop 2024-04-25T11:16:11.679 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:10.927+0000 7f780359e700 1 -- 172.21.15.69:0/3289124370 shutdown_connections 2024-04-25T11:16:11.679 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:10.927+0000 7f780359e700 1 --2- 172.21.15.69:0/3289124370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78041086a0 0x7f7804108ab0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:11.679 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:10.927+0000 7f780359e700 1 -- 172.21.15.69:0/3289124370 >> 172.21.15.69:0/3289124370 conn(0x7f7804103d00 msgr2=0x7f7804106190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:11.679 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:10.927+0000 7f780359e700 1 -- 172.21.15.69:0/3289124370 shutdown_connections 2024-04-25T11:16:11.679 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:10.927+0000 7f780359e700 1 -- 172.21.15.69:0/3289124370 wait complete. 2024-04-25T11:16:11.679 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:10.927+0000 7f780359e700 1 Processor -- start 2024-04-25T11:16:11.679 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:10.927+0000 7f780359e700 1 -- start start 2024-04-25T11:16:11.679 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:10.928+0000 7f780359e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78041086a0 0x7f780419d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:11.679 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:10.928+0000 7f780359e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f780410ca00 con 0x7f78041086a0 2024-04-25T11:16:11.679 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:10.928+0000 7f780259c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78041086a0 0x7f780419d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:11.680 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:10.928+0000 7f780259c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78041086a0 0x7f780419d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35458/0 (socket says 172.21.15.69:35458) 2024-04-25T11:16:11.680 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:10.928+0000 7f780259c700 1 -- 172.21.15.69:0/3913091096 learned_addr learned my addr 172.21.15.69:0/3913091096 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:16:11.680 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:10.928+0000 7f780259c700 1 -- 172.21.15.69:0/3913091096 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f780419df00 con 0x7f78041086a0 2024-04-25T11:16:11.680 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:10.928+0000 7f780259c700 1 --2- 172.21.15.69:0/3913091096 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78041086a0 0x7f780419d9c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f77f40119b0 tx=0x7f77f4011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:11.680 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:10.928+0000 7f77f37fe700 1 -- 172.21.15.69:0/3913091096 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77f4003970 con 0x7f78041086a0 2024-04-25T11:16:11.680 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:10.928+0000 7f77f37fe700 1 -- 172.21.15.69:0/3913091096 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f77f4003b10 con 0x7f78041086a0 2024-04-25T11:16:11.680 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:10.928+0000 7f77f37fe700 1 -- 172.21.15.69:0/3913091096 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77f4024610 con 0x7f78041086a0 2024-04-25T11:16:11.680 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:10.928+0000 7f780359e700 1 -- 172.21.15.69:0/3913091096 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f780419e190 con 0x7f78041086a0 2024-04-25T11:16:11.680 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:10.929+0000 7f780359e700 1 -- 172.21.15.69:0/3913091096 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78041a0da0 con 0x7f78041086a0 2024-04-25T11:16:11.680 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:10.929+0000 7f77f37fe700 1 -- 172.21.15.69:0/3913091096 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f77f402d3f0 con 0x7f78041086a0 2024-04-25T11:16:11.680 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:10.929+0000 7f780359e700 1 -- 172.21.15.69:0/3913091096 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7804064cf0 con 0x7f78041086a0 2024-04-25T11:16:11.680 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:10.930+0000 7f77f37fe700 1 --2- 172.21.15.69:0/3913091096 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f77ec03d630 0x7f77ec03fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:11.680 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:10.930+0000 7f77f37fe700 1 -- 172.21.15.69:0/3913091096 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f77f4052770 con 0x7f78041086a0 2024-04-25T11:16:11.680 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:10.930+0000 7f7801d9b700 1 --2- 172.21.15.69:0/3913091096 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f77ec03d630 0x7f77ec03fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:11.681 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:10.930+0000 7f7801d9b700 1 --2- 172.21.15.69:0/3913091096 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f77ec03d630 0x7f77ec03fb40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f77f8006fd0 tx=0x7f77f8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:11.681 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:10.935+0000 7f77f37fe700 1 -- 172.21.15.69:0/3913091096 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f77f40094f0 con 0x7f78041086a0 2024-04-25T11:16:11.681 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:11.130+0000 7f780359e700 1 -- 172.21.15.69:0/3913091096 --> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f780402e880 con 0x7f77ec03d630 2024-04-25T11:16:11.681 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:11.131+0000 7f77f37fe700 1 -- 172.21.15.69:0/3913091096 <== mgr.14120 v2:172.21.15.69:6800/1097124842 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f780402e880 con 0x7f77ec03d630 2024-04-25T11:16:11.681 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:11.133+0000 7f780359e700 1 -- 172.21.15.69:0/3913091096 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f77ec03d630 msgr2=0x7f77ec03fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:11.681 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:11.133+0000 7f780359e700 1 --2- 172.21.15.69:0/3913091096 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f77ec03d630 0x7f77ec03fb40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f77f8006fd0 tx=0x7f77f8006e40 comp rx=0 tx=0).stop 2024-04-25T11:16:11.681 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:11.133+0000 7f780359e700 1 -- 172.21.15.69:0/3913091096 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78041086a0 msgr2=0x7f780419d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:11.681 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:11.133+0000 7f780359e700 1 --2- 172.21.15.69:0/3913091096 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78041086a0 0x7f780419d9c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f77f40119b0 tx=0x7f77f4011a90 comp rx=0 tx=0).stop 2024-04-25T11:16:11.681 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:11.133+0000 7f780359e700 1 -- 172.21.15.69:0/3913091096 shutdown_connections 2024-04-25T11:16:11.681 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:11.133+0000 7f780359e700 1 --2- 172.21.15.69:0/3913091096 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f77ec03d630 0x7f77ec03fb40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:11.681 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:11.133+0000 7f780359e700 1 --2- 172.21.15.69:0/3913091096 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78041086a0 0x7f780419d9c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:11.681 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:11.133+0000 7f780359e700 1 -- 172.21.15.69:0/3913091096 >> 172.21.15.69:0/3913091096 conn(0x7f7804103d00 msgr2=0x7f7804193f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:11.681 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:11.133+0000 7f780359e700 1 -- 172.21.15.69:0/3913091096 shutdown_connections 2024-04-25T11:16:11.682 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:11.133+0000 7f780359e700 1 -- 172.21.15.69:0/3913091096 wait complete. 2024-04-25T11:16:11.682 INFO:teuthology.orchestra.run.smithi069.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-25T11:16:11.682 INFO:teuthology.orchestra.run.smithi069.stdout:Adding key to root@localhost authorized_keys... 2024-04-25T11:16:11.682 INFO:teuthology.orchestra.run.smithi069.stdout:Adding host smithi069... 2024-04-25T11:16:12.301 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:12 smithi069 ceph-mon[29188]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:16:13.152 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:13 smithi069 ceph-mon[29188]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi069", "addr": "172.21.15.69", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:16:13.152 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:13 smithi069 ceph-mon[29188]: Deploying cephadm binary to smithi069 2024-04-25T11:16:17.009 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout Added host 'smithi069' with addr '172.21.15.69' 2024-04-25T11:16:17.010 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:12.087+0000 7f1432240700 1 Processor -- start 2024-04-25T11:16:17.010 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:12.087+0000 7f1432240700 1 -- start start 2024-04-25T11:16:17.010 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:12.087+0000 7f1432240700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f142c07ccf0 0x7f142c10c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:17.010 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:12.088+0000 7f1432240700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f142c10ca20 con 0x7f142c07ccf0 2024-04-25T11:16:17.010 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:12.088+0000 7f142b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f142c07ccf0 0x7f142c10c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:17.010 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:12.088+0000 7f142b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f142c07ccf0 0x7f142c10c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35464/0 (socket says 172.21.15.69:35464) 2024-04-25T11:16:17.010 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:12.088+0000 7f142b7fe700 1 -- 172.21.15.69:0/2222143913 learned_addr learned my addr 172.21.15.69:0/2222143913 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:16:17.010 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:12.088+0000 7f142b7fe700 1 -- 172.21.15.69:0/2222143913 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f142c10cba0 con 0x7f142c07ccf0 2024-04-25T11:16:17.011 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:12.088+0000 7f142b7fe700 1 --2- 172.21.15.69:0/2222143913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f142c07ccf0 0x7f142c10c410 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f141c0098a0 tx=0x7f141c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520d9d2caf5f542 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:17.011 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:12.088+0000 7f142affd700 1 -- 172.21.15.69:0/2222143913 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f141c004160 con 0x7f142c07ccf0 2024-04-25T11:16:17.011 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:12.088+0000 7f142affd700 1 -- 172.21.15.69:0/2222143913 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f141c004300 con 0x7f142c07ccf0 2024-04-25T11:16:17.011 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:12.088+0000 7f142affd700 1 -- 172.21.15.69:0/2222143913 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f141c0119e0 con 0x7f142c07ccf0 2024-04-25T11:16:17.011 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:12.089+0000 7f1432240700 1 -- 172.21.15.69:0/2222143913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f142c07ccf0 msgr2=0x7f142c10c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:17.011 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:12.089+0000 7f1432240700 1 --2- 172.21.15.69:0/2222143913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f142c07ccf0 0x7f142c10c410 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f141c0098a0 tx=0x7f141c009bb0 comp rx=0 tx=0).stop 2024-04-25T11:16:17.011 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:12.089+0000 7f1432240700 1 -- 172.21.15.69:0/2222143913 shutdown_connections 2024-04-25T11:16:17.011 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:12.089+0000 7f1432240700 1 --2- 172.21.15.69:0/2222143913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f142c07ccf0 0x7f142c10c410 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:17.011 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:12.089+0000 7f1432240700 1 -- 172.21.15.69:0/2222143913 >> 172.21.15.69:0/2222143913 conn(0x7f142c103ee0 msgr2=0x7f142c106330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:17.011 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:12.089+0000 7f1432240700 1 -- 172.21.15.69:0/2222143913 shutdown_connections 2024-04-25T11:16:17.011 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:12.089+0000 7f1432240700 1 -- 172.21.15.69:0/2222143913 wait complete. 2024-04-25T11:16:17.011 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:12.090+0000 7f1432240700 1 Processor -- start 2024-04-25T11:16:17.011 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:12.090+0000 7f1432240700 1 -- start start 2024-04-25T11:16:17.011 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:12.090+0000 7f1432240700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f142c07ccf0 0x7f142c19daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:17.011 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:12.090+0000 7f1432240700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f142c10db50 con 0x7f142c07ccf0 2024-04-25T11:16:17.012 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:12.090+0000 7f142b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f142c07ccf0 0x7f142c19daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:17.012 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:12.090+0000 7f142b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f142c07ccf0 0x7f142c19daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35480/0 (socket says 172.21.15.69:35480) 2024-04-25T11:16:17.012 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:12.090+0000 7f142b7fe700 1 -- 172.21.15.69:0/1744755203 learned_addr learned my addr 172.21.15.69:0/1744755203 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:16:17.012 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:12.091+0000 7f142b7fe700 1 -- 172.21.15.69:0/1744755203 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f142c19e030 con 0x7f142c07ccf0 2024-04-25T11:16:17.012 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:12.091+0000 7f142b7fe700 1 --2- 172.21.15.69:0/1744755203 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f142c07ccf0 0x7f142c19daf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f141c0116a0 tx=0x7f141c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:17.012 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:12.091+0000 7f14297fa700 1 -- 172.21.15.69:0/1744755203 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f141c0039f0 con 0x7f142c07ccf0 2024-04-25T11:16:17.012 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:12.091+0000 7f1432240700 1 -- 172.21.15.69:0/1744755203 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f142c19e2c0 con 0x7f142c07ccf0 2024-04-25T11:16:17.012 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:12.091+0000 7f14297fa700 1 -- 172.21.15.69:0/1744755203 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f141c003b90 con 0x7f142c07ccf0 2024-04-25T11:16:17.012 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:12.091+0000 7f14297fa700 1 -- 172.21.15.69:0/1744755203 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f141c01a600 con 0x7f142c07ccf0 2024-04-25T11:16:17.012 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:12.091+0000 7f1432240700 1 -- 172.21.15.69:0/1744755203 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f142c1a0ed0 con 0x7f142c07ccf0 2024-04-25T11:16:17.012 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:12.092+0000 7f1432240700 1 -- 172.21.15.69:0/1744755203 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f142c064cf0 con 0x7f142c07ccf0 2024-04-25T11:16:17.012 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:12.092+0000 7f14297fa700 1 -- 172.21.15.69:0/1744755203 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f141c01a900 con 0x7f142c07ccf0 2024-04-25T11:16:17.012 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:12.092+0000 7f14297fa700 1 --2- 172.21.15.69:0/1744755203 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f141803d5e0 0x7f141803faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:17.012 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:12.092+0000 7f14297fa700 1 -- 172.21.15.69:0/1744755203 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f141c052a60 con 0x7f142c07ccf0 2024-04-25T11:16:17.013 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:12.092+0000 7f1422ffd700 1 --2- 172.21.15.69:0/1744755203 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f141803d5e0 0x7f141803faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:17.013 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:12.093+0000 7f1422ffd700 1 --2- 172.21.15.69:0/1744755203 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f141803d5e0 0x7f141803faf0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1414006fd0 tx=0x7f1414006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:17.013 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:12.097+0000 7f14297fa700 1 -- 172.21.15.69:0/1744755203 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f141c02c350 con 0x7f142c07ccf0 2024-04-25T11:16:17.013 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:12.300+0000 7f1432240700 1 -- 172.21.15.69:0/1744755203 --> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi069", "addr": "172.21.15.69", "target": ["mon-mgr", ""]}) -- 0x7f142c105f30 con 0x7f141803d5e0 2024-04-25T11:16:17.013 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:16.381+0000 7f14297fa700 1 -- 172.21.15.69:0/1744755203 <== mgr.14120 v2:172.21.15.69:6800/1097124842 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f142c105f30 con 0x7f141803d5e0 2024-04-25T11:16:17.013 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:16.384+0000 7f1432240700 1 -- 172.21.15.69:0/1744755203 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f141803d5e0 msgr2=0x7f141803faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:17.013 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:16.384+0000 7f1432240700 1 --2- 172.21.15.69:0/1744755203 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f141803d5e0 0x7f141803faf0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1414006fd0 tx=0x7f1414006e40 comp rx=0 tx=0).stop 2024-04-25T11:16:17.013 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:16.384+0000 7f1432240700 1 -- 172.21.15.69:0/1744755203 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f142c07ccf0 msgr2=0x7f142c19daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:17.013 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:16.384+0000 7f1432240700 1 --2- 172.21.15.69:0/1744755203 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f142c07ccf0 0x7f142c19daf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f141c0116a0 tx=0x7f141c011a60 comp rx=0 tx=0).stop 2024-04-25T11:16:17.013 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:16.384+0000 7f1432240700 1 -- 172.21.15.69:0/1744755203 shutdown_connections 2024-04-25T11:16:17.013 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:16.384+0000 7f1432240700 1 --2- 172.21.15.69:0/1744755203 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f141803d5e0 0x7f141803faf0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:17.013 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:16.384+0000 7f1432240700 1 --2- 172.21.15.69:0/1744755203 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f142c07ccf0 0x7f142c19daf0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:17.013 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:16.384+0000 7f1432240700 1 -- 172.21.15.69:0/1744755203 >> 172.21.15.69:0/1744755203 conn(0x7f142c103ee0 msgr2=0x7f142c105820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:17.013 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:16.384+0000 7f1432240700 1 -- 172.21.15.69:0/1744755203 shutdown_connections 2024-04-25T11:16:17.013 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:16.384+0000 7f1432240700 1 -- 172.21.15.69:0/1744755203 wait complete. 2024-04-25T11:16:17.014 INFO:teuthology.orchestra.run.smithi069.stdout:Deploying unmanaged mon service... 2024-04-25T11:16:17.616 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:17 smithi069 ceph-mon[29188]: from='mgr.14120 172.21.15.69:0/157690041' entity='mgr.x' 2024-04-25T11:16:17.616 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:17 smithi069 ceph-mon[29188]: Added host smithi069 2024-04-25T11:16:17.616 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:17 smithi069 ceph-mon[29188]: from='mgr.14120 172.21.15.69:0/157690041' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:16:18.949 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:18 smithi069 ceph-mon[29188]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:16:18.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:18 smithi069 ceph-mon[29188]: Saving service mon spec with placement count:5 2024-04-25T11:16:18.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:18 smithi069 ceph-mon[29188]: from='mgr.14120 172.21.15.69:0/157690041' entity='mgr.x' 2024-04-25T11:16:19.216 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-25T11:16:19.216 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.475+0000 7f5ecab86700 1 Processor -- start 2024-04-25T11:16:19.216 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.475+0000 7f5ecab86700 1 -- start start 2024-04-25T11:16:19.216 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.475+0000 7f5ecab86700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ec407d6e0 0x7f5ec410e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:19.216 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.475+0000 7f5ecab86700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec410edf0 con 0x7f5ec407d6e0 2024-04-25T11:16:19.216 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.475+0000 7f5ec8922700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ec407d6e0 0x7f5ec410e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:19.216 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.475+0000 7f5ec8922700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ec407d6e0 0x7f5ec410e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42566/0 (socket says 172.21.15.69:42566) 2024-04-25T11:16:19.217 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.475+0000 7f5ec8922700 1 -- 172.21.15.69:0/2386630683 learned_addr learned my addr 172.21.15.69:0/2386630683 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:16:19.217 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.476+0000 7f5ec8922700 1 -- 172.21.15.69:0/2386630683 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ec410ef70 con 0x7f5ec407d6e0 2024-04-25T11:16:19.217 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.476+0000 7f5ec8922700 1 --2- 172.21.15.69:0/2386630683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ec407d6e0 0x7f5ec410e8b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5eb80098a0 tx=0x7f5eb8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d25257afeeafd11a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:19.217 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.476+0000 7f5ec37fe700 1 -- 172.21.15.69:0/2386630683 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5eb8004160 con 0x7f5ec407d6e0 2024-04-25T11:16:19.217 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.476+0000 7f5ec37fe700 1 -- 172.21.15.69:0/2386630683 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f5eb8004300 con 0x7f5ec407d6e0 2024-04-25T11:16:19.217 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.476+0000 7f5ec37fe700 1 -- 172.21.15.69:0/2386630683 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5eb80119e0 con 0x7f5ec407d6e0 2024-04-25T11:16:19.217 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.476+0000 7f5ecab86700 1 -- 172.21.15.69:0/2386630683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ec407d6e0 msgr2=0x7f5ec410e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:19.217 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.476+0000 7f5ecab86700 1 --2- 172.21.15.69:0/2386630683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ec407d6e0 0x7f5ec410e8b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5eb80098a0 tx=0x7f5eb8009bb0 comp rx=0 tx=0).stop 2024-04-25T11:16:19.217 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.477+0000 7f5ecab86700 1 -- 172.21.15.69:0/2386630683 shutdown_connections 2024-04-25T11:16:19.217 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.477+0000 7f5ecab86700 1 --2- 172.21.15.69:0/2386630683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ec407d6e0 0x7f5ec410e8b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:19.217 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.477+0000 7f5ecab86700 1 -- 172.21.15.69:0/2386630683 >> 172.21.15.69:0/2386630683 conn(0x7f5ec4103ea0 msgr2=0x7f5ec4106330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:19.217 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.477+0000 7f5ecab86700 1 -- 172.21.15.69:0/2386630683 shutdown_connections 2024-04-25T11:16:19.217 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.477+0000 7f5ecab86700 1 -- 172.21.15.69:0/2386630683 wait complete. 2024-04-25T11:16:19.218 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.477+0000 7f5ecab86700 1 Processor -- start 2024-04-25T11:16:19.218 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.477+0000 7f5ecab86700 1 -- start start 2024-04-25T11:16:19.218 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.477+0000 7f5ecab86700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ec407d6e0 0x7f5ec419fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:19.218 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.477+0000 7f5ecab86700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec410fdb0 con 0x7f5ec407d6e0 2024-04-25T11:16:19.218 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.478+0000 7f5ec8922700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ec407d6e0 0x7f5ec419fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:19.218 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.478+0000 7f5ec8922700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ec407d6e0 0x7f5ec419fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42578/0 (socket says 172.21.15.69:42578) 2024-04-25T11:16:19.218 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.478+0000 7f5ec8922700 1 -- 172.21.15.69:0/122418890 learned_addr learned my addr 172.21.15.69:0/122418890 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:16:19.218 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.478+0000 7f5ec8922700 1 -- 172.21.15.69:0/122418890 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ec41a0220 con 0x7f5ec407d6e0 2024-04-25T11:16:19.218 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.478+0000 7f5ec8922700 1 --2- 172.21.15.69:0/122418890 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ec407d6e0 0x7f5ec419fce0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8003f20 tx=0x7f5eb8011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:19.218 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.478+0000 7f5ec1ffb700 1 -- 172.21.15.69:0/122418890 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5eb80038b0 con 0x7f5ec407d6e0 2024-04-25T11:16:19.218 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.478+0000 7f5ecab86700 1 -- 172.21.15.69:0/122418890 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ec41a04b0 con 0x7f5ec407d6e0 2024-04-25T11:16:19.218 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.478+0000 7f5ec1ffb700 1 -- 172.21.15.69:0/122418890 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f5eb8003a50 con 0x7f5ec407d6e0 2024-04-25T11:16:19.218 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.478+0000 7f5ec1ffb700 1 -- 172.21.15.69:0/122418890 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5eb80195a0 con 0x7f5ec407d6e0 2024-04-25T11:16:19.219 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.478+0000 7f5ecab86700 1 -- 172.21.15.69:0/122418890 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ec41a30c0 con 0x7f5ec407d6e0 2024-04-25T11:16:19.219 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.479+0000 7f5ec1ffb700 1 -- 172.21.15.69:0/122418890 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f5eb8019860 con 0x7f5ec407d6e0 2024-04-25T11:16:19.219 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.479+0000 7f5ecab86700 1 -- 172.21.15.69:0/122418890 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ec4064cf0 con 0x7f5ec407d6e0 2024-04-25T11:16:19.219 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.480+0000 7f5ec1ffb700 1 --2- 172.21.15.69:0/122418890 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f5eb403d5e0 0x7f5eb403faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:19.219 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.480+0000 7f5ec1ffb700 1 -- 172.21.15.69:0/122418890 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5eb8052b50 con 0x7f5ec407d6e0 2024-04-25T11:16:19.219 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.480+0000 7f5ec3fff700 1 --2- 172.21.15.69:0/122418890 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f5eb403d5e0 0x7f5eb403faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:19.219 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.480+0000 7f5ec3fff700 1 --2- 172.21.15.69:0/122418890 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f5eb403d5e0 0x7f5eb403faf0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0006fd0 tx=0x7f5eb0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:19.219 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.485+0000 7f5ec1ffb700 1 -- 172.21.15.69:0/122418890 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5eb8018dc0 con 0x7f5ec407d6e0 2024-04-25T11:16:19.219 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.702+0000 7f5ecab86700 1 -- 172.21.15.69:0/122418890 --> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f5ec4104e20 con 0x7f5eb403d5e0 2024-04-25T11:16:19.219 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.709+0000 7f5ec1ffb700 1 -- 172.21.15.69:0/122418890 <== mgr.14120 v2:172.21.15.69:6800/1097124842 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f5ec4104e20 con 0x7f5eb403d5e0 2024-04-25T11:16:19.219 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.712+0000 7f5ecab86700 1 -- 172.21.15.69:0/122418890 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f5eb403d5e0 msgr2=0x7f5eb403faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:19.219 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.712+0000 7f5ecab86700 1 --2- 172.21.15.69:0/122418890 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f5eb403d5e0 0x7f5eb403faf0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0006fd0 tx=0x7f5eb0006e40 comp rx=0 tx=0).stop 2024-04-25T11:16:19.219 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.712+0000 7f5ecab86700 1 -- 172.21.15.69:0/122418890 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ec407d6e0 msgr2=0x7f5ec419fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:19.220 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.712+0000 7f5ecab86700 1 --2- 172.21.15.69:0/122418890 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ec407d6e0 0x7f5ec419fce0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8003f20 tx=0x7f5eb8011a60 comp rx=0 tx=0).stop 2024-04-25T11:16:19.220 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.712+0000 7f5ecab86700 1 -- 172.21.15.69:0/122418890 shutdown_connections 2024-04-25T11:16:19.220 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.712+0000 7f5ecab86700 1 --2- 172.21.15.69:0/122418890 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f5eb403d5e0 0x7f5eb403faf0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:19.220 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.712+0000 7f5ecab86700 1 --2- 172.21.15.69:0/122418890 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ec407d6e0 0x7f5ec419fce0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:19.220 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.712+0000 7f5ecab86700 1 -- 172.21.15.69:0/122418890 >> 172.21.15.69:0/122418890 conn(0x7f5ec4103ea0 msgr2=0x7f5ec4104710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:19.220 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.712+0000 7f5ecab86700 1 -- 172.21.15.69:0/122418890 shutdown_connections 2024-04-25T11:16:19.220 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:17.712+0000 7f5ecab86700 1 -- 172.21.15.69:0/122418890 wait complete. 2024-04-25T11:16:19.220 INFO:teuthology.orchestra.run.smithi069.stdout:Deploying unmanaged mgr service... 2024-04-25T11:16:19.908 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:19 smithi069 ceph-mon[29188]: from='mgr.14120 172.21.15.69:0/157690041' entity='mgr.x' 2024-04-25T11:16:19.908 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:19 smithi069 ceph-mon[29188]: from='mgr.14120 172.21.15.69:0/157690041' entity='mgr.x' 2024-04-25T11:16:20.505 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-25T11:16:20.505 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.667+0000 7f365f0e6700 1 Processor -- start 2024-04-25T11:16:20.505 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.667+0000 7f365f0e6700 1 -- start start 2024-04-25T11:16:20.505 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.667+0000 7f365f0e6700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f365807d720 0x7f365810c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:20.505 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.667+0000 7f365f0e6700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f365810ca40 con 0x7f365807d720 2024-04-25T11:16:20.506 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.667+0000 7f365ce82700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f365807d720 0x7f365810c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:20.506 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.667+0000 7f365ce82700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f365807d720 0x7f365810c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42580/0 (socket says 172.21.15.69:42580) 2024-04-25T11:16:20.506 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.667+0000 7f365ce82700 1 -- 172.21.15.69:0/1424490622 learned_addr learned my addr 172.21.15.69:0/1424490622 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:16:20.506 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.668+0000 7f365ce82700 1 -- 172.21.15.69:0/1424490622 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f365810cbc0 con 0x7f365807d720 2024-04-25T11:16:20.506 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.668+0000 7f365ce82700 1 --2- 172.21.15.69:0/1424490622 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f365807d720 0x7f365810c430 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f364c0098a0 tx=0x7f364c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cabe77e5126c37a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:20.506 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.668+0000 7f36577fe700 1 -- 172.21.15.69:0/1424490622 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f364c004160 con 0x7f365807d720 2024-04-25T11:16:20.506 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.668+0000 7f36577fe700 1 -- 172.21.15.69:0/1424490622 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f364c004300 con 0x7f365807d720 2024-04-25T11:16:20.506 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.668+0000 7f36577fe700 1 -- 172.21.15.69:0/1424490622 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f364c011960 con 0x7f365807d720 2024-04-25T11:16:20.506 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.669+0000 7f365f0e6700 1 -- 172.21.15.69:0/1424490622 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f365807d720 msgr2=0x7f365810c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:20.506 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.669+0000 7f365f0e6700 1 --2- 172.21.15.69:0/1424490622 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f365807d720 0x7f365810c430 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f364c0098a0 tx=0x7f364c009bb0 comp rx=0 tx=0).stop 2024-04-25T11:16:20.506 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.669+0000 7f365f0e6700 1 -- 172.21.15.69:0/1424490622 shutdown_connections 2024-04-25T11:16:20.506 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.669+0000 7f365f0e6700 1 --2- 172.21.15.69:0/1424490622 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f365807d720 0x7f365810c430 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:20.507 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.669+0000 7f365f0e6700 1 -- 172.21.15.69:0/1424490622 >> 172.21.15.69:0/1424490622 conn(0x7f3658103ea0 msgr2=0x7f36581062f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:20.507 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.669+0000 7f365f0e6700 1 -- 172.21.15.69:0/1424490622 shutdown_connections 2024-04-25T11:16:20.507 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.669+0000 7f365f0e6700 1 -- 172.21.15.69:0/1424490622 wait complete. 2024-04-25T11:16:20.507 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.670+0000 7f365f0e6700 1 Processor -- start 2024-04-25T11:16:20.507 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.670+0000 7f365f0e6700 1 -- start start 2024-04-25T11:16:20.507 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.670+0000 7f365f0e6700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f365807d720 0x7f365819da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:20.507 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.670+0000 7f365f0e6700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f365810dc20 con 0x7f365807d720 2024-04-25T11:16:20.507 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.670+0000 7f365ce82700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f365807d720 0x7f365819da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:20.507 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.670+0000 7f365ce82700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f365807d720 0x7f365819da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42586/0 (socket says 172.21.15.69:42586) 2024-04-25T11:16:20.507 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.670+0000 7f365ce82700 1 -- 172.21.15.69:0/1860754552 learned_addr learned my addr 172.21.15.69:0/1860754552 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:16:20.507 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.670+0000 7f365ce82700 1 -- 172.21.15.69:0/1860754552 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f365819dfc0 con 0x7f365807d720 2024-04-25T11:16:20.507 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.671+0000 7f365ce82700 1 --2- 172.21.15.69:0/1860754552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f365807d720 0x7f365819da80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f364c011670 tx=0x7f364c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:20.507 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.671+0000 7f3655ffb700 1 -- 172.21.15.69:0/1860754552 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f364c003aa0 con 0x7f365807d720 2024-04-25T11:16:20.507 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.671+0000 7f3655ffb700 1 -- 172.21.15.69:0/1860754552 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f364c01b8d0 con 0x7f365807d720 2024-04-25T11:16:20.508 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.671+0000 7f365f0e6700 1 -- 172.21.15.69:0/1860754552 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f365819e250 con 0x7f365807d720 2024-04-25T11:16:20.508 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.671+0000 7f3655ffb700 1 -- 172.21.15.69:0/1860754552 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f364c0197f0 con 0x7f365807d720 2024-04-25T11:16:20.508 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.671+0000 7f365f0e6700 1 -- 172.21.15.69:0/1860754552 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36581a0e60 con 0x7f365807d720 2024-04-25T11:16:20.508 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.672+0000 7f3655ffb700 1 -- 172.21.15.69:0/1860754552 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f364c019990 con 0x7f365807d720 2024-04-25T11:16:20.508 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.672+0000 7f365f0e6700 1 -- 172.21.15.69:0/1860754552 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3658064cf0 con 0x7f365807d720 2024-04-25T11:16:20.508 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.672+0000 7f3655ffb700 1 --2- 172.21.15.69:0/1860754552 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f364803d630 0x7f364803fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:20.508 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.673+0000 7f3655ffb700 1 -- 172.21.15.69:0/1860754552 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f364c052d50 con 0x7f365807d720 2024-04-25T11:16:20.508 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.673+0000 7f3657fff700 1 --2- 172.21.15.69:0/1860754552 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f364803d630 0x7f364803fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:20.508 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.673+0000 7f3657fff700 1 --2- 172.21.15.69:0/1860754552 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f364803d630 0x7f364803fb40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3644006fd0 tx=0x7f3644006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:20.508 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.678+0000 7f3655ffb700 1 -- 172.21.15.69:0/1860754552 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f364c01f030 con 0x7f365807d720 2024-04-25T11:16:20.508 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.900+0000 7f365f0e6700 1 -- 172.21.15.69:0/1860754552 --> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f3658104d70 con 0x7f364803d630 2024-04-25T11:16:20.508 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.907+0000 7f3655ffb700 1 -- 172.21.15.69:0/1860754552 <== mgr.14120 v2:172.21.15.69:6800/1097124842 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f3658104d70 con 0x7f364803d630 2024-04-25T11:16:20.508 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.909+0000 7f365f0e6700 1 -- 172.21.15.69:0/1860754552 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f364803d630 msgr2=0x7f364803fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:20.508 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.909+0000 7f365f0e6700 1 --2- 172.21.15.69:0/1860754552 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f364803d630 0x7f364803fb40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3644006fd0 tx=0x7f3644006e40 comp rx=0 tx=0).stop 2024-04-25T11:16:20.509 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.909+0000 7f365f0e6700 1 -- 172.21.15.69:0/1860754552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f365807d720 msgr2=0x7f365819da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:20.509 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.909+0000 7f365f0e6700 1 --2- 172.21.15.69:0/1860754552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f365807d720 0x7f365819da80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f364c011670 tx=0x7f364c011a30 comp rx=0 tx=0).stop 2024-04-25T11:16:20.509 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.909+0000 7f365f0e6700 1 -- 172.21.15.69:0/1860754552 shutdown_connections 2024-04-25T11:16:20.509 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.909+0000 7f365f0e6700 1 --2- 172.21.15.69:0/1860754552 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f364803d630 0x7f364803fb40 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:20.509 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.909+0000 7f365f0e6700 1 --2- 172.21.15.69:0/1860754552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f365807d720 0x7f365819da80 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:20.509 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.909+0000 7f365f0e6700 1 -- 172.21.15.69:0/1860754552 >> 172.21.15.69:0/1860754552 conn(0x7f3658103ea0 msgr2=0x7f3658104660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:20.509 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.910+0000 7f365f0e6700 1 -- 172.21.15.69:0/1860754552 shutdown_connections 2024-04-25T11:16:20.509 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:19.910+0000 7f365f0e6700 1 -- 172.21.15.69:0/1860754552 wait complete. 2024-04-25T11:16:21.164 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:20 smithi069 ceph-mon[29188]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:16:21.164 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:20 smithi069 ceph-mon[29188]: Saving service mgr spec with placement count:2 2024-04-25T11:16:21.165 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:20 smithi069 ceph-mon[29188]: from='mgr.14120 172.21.15.69:0/157690041' entity='mgr.x' 2024-04-25T11:16:21.954 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:20.930+0000 7ff068072700 1 Processor -- start 2024-04-25T11:16:21.954 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:20.930+0000 7ff068072700 1 -- start start 2024-04-25T11:16:21.954 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:20.930+0000 7ff068072700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff060108820 0x7ff060108c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:21.955 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:20.930+0000 7ff068072700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff060109220 con 0x7ff060108820 2024-04-25T11:16:21.955 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:20.930+0000 7ff065e0e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff060108820 0x7ff060108c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:21.955 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:20.930+0000 7ff065e0e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff060108820 0x7ff060108c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42596/0 (socket says 172.21.15.69:42596) 2024-04-25T11:16:21.955 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:20.930+0000 7ff065e0e700 1 -- 172.21.15.69:0/2205448660 learned_addr learned my addr 172.21.15.69:0/2205448660 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:16:21.955 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:20.931+0000 7ff065e0e700 1 -- 172.21.15.69:0/2205448660 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff060109950 con 0x7ff060108820 2024-04-25T11:16:21.955 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:20.931+0000 7ff065e0e700 1 --2- 172.21.15.69:0/2205448660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff060108820 0x7ff060108c30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff05c0098a0 tx=0x7ff05c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28cd436a54d567ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:21.955 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:20.931+0000 7ff064e0c700 1 -- 172.21.15.69:0/2205448660 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff05c004160 con 0x7ff060108820 2024-04-25T11:16:21.955 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:20.931+0000 7ff064e0c700 1 -- 172.21.15.69:0/2205448660 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff05c004300 con 0x7ff060108820 2024-04-25T11:16:21.955 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:20.931+0000 7ff064e0c700 1 -- 172.21.15.69:0/2205448660 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff05c011960 con 0x7ff060108820 2024-04-25T11:16:21.955 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:20.932+0000 7ff068072700 1 -- 172.21.15.69:0/2205448660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff060108820 msgr2=0x7ff060108c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:21.955 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:20.932+0000 7ff068072700 1 --2- 172.21.15.69:0/2205448660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff060108820 0x7ff060108c30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff05c0098a0 tx=0x7ff05c009bb0 comp rx=0 tx=0).stop 2024-04-25T11:16:21.955 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:20.932+0000 7ff068072700 1 -- 172.21.15.69:0/2205448660 shutdown_connections 2024-04-25T11:16:21.955 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:20.932+0000 7ff068072700 1 --2- 172.21.15.69:0/2205448660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff060108820 0x7ff060108c30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:21.956 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:20.932+0000 7ff068072700 1 -- 172.21.15.69:0/2205448660 >> 172.21.15.69:0/2205448660 conn(0x7ff060103e60 msgr2=0x7ff060106310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:21.956 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:20.932+0000 7ff068072700 1 -- 172.21.15.69:0/2205448660 shutdown_connections 2024-04-25T11:16:21.956 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:20.932+0000 7ff068072700 1 -- 172.21.15.69:0/2205448660 wait complete. 2024-04-25T11:16:21.956 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:20.932+0000 7ff068072700 1 Processor -- start 2024-04-25T11:16:21.956 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:20.933+0000 7ff068072700 1 -- start start 2024-04-25T11:16:21.956 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:20.933+0000 7ff068072700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff060108820 0x7ff06007d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:21.956 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:20.933+0000 7ff068072700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff06010cd00 con 0x7ff060108820 2024-04-25T11:16:21.956 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:20.933+0000 7ff065e0e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff060108820 0x7ff06007d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:21.956 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:20.933+0000 7ff065e0e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff060108820 0x7ff06007d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42606/0 (socket says 172.21.15.69:42606) 2024-04-25T11:16:21.956 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:20.933+0000 7ff065e0e700 1 -- 172.21.15.69:0/2746764126 learned_addr learned my addr 172.21.15.69:0/2746764126 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:16:21.956 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:20.933+0000 7ff065e0e700 1 -- 172.21.15.69:0/2746764126 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff06007d8e0 con 0x7ff060108820 2024-04-25T11:16:21.956 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:20.933+0000 7ff065e0e700 1 --2- 172.21.15.69:0/2746764126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff060108820 0x7ff06007d3a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff05c0119b0 tx=0x7ff05c011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:21.956 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:20.933+0000 7ff056ffd700 1 -- 172.21.15.69:0/2746764126 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff05c003aa0 con 0x7ff060108820 2024-04-25T11:16:21.956 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:20.933+0000 7ff056ffd700 1 -- 172.21.15.69:0/2746764126 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff05c01b8d0 con 0x7ff060108820 2024-04-25T11:16:21.957 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:20.933+0000 7ff056ffd700 1 -- 172.21.15.69:0/2746764126 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff05c0237d0 con 0x7ff060108820 2024-04-25T11:16:21.957 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:20.933+0000 7ff068072700 1 -- 172.21.15.69:0/2746764126 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0601a08e0 con 0x7ff060108820 2024-04-25T11:16:21.957 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:20.933+0000 7ff068072700 1 -- 172.21.15.69:0/2746764126 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0601a0d60 con 0x7ff060108820 2024-04-25T11:16:21.957 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:20.934+0000 7ff068072700 1 -- 172.21.15.69:0/2746764126 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff060064cf0 con 0x7ff060108820 2024-04-25T11:16:21.957 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:20.934+0000 7ff056ffd700 1 -- 172.21.15.69:0/2746764126 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7ff05c02c3f0 con 0x7ff060108820 2024-04-25T11:16:21.957 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:20.935+0000 7ff056ffd700 1 --2- 172.21.15.69:0/2746764126 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7ff04c03d630 0x7ff04c03fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:21.957 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:20.935+0000 7ff056ffd700 1 -- 172.21.15.69:0/2746764126 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff05c052c50 con 0x7ff060108820 2024-04-25T11:16:21.957 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:20.935+0000 7ff06560d700 1 --2- 172.21.15.69:0/2746764126 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7ff04c03d630 0x7ff04c03fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:21.957 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:20.935+0000 7ff06560d700 1 --2- 172.21.15.69:0/2746764126 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7ff04c03d630 0x7ff04c03fb40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff050006fd0 tx=0x7ff050006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:21.957 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:20.940+0000 7ff056ffd700 1 -- 172.21.15.69:0/2746764126 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff05c01f030 con 0x7ff060108820 2024-04-25T11:16:21.957 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:21.164+0000 7ff068072700 1 -- 172.21.15.69:0/2746764126 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7ff0601a0fe0 con 0x7ff060108820 2024-04-25T11:16:21.957 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:21.169+0000 7ff056ffd700 1 -- 172.21.15.69:0/2746764126 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7ff05c00fdd0 con 0x7ff060108820 2024-04-25T11:16:21.957 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:21.171+0000 7ff068072700 1 -- 172.21.15.69:0/2746764126 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7ff04c03d630 msgr2=0x7ff04c03fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:21.958 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:21.171+0000 7ff068072700 1 --2- 172.21.15.69:0/2746764126 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7ff04c03d630 0x7ff04c03fb40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff050006fd0 tx=0x7ff050006e40 comp rx=0 tx=0).stop 2024-04-25T11:16:21.958 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:21.171+0000 7ff068072700 1 -- 172.21.15.69:0/2746764126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff060108820 msgr2=0x7ff06007d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:21.958 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:21.171+0000 7ff068072700 1 --2- 172.21.15.69:0/2746764126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff060108820 0x7ff06007d3a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff05c0119b0 tx=0x7ff05c011a90 comp rx=0 tx=0).stop 2024-04-25T11:16:21.958 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:21.171+0000 7ff068072700 1 -- 172.21.15.69:0/2746764126 shutdown_connections 2024-04-25T11:16:21.958 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:21.171+0000 7ff068072700 1 --2- 172.21.15.69:0/2746764126 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7ff04c03d630 0x7ff04c03fb40 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:21.958 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:21.171+0000 7ff068072700 1 --2- 172.21.15.69:0/2746764126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff060108820 0x7ff06007d3a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:21.958 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:21.171+0000 7ff068072700 1 -- 172.21.15.69:0/2746764126 >> 172.21.15.69:0/2746764126 conn(0x7ff060103e60 msgr2=0x7ff060104690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:21.958 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:21.171+0000 7ff068072700 1 -- 172.21.15.69:0/2746764126 shutdown_connections 2024-04-25T11:16:21.958 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:21.171+0000 7ff068072700 1 -- 172.21.15.69:0/2746764126 wait complete. 2024-04-25T11:16:22.402 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:22 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2746764126' entity='client.admin' 2024-04-25T11:16:23.404 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.603+0000 7f2436652700 1 Processor -- start 2024-04-25T11:16:23.404 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.603+0000 7f2436652700 1 -- start start 2024-04-25T11:16:23.404 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.604+0000 7f2436652700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f243007d560 0x7f243010c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:23.404 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.604+0000 7f2436652700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f243010ca70 con 0x7f243007d560 2024-04-25T11:16:23.404 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.604+0000 7f242ffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f243007d560 0x7f243010c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:23.404 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.604+0000 7f242ffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f243007d560 0x7f243010c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33268/0 (socket says 172.21.15.69:33268) 2024-04-25T11:16:23.405 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.604+0000 7f242ffff700 1 -- 172.21.15.69:0/512980203 learned_addr learned my addr 172.21.15.69:0/512980203 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:16:23.405 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.604+0000 7f242ffff700 1 -- 172.21.15.69:0/512980203 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f243010cbf0 con 0x7f243007d560 2024-04-25T11:16:23.405 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.604+0000 7f242ffff700 1 --2- 172.21.15.69:0/512980203 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f243007d560 0x7f243010c480 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f24200098a0 tx=0x7f2420009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eade9675cb596184 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:23.405 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.605+0000 7f242effd700 1 -- 172.21.15.69:0/512980203 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2420004160 con 0x7f243007d560 2024-04-25T11:16:23.405 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.605+0000 7f242effd700 1 -- 172.21.15.69:0/512980203 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2420004300 con 0x7f243007d560 2024-04-25T11:16:23.405 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.605+0000 7f242effd700 1 -- 172.21.15.69:0/512980203 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2420011960 con 0x7f243007d560 2024-04-25T11:16:23.405 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.605+0000 7f2436652700 1 -- 172.21.15.69:0/512980203 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f243007d560 msgr2=0x7f243010c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:23.405 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.605+0000 7f2436652700 1 --2- 172.21.15.69:0/512980203 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f243007d560 0x7f243010c480 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f24200098a0 tx=0x7f2420009bb0 comp rx=0 tx=0).stop 2024-04-25T11:16:23.405 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.605+0000 7f2436652700 1 -- 172.21.15.69:0/512980203 shutdown_connections 2024-04-25T11:16:23.405 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.605+0000 7f2436652700 1 --2- 172.21.15.69:0/512980203 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f243007d560 0x7f243010c480 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:23.405 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.605+0000 7f2436652700 1 -- 172.21.15.69:0/512980203 >> 172.21.15.69:0/512980203 conn(0x7f2430103eb0 msgr2=0x7f2430106320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:23.405 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.605+0000 7f2436652700 1 -- 172.21.15.69:0/512980203 shutdown_connections 2024-04-25T11:16:23.405 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.605+0000 7f2436652700 1 -- 172.21.15.69:0/512980203 wait complete. 2024-04-25T11:16:23.406 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.606+0000 7f2436652700 1 Processor -- start 2024-04-25T11:16:23.406 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.606+0000 7f2436652700 1 -- start start 2024-04-25T11:16:23.406 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.606+0000 7f2436652700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f243007d560 0x7f243019fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:23.406 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.606+0000 7f2436652700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f243010dc50 con 0x7f243007d560 2024-04-25T11:16:23.406 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.606+0000 7f242ffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f243007d560 0x7f243019fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:23.406 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.607+0000 7f242ffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f243007d560 0x7f243019fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33272/0 (socket says 172.21.15.69:33272) 2024-04-25T11:16:23.406 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.607+0000 7f242ffff700 1 -- 172.21.15.69:0/120855896 learned_addr learned my addr 172.21.15.69:0/120855896 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:16:23.406 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.607+0000 7f242ffff700 1 -- 172.21.15.69:0/120855896 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24301a0270 con 0x7f243007d560 2024-04-25T11:16:23.406 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.607+0000 7f242ffff700 1 --2- 172.21.15.69:0/120855896 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f243007d560 0x7f243019fd30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2420011670 tx=0x7f2420011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:23.406 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.607+0000 7f242d7fa700 1 -- 172.21.15.69:0/120855896 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2420003a60 con 0x7f243007d560 2024-04-25T11:16:23.406 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.607+0000 7f2436652700 1 -- 172.21.15.69:0/120855896 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24301a0500 con 0x7f243007d560 2024-04-25T11:16:23.406 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.607+0000 7f242d7fa700 1 -- 172.21.15.69:0/120855896 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f242001b8d0 con 0x7f243007d560 2024-04-25T11:16:23.406 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.607+0000 7f242d7fa700 1 -- 172.21.15.69:0/120855896 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2420023750 con 0x7f243007d560 2024-04-25T11:16:23.407 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.607+0000 7f2436652700 1 -- 172.21.15.69:0/120855896 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24301a3100 con 0x7f243007d560 2024-04-25T11:16:23.407 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.608+0000 7f2436652700 1 -- 172.21.15.69:0/120855896 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2430064cf0 con 0x7f243007d560 2024-04-25T11:16:23.407 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.608+0000 7f242d7fa700 1 -- 172.21.15.69:0/120855896 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f2420023a70 con 0x7f243007d560 2024-04-25T11:16:23.407 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.608+0000 7f242d7fa700 1 --2- 172.21.15.69:0/120855896 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f241803d5e0 0x7f241803faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:23.407 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.609+0000 7f242d7fa700 1 -- 172.21.15.69:0/120855896 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2420052ed0 con 0x7f243007d560 2024-04-25T11:16:23.407 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.609+0000 7f242f7fe700 1 --2- 172.21.15.69:0/120855896 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f241803d5e0 0x7f241803faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:23.407 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.609+0000 7f242f7fe700 1 --2- 172.21.15.69:0/120855896 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f241803d5e0 0x7f241803faf0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2424006fd0 tx=0x7f2424006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:23.407 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.614+0000 7f242d7fa700 1 -- 172.21.15.69:0/120855896 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f242001f030 con 0x7f243007d560 2024-04-25T11:16:23.407 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.818+0000 7f2436652700 1 -- 172.21.15.69:0/120855896 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f24301a33f0 con 0x7f243007d560 2024-04-25T11:16:23.407 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.822+0000 7f242d7fa700 1 -- 172.21.15.69:0/120855896 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f24200292a0 con 0x7f243007d560 2024-04-25T11:16:23.407 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.824+0000 7f2436652700 1 -- 172.21.15.69:0/120855896 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f241803d5e0 msgr2=0x7f241803faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:23.407 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.824+0000 7f2436652700 1 --2- 172.21.15.69:0/120855896 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f241803d5e0 0x7f241803faf0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2424006fd0 tx=0x7f2424006e40 comp rx=0 tx=0).stop 2024-04-25T11:16:23.407 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.824+0000 7f2436652700 1 -- 172.21.15.69:0/120855896 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f243007d560 msgr2=0x7f243019fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:23.407 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.824+0000 7f2436652700 1 --2- 172.21.15.69:0/120855896 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f243007d560 0x7f243019fd30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2420011670 tx=0x7f2420011a30 comp rx=0 tx=0).stop 2024-04-25T11:16:23.408 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.825+0000 7f2436652700 1 -- 172.21.15.69:0/120855896 shutdown_connections 2024-04-25T11:16:23.408 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.825+0000 7f2436652700 1 --2- 172.21.15.69:0/120855896 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7f241803d5e0 0x7f241803faf0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:23.408 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.825+0000 7f2436652700 1 --2- 172.21.15.69:0/120855896 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f243007d560 0x7f243019fd30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:23.408 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.825+0000 7f2436652700 1 -- 172.21.15.69:0/120855896 >> 172.21.15.69:0/120855896 conn(0x7f2430103eb0 msgr2=0x7f2430108350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:23.408 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.825+0000 7f2436652700 1 -- 172.21.15.69:0/120855896 shutdown_connections 2024-04-25T11:16:23.408 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:22.825+0000 7f2436652700 1 -- 172.21.15.69:0/120855896 wait complete. 2024-04-25T11:16:23.408 INFO:teuthology.orchestra.run.smithi069.stdout:Enabling the dashboard module... 2024-04-25T11:16:24.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:23 smithi069 ceph-mon[29188]: from='mgr.14120 172.21.15.69:0/157690041' entity='mgr.x' 2024-04-25T11:16:24.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:23 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/120855896' entity='client.admin' 2024-04-25T11:16:25.111 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:24 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2382203215' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-25T11:16:25.112 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:24 smithi069 ceph-mon[29188]: from='mgr.14120 172.21.15.69:0/157690041' entity='mgr.x' 2024-04-25T11:16:25.112 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:24 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: ignoring --setuser ceph since I am not root 2024-04-25T11:16:25.112 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:24 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: ignoring --setgroup ceph since I am not root 2024-04-25T11:16:25.112 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:24 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:16:24.940+0000 7f27109fc200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T11:16:25.450 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:25 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:16:25.125+0000 7f27109fc200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:16:25.450 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:25 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:16:25.231+0000 7f27109fc200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T11:16:25.756 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:23.827+0000 7fb8342f0700 1 Processor -- start 2024-04-25T11:16:25.756 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:23.827+0000 7fb8342f0700 1 -- start start 2024-04-25T11:16:25.756 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:23.827+0000 7fb8342f0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb82c1087e0 0x7fb82c108bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:25.756 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:23.827+0000 7fb8342f0700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c1091e0 con 0x7fb82c1087e0 2024-04-25T11:16:25.756 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:23.828+0000 7fb83208c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb82c1087e0 0x7fb82c108bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:25.756 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:23.828+0000 7fb83208c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb82c1087e0 0x7fb82c108bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33280/0 (socket says 172.21.15.69:33280) 2024-04-25T11:16:25.756 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:23.828+0000 7fb83208c700 1 -- 172.21.15.69:0/2031566169 learned_addr learned my addr 172.21.15.69:0/2031566169 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:16:25.756 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:23.828+0000 7fb83208c700 1 -- 172.21.15.69:0/2031566169 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb82c109910 con 0x7fb82c1087e0 2024-04-25T11:16:25.757 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:23.828+0000 7fb83208c700 1 --2- 172.21.15.69:0/2031566169 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb82c1087e0 0x7fb82c108bf0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb81c0098a0 tx=0x7fb81c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66987ac4aee7bbc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:25.757 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:23.828+0000 7fb83108a700 1 -- 172.21.15.69:0/2031566169 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb81c004160 con 0x7fb82c1087e0 2024-04-25T11:16:25.757 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:23.828+0000 7fb83108a700 1 -- 172.21.15.69:0/2031566169 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb81c004300 con 0x7fb82c1087e0 2024-04-25T11:16:25.757 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:23.828+0000 7fb83108a700 1 -- 172.21.15.69:0/2031566169 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb81c011960 con 0x7fb82c1087e0 2024-04-25T11:16:25.757 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:23.829+0000 7fb8342f0700 1 -- 172.21.15.69:0/2031566169 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb82c1087e0 msgr2=0x7fb82c108bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:25.757 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:23.829+0000 7fb8342f0700 1 --2- 172.21.15.69:0/2031566169 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb82c1087e0 0x7fb82c108bf0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb81c0098a0 tx=0x7fb81c009bb0 comp rx=0 tx=0).stop 2024-04-25T11:16:25.757 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:23.829+0000 7fb8342f0700 1 -- 172.21.15.69:0/2031566169 shutdown_connections 2024-04-25T11:16:25.757 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:23.829+0000 7fb8342f0700 1 --2- 172.21.15.69:0/2031566169 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb82c1087e0 0x7fb82c108bf0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:25.757 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:23.829+0000 7fb8342f0700 1 -- 172.21.15.69:0/2031566169 >> 172.21.15.69:0/2031566169 conn(0x7fb82c103ee0 msgr2=0x7fb82c106330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:25.757 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:23.829+0000 7fb8342f0700 1 -- 172.21.15.69:0/2031566169 shutdown_connections 2024-04-25T11:16:25.757 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:23.829+0000 7fb8342f0700 1 -- 172.21.15.69:0/2031566169 wait complete. 2024-04-25T11:16:25.757 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:23.830+0000 7fb8342f0700 1 Processor -- start 2024-04-25T11:16:25.757 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:23.830+0000 7fb8342f0700 1 -- start start 2024-04-25T11:16:25.757 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:23.830+0000 7fb8342f0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb82c1087e0 0x7fb82c19db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:25.757 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:23.830+0000 7fb8342f0700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c10ccc0 con 0x7fb82c1087e0 2024-04-25T11:16:25.758 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:23.830+0000 7fb83208c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb82c1087e0 0x7fb82c19db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:25.758 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:23.830+0000 7fb83208c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb82c1087e0 0x7fb82c19db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33288/0 (socket says 172.21.15.69:33288) 2024-04-25T11:16:25.758 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:23.830+0000 7fb83208c700 1 -- 172.21.15.69:0/2382203215 learned_addr learned my addr 172.21.15.69:0/2382203215 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:16:25.758 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:23.831+0000 7fb83208c700 1 -- 172.21.15.69:0/2382203215 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb82c19e0b0 con 0x7fb82c1087e0 2024-04-25T11:16:25.758 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:23.831+0000 7fb83208c700 1 --2- 172.21.15.69:0/2382203215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb82c1087e0 0x7fb82c19db70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb81c0119b0 tx=0x7fb81c011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:25.758 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:23.831+0000 7fb8237fe700 1 -- 172.21.15.69:0/2382203215 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb81c003aa0 con 0x7fb82c1087e0 2024-04-25T11:16:25.758 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:23.831+0000 7fb8237fe700 1 -- 172.21.15.69:0/2382203215 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb81c01b8d0 con 0x7fb82c1087e0 2024-04-25T11:16:25.758 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:23.831+0000 7fb8237fe700 1 -- 172.21.15.69:0/2382203215 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb81c023860 con 0x7fb82c1087e0 2024-04-25T11:16:25.758 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:23.831+0000 7fb8342f0700 1 -- 172.21.15.69:0/2382203215 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb82c19e340 con 0x7fb82c1087e0 2024-04-25T11:16:25.758 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:23.831+0000 7fb8342f0700 1 -- 172.21.15.69:0/2382203215 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb82c1a0f50 con 0x7fb82c1087e0 2024-04-25T11:16:25.758 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:23.832+0000 7fb8237fe700 1 -- 172.21.15.69:0/2382203215 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7fb81c02c3f0 con 0x7fb82c1087e0 2024-04-25T11:16:25.758 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:23.832+0000 7fb8342f0700 1 -- 172.21.15.69:0/2382203215 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb82c064cf0 con 0x7fb82c1087e0 2024-04-25T11:16:25.759 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:23.832+0000 7fb8237fe700 1 --2- 172.21.15.69:0/2382203215 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7fb81803d630 0x7fb81803fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:25.759 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:23.832+0000 7fb8237fe700 1 -- 172.21.15.69:0/2382203215 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb81c052d70 con 0x7fb82c1087e0 2024-04-25T11:16:25.759 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:23.832+0000 7fb83188b700 1 --2- 172.21.15.69:0/2382203215 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7fb81803d630 0x7fb81803fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:25.759 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:23.833+0000 7fb83188b700 1 --2- 172.21.15.69:0/2382203215 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7fb81803d630 0x7fb81803fb40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb828006fd0 tx=0x7fb828006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:25.759 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:23.838+0000 7fb8237fe700 1 -- 172.21.15.69:0/2382203215 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb81c01f030 con 0x7fb82c1087e0 2024-04-25T11:16:25.759 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:24.109+0000 7fb8342f0700 1 -- 172.21.15.69:0/2382203215 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fb82c1a1120 con 0x7fb82c1087e0 2024-04-25T11:16:25.759 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:24.735+0000 7fb8237fe700 1 -- 172.21.15.69:0/2382203215 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fb82c1a1120 con 0x7fb82c1087e0 2024-04-25T11:16:25.759 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:24.737+0000 7fb8237fe700 1 -- 172.21.15.69:0/2382203215 <== mon.0 v2:172.21.15.69:3300/0 8 ==== mgrmap(e 9) ==== 49529+0+0 (secure 0 0 0) 0x7fb81c02cb00 con 0x7fb82c1087e0 2024-04-25T11:16:25.759 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:24.738+0000 7fb8342f0700 1 -- 172.21.15.69:0/2382203215 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7fb81803d630 msgr2=0x7fb81803fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:25.759 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:24.738+0000 7fb8342f0700 1 --2- 172.21.15.69:0/2382203215 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7fb81803d630 0x7fb81803fb40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb828006fd0 tx=0x7fb828006e40 comp rx=0 tx=0).stop 2024-04-25T11:16:25.759 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:24.738+0000 7fb8342f0700 1 -- 172.21.15.69:0/2382203215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb82c1087e0 msgr2=0x7fb82c19db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:25.759 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:24.738+0000 7fb8342f0700 1 --2- 172.21.15.69:0/2382203215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb82c1087e0 0x7fb82c19db70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb81c0119b0 tx=0x7fb81c011a90 comp rx=0 tx=0).stop 2024-04-25T11:16:25.759 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:24.738+0000 7fb8342f0700 1 -- 172.21.15.69:0/2382203215 shutdown_connections 2024-04-25T11:16:25.760 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:24.738+0000 7fb8342f0700 1 --2- 172.21.15.69:0/2382203215 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7fb81803d630 0x7fb81803fb40 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:25.760 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:24.738+0000 7fb8342f0700 1 --2- 172.21.15.69:0/2382203215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb82c1087e0 0x7fb82c19db70 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:25.760 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:24.738+0000 7fb8342f0700 1 -- 172.21.15.69:0/2382203215 >> 172.21.15.69:0/2382203215 conn(0x7fb82c103ee0 msgr2=0x7fb82c105840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:25.760 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:24.738+0000 7fb8342f0700 1 -- 172.21.15.69:0/2382203215 shutdown_connections 2024-04-25T11:16:25.760 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:24.738+0000 7fb8342f0700 1 -- 172.21.15.69:0/2382203215 wait complete. 2024-04-25T11:16:25.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:25 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2382203215' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-25T11:16:25.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:25 smithi069 ceph-mon[29188]: mgrmap e9: x(active, since 19s) 2024-04-25T11:16:25.858 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:25 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:16:25.541+0000 7f27109fc200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T11:16:25.858 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:25 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:16:25.627+0000 7f27109fc200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T11:16:27.192 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:26 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/3061728645' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T11:16:27.193 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:26 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:16:26.838+0000 7f27109fc200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T11:16:27.193 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:27 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:16:27.020+0000 7f27109fc200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T11:16:27.193 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:27 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:16:27.192+0000 7f27109fc200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T11:16:27.523 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout { 2024-04-25T11:16:27.524 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-25T11:16:27.524 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T11:16:27.524 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T11:16:27.524 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T11:16:27.524 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout } 2024-04-25T11:16:27.524 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.649+0000 7fe3d5956700 1 Processor -- start 2024-04-25T11:16:27.524 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.649+0000 7fe3d5956700 1 -- start start 2024-04-25T11:16:27.524 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.649+0000 7fe3d5956700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3d01082a0 0x7fe3d01086b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:27.524 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.649+0000 7fe3d5956700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d007d710 con 0x7fe3d01082a0 2024-04-25T11:16:27.524 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.649+0000 7fe3ceffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3d01082a0 0x7fe3d01086b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:27.524 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.649+0000 7fe3ceffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3d01082a0 0x7fe3d01086b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33326/0 (socket says 172.21.15.69:33326) 2024-04-25T11:16:27.524 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.649+0000 7fe3ceffd700 1 -- 172.21.15.69:0/96236797 learned_addr learned my addr 172.21.15.69:0/96236797 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:16:27.524 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.650+0000 7fe3ceffd700 1 -- 172.21.15.69:0/96236797 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d0108bf0 con 0x7fe3d01082a0 2024-04-25T11:16:27.524 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.650+0000 7fe3ceffd700 1 --2- 172.21.15.69:0/96236797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3d01082a0 0x7fe3d01086b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe3c00098a0 tx=0x7fe3c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b51be8692201b050 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:27.525 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.650+0000 7fe3cdffb700 1 -- 172.21.15.69:0/96236797 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3c0004160 con 0x7fe3d01082a0 2024-04-25T11:16:27.525 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.650+0000 7fe3cdffb700 1 -- 172.21.15.69:0/96236797 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe3c0004300 con 0x7fe3d01082a0 2024-04-25T11:16:27.525 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.650+0000 7fe3cdffb700 1 -- 172.21.15.69:0/96236797 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3c0011960 con 0x7fe3d01082a0 2024-04-25T11:16:27.525 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.651+0000 7fe3d5956700 1 -- 172.21.15.69:0/96236797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3d01082a0 msgr2=0x7fe3d01086b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:27.525 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.651+0000 7fe3d5956700 1 --2- 172.21.15.69:0/96236797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3d01082a0 0x7fe3d01086b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe3c00098a0 tx=0x7fe3c0009bb0 comp rx=0 tx=0).stop 2024-04-25T11:16:27.525 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.651+0000 7fe3d5956700 1 -- 172.21.15.69:0/96236797 shutdown_connections 2024-04-25T11:16:27.525 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.651+0000 7fe3d5956700 1 --2- 172.21.15.69:0/96236797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3d01082a0 0x7fe3d01086b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:27.525 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.651+0000 7fe3d5956700 1 -- 172.21.15.69:0/96236797 >> 172.21.15.69:0/96236797 conn(0x7fe3d0103ef0 msgr2=0x7fe3d0106340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:27.525 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.651+0000 7fe3d5956700 1 -- 172.21.15.69:0/96236797 shutdown_connections 2024-04-25T11:16:27.525 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.651+0000 7fe3d5956700 1 -- 172.21.15.69:0/96236797 wait complete. 2024-04-25T11:16:27.525 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.651+0000 7fe3d5956700 1 Processor -- start 2024-04-25T11:16:27.525 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.652+0000 7fe3d5956700 1 -- start start 2024-04-25T11:16:27.525 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.652+0000 7fe3d5956700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3d01082a0 0x7fe3d019fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:27.525 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.652+0000 7fe3d5956700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d0109c50 con 0x7fe3d01082a0 2024-04-25T11:16:27.525 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.652+0000 7fe3ceffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3d01082a0 0x7fe3d019fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:27.526 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.652+0000 7fe3ceffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3d01082a0 0x7fe3d019fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33338/0 (socket says 172.21.15.69:33338) 2024-04-25T11:16:27.526 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.652+0000 7fe3ceffd700 1 -- 172.21.15.69:0/3061728645 learned_addr learned my addr 172.21.15.69:0/3061728645 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:16:27.526 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.652+0000 7fe3ceffd700 1 -- 172.21.15.69:0/3061728645 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d01a03c0 con 0x7fe3d01082a0 2024-04-25T11:16:27.526 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.652+0000 7fe3ceffd700 1 --2- 172.21.15.69:0/3061728645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3d01082a0 0x7fe3d019fe80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0011670 tx=0x7fe3c0011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:27.526 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.652+0000 7fe3d4954700 1 -- 172.21.15.69:0/3061728645 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3c0003a60 con 0x7fe3d01082a0 2024-04-25T11:16:27.526 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.653+0000 7fe3d4954700 1 -- 172.21.15.69:0/3061728645 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe3c001b8d0 con 0x7fe3d01082a0 2024-04-25T11:16:27.526 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.653+0000 7fe3d5956700 1 -- 172.21.15.69:0/3061728645 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3d01a0650 con 0x7fe3d01082a0 2024-04-25T11:16:27.526 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.653+0000 7fe3d4954700 1 -- 172.21.15.69:0/3061728645 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3c0023750 con 0x7fe3d01082a0 2024-04-25T11:16:27.526 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.653+0000 7fe3d5956700 1 -- 172.21.15.69:0/3061728645 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3d01a3260 con 0x7fe3d01082a0 2024-04-25T11:16:27.526 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.654+0000 7fe3d4954700 1 -- 172.21.15.69:0/3061728645 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 9) ==== 49529+0+0 (secure 0 0 0) 0x7fe3c0023a40 con 0x7fe3d01082a0 2024-04-25T11:16:27.526 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.654+0000 7fe3d5956700 1 -- 172.21.15.69:0/3061728645 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3d0064cf0 con 0x7fe3d01082a0 2024-04-25T11:16:27.527 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.654+0000 7fe3d4954700 1 --2- 172.21.15.69:0/3061728645 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7fe3b803d690 0x7fe3b803fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:27.527 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.654+0000 7fe3d4954700 1 -- 172.21.15.69:0/3061728645 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe3c0052e50 con 0x7fe3d01082a0 2024-04-25T11:16:27.527 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.654+0000 7fe3ce7fc700 1 -- 172.21.15.69:0/3061728645 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7fe3b803d690 msgr2=0x7fe3b803fba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/1097124842 2024-04-25T11:16:27.527 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.654+0000 7fe3ce7fc700 1 --2- 172.21.15.69:0/3061728645 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7fe3b803d690 0x7fe3b803fba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T11:16:27.527 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.659+0000 7fe3d4954700 1 -- 172.21.15.69:0/3061728645 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe3c001f030 con 0x7fe3d01082a0 2024-04-25T11:16:27.527 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.855+0000 7fe3ce7fc700 1 -- 172.21.15.69:0/3061728645 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7fe3b803d690 msgr2=0x7fe3b803fba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/1097124842 2024-04-25T11:16:27.527 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.855+0000 7fe3ce7fc700 1 --2- 172.21.15.69:0/3061728645 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7fe3b803d690 0x7fe3b803fba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T11:16:27.527 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.917+0000 7fe3d5956700 1 -- 172.21.15.69:0/3061728645 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fe3d002e690 con 0x7fe3d01082a0 2024-04-25T11:16:27.527 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.917+0000 7fe3d4954700 1 -- 172.21.15.69:0/3061728645 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fe3c0054070 con 0x7fe3d01082a0 2024-04-25T11:16:27.527 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.919+0000 7fe3d5956700 1 -- 172.21.15.69:0/3061728645 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7fe3b803d690 msgr2=0x7fe3b803fba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:16:27.527 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.919+0000 7fe3d5956700 1 --2- 172.21.15.69:0/3061728645 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7fe3b803d690 0x7fe3b803fba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:27.527 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.919+0000 7fe3d5956700 1 -- 172.21.15.69:0/3061728645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3d01082a0 msgr2=0x7fe3d019fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:27.527 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.919+0000 7fe3d5956700 1 --2- 172.21.15.69:0/3061728645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3d01082a0 0x7fe3d019fe80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0011670 tx=0x7fe3c0011a30 comp rx=0 tx=0).stop 2024-04-25T11:16:27.527 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.919+0000 7fe3d5956700 1 -- 172.21.15.69:0/3061728645 shutdown_connections 2024-04-25T11:16:27.528 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.919+0000 7fe3d5956700 1 --2- 172.21.15.69:0/3061728645 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7fe3b803d690 0x7fe3b803fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:27.528 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.919+0000 7fe3d5956700 1 --2- 172.21.15.69:0/3061728645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3d01082a0 0x7fe3d019fe80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:27.528 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.919+0000 7fe3d5956700 1 -- 172.21.15.69:0/3061728645 >> 172.21.15.69:0/3061728645 conn(0x7fe3d0103ef0 msgr2=0x7fe3d0193e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:27.528 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.920+0000 7fe3d5956700 1 -- 172.21.15.69:0/3061728645 shutdown_connections 2024-04-25T11:16:27.528 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:26.920+0000 7fe3d5956700 1 -- 172.21.15.69:0/3061728645 wait complete. 2024-04-25T11:16:27.528 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for the mgr to restart... 2024-04-25T11:16:27.528 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for mgr epoch 9... 2024-04-25T11:16:27.740 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:27 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:16:27.489+0000 7f27109fc200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T11:16:27.740 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:27 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:16:27.587+0000 7f27109fc200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T11:16:28.142 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:27 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:16:27.793+0000 7f27109fc200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T11:16:28.142 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:27 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:16:27.998+0000 7f27109fc200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:16:28.949 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:28 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:16:28.594+0000 7f27109fc200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T11:16:29.321 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:29 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:16:29.019+0000 7f27109fc200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T11:16:29.321 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:29 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:16:29.116+0000 7f27109fc200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T11:16:29.586 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:29 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:16:29.320+0000 7f27109fc200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T11:16:29.863 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:29 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:16:29.862+0000 7f27109fc200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T11:16:30.199 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:29 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:16:29.961+0000 7f27109fc200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T11:16:30.524 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:30 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:16:30.212+0000 7f27109fc200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T11:16:30.524 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:30 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:16:30.348+0000 7f27109fc200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T11:16:30.524 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:30 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:16:30.448+0000 7f27109fc200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T11:16:30.867 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:30 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:16:30.569+0000 7f27109fc200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T11:16:30.867 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:30 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:16:30.867+0000 7f27109fc200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T11:16:31.740 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:31 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:16:31.466+0000 7f27109fc200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T11:16:31.740 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:16:31 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:16:31.685+0000 7f27109fc200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T11:16:32.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:31 smithi069 ceph-mon[29188]: Active manager daemon x restarted 2024-04-25T11:16:32.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:31 smithi069 ceph-mon[29188]: Activating manager daemon x 2024-04-25T11:16:32.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:31 smithi069 ceph-mon[29188]: osdmap e3: 0 total, 0 up, 0 in 2024-04-25T11:16:32.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:31 smithi069 ceph-mon[29188]: mgrmap e10: x(active, starting, since 0.00431121s) 2024-04-25T11:16:32.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:31 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T11:16:32.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:31 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T11:16:32.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:31 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T11:16:32.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:31 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T11:16:32.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:31 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T11:16:32.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:31 smithi069 ceph-mon[29188]: Manager daemon x is now available 2024-04-25T11:16:32.201 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:31 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:16:32.909 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:32 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T11:16:32.910 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:32 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T11:16:32.910 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:32 smithi069 ceph-mon[29188]: mgrmap e11: x(active, since 1.00793s) 2024-04-25T11:16:33.717 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout { 2024-04-25T11:16:33.717 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-25T11:16:33.717 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T11:16:33.718 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout } 2024-04-25T11:16:33.718 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:27.936+0000 7feda6734700 1 Processor -- start 2024-04-25T11:16:33.718 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:27.936+0000 7feda6734700 1 -- start start 2024-04-25T11:16:33.718 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:27.936+0000 7feda6734700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feda01098f0 0x7feda0109d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:33.718 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:27.936+0000 7feda6734700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7feda010a2f0 con 0x7feda01098f0 2024-04-25T11:16:33.718 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:27.936+0000 7fed9ffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feda01098f0 0x7feda0109d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:33.718 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:27.936+0000 7fed9ffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feda01098f0 0x7feda0109d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33342/0 (socket says 172.21.15.69:33342) 2024-04-25T11:16:33.718 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:27.936+0000 7fed9ffff700 1 -- 172.21.15.69:0/1709330737 learned_addr learned my addr 172.21.15.69:0/1709330737 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:16:33.718 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:27.936+0000 7fed9ffff700 1 -- 172.21.15.69:0/1709330737 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda010aa20 con 0x7feda01098f0 2024-04-25T11:16:33.718 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:27.937+0000 7fed9ffff700 1 --2- 172.21.15.69:0/1709330737 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feda01098f0 0x7feda0109d00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fed900098a0 tx=0x7fed90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dfec28e8935d7f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:33.718 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:27.937+0000 7fed9effd700 1 -- 172.21.15.69:0/1709330737 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed90004160 con 0x7feda01098f0 2024-04-25T11:16:33.718 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:27.937+0000 7fed9effd700 1 -- 172.21.15.69:0/1709330737 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed90004300 con 0x7feda01098f0 2024-04-25T11:16:33.718 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:27.937+0000 7fed9effd700 1 -- 172.21.15.69:0/1709330737 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed90011960 con 0x7feda01098f0 2024-04-25T11:16:33.719 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:27.938+0000 7feda6734700 1 -- 172.21.15.69:0/1709330737 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feda01098f0 msgr2=0x7feda0109d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:33.719 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:27.938+0000 7feda6734700 1 --2- 172.21.15.69:0/1709330737 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feda01098f0 0x7feda0109d00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fed900098a0 tx=0x7fed90009bb0 comp rx=0 tx=0).stop 2024-04-25T11:16:33.719 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:27.938+0000 7feda6734700 1 -- 172.21.15.69:0/1709330737 shutdown_connections 2024-04-25T11:16:33.719 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:27.938+0000 7feda6734700 1 --2- 172.21.15.69:0/1709330737 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feda01098f0 0x7feda0109d00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:33.719 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:27.938+0000 7feda6734700 1 -- 172.21.15.69:0/1709330737 >> 172.21.15.69:0/1709330737 conn(0x7feda0104f50 msgr2=0x7feda01073e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:33.719 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:27.938+0000 7feda6734700 1 -- 172.21.15.69:0/1709330737 shutdown_connections 2024-04-25T11:16:33.719 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:27.938+0000 7feda6734700 1 -- 172.21.15.69:0/1709330737 wait complete. 2024-04-25T11:16:33.719 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:27.938+0000 7feda6734700 1 Processor -- start 2024-04-25T11:16:33.719 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:27.938+0000 7feda6734700 1 -- start start 2024-04-25T11:16:33.719 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:27.939+0000 7feda6734700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feda01098f0 0x7feda019dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:33.719 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:27.939+0000 7feda6734700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7feda010ddd0 con 0x7feda01098f0 2024-04-25T11:16:33.719 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:27.939+0000 7fed9ffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feda01098f0 0x7feda019dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:33.719 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:27.939+0000 7fed9ffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feda01098f0 0x7feda019dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33344/0 (socket says 172.21.15.69:33344) 2024-04-25T11:16:33.720 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:27.939+0000 7fed9ffff700 1 -- 172.21.15.69:0/2151655444 learned_addr learned my addr 172.21.15.69:0/2151655444 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:16:33.720 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:27.939+0000 7fed9ffff700 1 -- 172.21.15.69:0/2151655444 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda019e000 con 0x7feda01098f0 2024-04-25T11:16:33.720 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:27.939+0000 7fed9ffff700 1 --2- 172.21.15.69:0/2151655444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feda01098f0 0x7feda019dac0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fed900119b0 tx=0x7fed90011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:33.720 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:27.939+0000 7fed9d7fa700 1 -- 172.21.15.69:0/2151655444 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed90003aa0 con 0x7feda01098f0 2024-04-25T11:16:33.720 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:27.939+0000 7fed9d7fa700 1 -- 172.21.15.69:0/2151655444 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed9001b8d0 con 0x7feda01098f0 2024-04-25T11:16:33.720 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:27.939+0000 7fed9d7fa700 1 -- 172.21.15.69:0/2151655444 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed900237d0 con 0x7feda01098f0 2024-04-25T11:16:33.720 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:27.939+0000 7feda6734700 1 -- 172.21.15.69:0/2151655444 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feda019e290 con 0x7feda01098f0 2024-04-25T11:16:33.720 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:27.940+0000 7feda6734700 1 -- 172.21.15.69:0/2151655444 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feda01a0ea0 con 0x7feda01098f0 2024-04-25T11:16:33.720 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:27.941+0000 7fed9d7fa700 1 -- 172.21.15.69:0/2151655444 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 9) ==== 49529+0+0 (secure 0 0 0) 0x7fed9002c3f0 con 0x7feda01098f0 2024-04-25T11:16:33.720 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:27.941+0000 7fed9d7fa700 1 --2- 172.21.15.69:0/2151655444 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7fed8803d690 0x7fed8803fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:33.720 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:27.941+0000 7fed9d7fa700 1 -- 172.21.15.69:0/2151655444 --> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fed880402b0 con 0x7fed8803d690 2024-04-25T11:16:33.720 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:27.941+0000 7fed9f7fe700 1 -- 172.21.15.69:0/2151655444 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7fed8803d690 msgr2=0x7fed8803fba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/1097124842 2024-04-25T11:16:33.720 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:27.941+0000 7fed9d7fa700 1 -- 172.21.15.69:0/2151655444 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fed90052cd0 con 0x7feda01098f0 2024-04-25T11:16:33.720 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:27.941+0000 7fed9f7fe700 1 --2- 172.21.15.69:0/2151655444 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7fed8803d690 0x7fed8803fba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T11:16:33.720 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:28.142+0000 7fed9f7fe700 1 -- 172.21.15.69:0/2151655444 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7fed8803d690 msgr2=0x7fed8803fba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/1097124842 2024-04-25T11:16:33.721 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:28.142+0000 7fed9f7fe700 1 --2- 172.21.15.69:0/2151655444 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7fed8803d690 0x7fed8803fba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T11:16:33.721 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:28.543+0000 7fed9f7fe700 1 -- 172.21.15.69:0/2151655444 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7fed8803d690 msgr2=0x7fed8803fba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/1097124842 2024-04-25T11:16:33.721 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:28.543+0000 7fed9f7fe700 1 --2- 172.21.15.69:0/2151655444 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7fed8803d690 0x7fed8803fba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T11:16:33.721 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:29.344+0000 7fed9f7fe700 1 -- 172.21.15.69:0/2151655444 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7fed8803d690 msgr2=0x7fed8803fba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/1097124842 2024-04-25T11:16:33.721 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:29.344+0000 7fed9f7fe700 1 --2- 172.21.15.69:0/2151655444 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7fed8803d690 0x7fed8803fba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T11:16:33.721 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:30.946+0000 7fed9f7fe700 1 -- 172.21.15.69:0/2151655444 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7fed8803d690 msgr2=0x7fed8803fba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/1097124842 2024-04-25T11:16:33.721 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:30.946+0000 7fed9f7fe700 1 --2- 172.21.15.69:0/2151655444 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7fed8803d690 0x7fed8803fba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T11:16:33.721 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:31.690+0000 7fed9d7fa700 1 -- 172.21.15.69:0/2151655444 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mgrmap(e 10) ==== 49296+0+0 (secure 0 0 0) 0x7fed9002c780 con 0x7feda01098f0 2024-04-25T11:16:33.721 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:31.690+0000 7fed9d7fa700 1 -- 172.21.15.69:0/2151655444 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7fed8803d690 msgr2=0x7fed8803fba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:16:33.722 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:31.690+0000 7fed9d7fa700 1 --2- 172.21.15.69:0/2151655444 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7fed8803d690 0x7fed8803fba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:33.722 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:32.691+0000 7fed9d7fa700 1 -- 172.21.15.69:0/2151655444 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mgrmap(e 11) ==== 49423+0+0 (secure 0 0 0) 0x7fed90054660 con 0x7feda01098f0 2024-04-25T11:16:33.722 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:32.692+0000 7fed9d7fa700 1 --2- 172.21.15.69:0/2151655444 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fed90052860 0x7fed88042fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:33.722 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:32.692+0000 7fed9d7fa700 1 -- 172.21.15.69:0/2151655444 --> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fed90008e50 con 0x7fed90052860 2024-04-25T11:16:33.722 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:32.692+0000 7fed9f7fe700 1 --2- 172.21.15.69:0/2151655444 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fed90052860 0x7fed88042fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:33.722 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:32.693+0000 7fed9f7fe700 1 --2- 172.21.15.69:0/2151655444 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fed90052860 0x7fed88042fe0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fed94003a10 tx=0x7fed94009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:33.722 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:32.694+0000 7fed9d7fa700 1 -- 172.21.15.69:0/2151655444 <== mgr.14152 v2:172.21.15.69:6800/4220065409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fed90008e50 con 0x7fed90052860 2024-04-25T11:16:33.722 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:32.702+0000 7feda6734700 1 -- 172.21.15.69:0/2151655444 --> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7feda0064cf0 con 0x7fed90052860 2024-04-25T11:16:33.723 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:32.702+0000 7fed9d7fa700 1 -- 172.21.15.69:0/2151655444 <== mgr.14152 v2:172.21.15.69:6800/4220065409 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7feda0064cf0 con 0x7fed90052860 2024-04-25T11:16:33.723 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:32.703+0000 7feda6734700 1 -- 172.21.15.69:0/2151655444 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fed90052860 msgr2=0x7fed88042fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:33.723 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:32.703+0000 7feda6734700 1 --2- 172.21.15.69:0/2151655444 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fed90052860 0x7fed88042fe0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fed94003a10 tx=0x7fed94009270 comp rx=0 tx=0).stop 2024-04-25T11:16:33.723 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:32.703+0000 7feda6734700 1 -- 172.21.15.69:0/2151655444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feda01098f0 msgr2=0x7feda019dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:33.723 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:32.703+0000 7feda6734700 1 --2- 172.21.15.69:0/2151655444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feda01098f0 0x7feda019dac0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fed900119b0 tx=0x7fed90011a90 comp rx=0 tx=0).stop 2024-04-25T11:16:33.723 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:32.703+0000 7feda6734700 1 -- 172.21.15.69:0/2151655444 shutdown_connections 2024-04-25T11:16:33.723 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:32.703+0000 7feda6734700 1 --2- 172.21.15.69:0/2151655444 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fed90052860 0x7fed88042fe0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:33.723 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:32.703+0000 7feda6734700 1 --2- 172.21.15.69:0/2151655444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feda01098f0 0x7feda019dac0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:33.723 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:32.703+0000 7feda6734700 1 --2- 172.21.15.69:0/2151655444 >> [v2:172.21.15.69:6800/1097124842,v1:172.21.15.69:6801/1097124842] conn(0x7fed8803d690 0x7fed8803fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:33.723 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:32.703+0000 7feda6734700 1 -- 172.21.15.69:0/2151655444 >> 172.21.15.69:0/2151655444 conn(0x7feda0104f50 msgr2=0x7feda0105780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:33.723 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:32.703+0000 7feda6734700 1 -- 172.21.15.69:0/2151655444 shutdown_connections 2024-04-25T11:16:33.723 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:32.703+0000 7feda6734700 1 -- 172.21.15.69:0/2151655444 wait complete. 2024-04-25T11:16:33.724 INFO:teuthology.orchestra.run.smithi069.stdout:mgr epoch 9 is available 2024-04-25T11:16:33.724 INFO:teuthology.orchestra.run.smithi069.stdout:Generating a dashboard self-signed certificate... 2024-04-25T11:16:34.019 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:33 smithi069 ceph-mon[29188]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T11:16:34.019 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:33 smithi069 ceph-mon[29188]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T11:16:34.019 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:33 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:16:35.063 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:34 smithi069 ceph-mon[29188]: [25/Apr/2024:11:16:32] ENGINE Bus STARTING 2024-04-25T11:16:35.064 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:34 smithi069 ceph-mon[29188]: [25/Apr/2024:11:16:33] ENGINE Serving on http://172.21.15.69:8765 2024-04-25T11:16:35.064 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:34 smithi069 ceph-mon[29188]: [25/Apr/2024:11:16:33] ENGINE Serving on https://172.21.15.69:7150 2024-04-25T11:16:35.064 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:34 smithi069 ceph-mon[29188]: [25/Apr/2024:11:16:33] ENGINE Bus STARTED 2024-04-25T11:16:35.064 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:34 smithi069 ceph-mon[29188]: mgrmap e12: x(active, since 2s) 2024-04-25T11:16:35.064 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:34 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:16:35.064 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:34 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:16:35.064 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:34 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:16:35.072 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-25T11:16:35.072 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.128+0000 7f70133d0700 1 Processor -- start 2024-04-25T11:16:35.072 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.128+0000 7f70133d0700 1 -- start start 2024-04-25T11:16:35.072 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.129+0000 7f70133d0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f700c10a3c0 0x7f700c10c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:35.072 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.129+0000 7f70133d0700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f700c07ce00 con 0x7f700c10a3c0 2024-04-25T11:16:35.072 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.129+0000 7f701116c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f700c10a3c0 0x7f700c10c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:35.072 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.129+0000 7f701116c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f700c10a3c0 0x7f700c10c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47294/0 (socket says 172.21.15.69:47294) 2024-04-25T11:16:35.072 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.129+0000 7f701116c700 1 -- 172.21.15.69:0/635702741 learned_addr learned my addr 172.21.15.69:0/635702741 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:16:35.072 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.129+0000 7f701116c700 1 -- 172.21.15.69:0/635702741 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f700c10cd20 con 0x7f700c10a3c0 2024-04-25T11:16:35.072 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.130+0000 7f701116c700 1 --2- 172.21.15.69:0/635702741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f700c10a3c0 0x7f700c10c7e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f70080098a0 tx=0x7f7008009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dbbd4b8f627f1b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:35.072 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.130+0000 7f7003fff700 1 -- 172.21.15.69:0/635702741 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7008004160 con 0x7f700c10a3c0 2024-04-25T11:16:35.073 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.130+0000 7f7003fff700 1 -- 172.21.15.69:0/635702741 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7008004300 con 0x7f700c10a3c0 2024-04-25T11:16:35.073 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.130+0000 7f7003fff700 1 -- 172.21.15.69:0/635702741 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7008011960 con 0x7f700c10a3c0 2024-04-25T11:16:35.073 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.131+0000 7f70133d0700 1 -- 172.21.15.69:0/635702741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f700c10a3c0 msgr2=0x7f700c10c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:35.073 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.131+0000 7f70133d0700 1 --2- 172.21.15.69:0/635702741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f700c10a3c0 0x7f700c10c7e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f70080098a0 tx=0x7f7008009bb0 comp rx=0 tx=0).stop 2024-04-25T11:16:35.073 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.131+0000 7f70133d0700 1 -- 172.21.15.69:0/635702741 shutdown_connections 2024-04-25T11:16:35.073 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.131+0000 7f70133d0700 1 --2- 172.21.15.69:0/635702741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f700c10a3c0 0x7f700c10c7e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:35.073 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.131+0000 7f70133d0700 1 -- 172.21.15.69:0/635702741 >> 172.21.15.69:0/635702741 conn(0x7f700c103ee0 msgr2=0x7f700c106330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:35.073 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.131+0000 7f70133d0700 1 -- 172.21.15.69:0/635702741 shutdown_connections 2024-04-25T11:16:35.073 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.131+0000 7f70133d0700 1 -- 172.21.15.69:0/635702741 wait complete. 2024-04-25T11:16:35.073 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.132+0000 7f70133d0700 1 Processor -- start 2024-04-25T11:16:35.074 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.132+0000 7f70133d0700 1 -- start start 2024-04-25T11:16:35.074 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.132+0000 7f70133d0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f700c10a3c0 0x7f700c19db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:35.074 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.132+0000 7f70133d0700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f700c10dc00 con 0x7f700c10a3c0 2024-04-25T11:16:35.074 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.132+0000 7f701116c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f700c10a3c0 0x7f700c19db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:35.074 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.132+0000 7f701116c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f700c10a3c0 0x7f700c19db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47310/0 (socket says 172.21.15.69:47310) 2024-04-25T11:16:35.074 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.132+0000 7f701116c700 1 -- 172.21.15.69:0/2505586852 learned_addr learned my addr 172.21.15.69:0/2505586852 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:16:35.074 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.133+0000 7f701116c700 1 -- 172.21.15.69:0/2505586852 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f700c19e070 con 0x7f700c10a3c0 2024-04-25T11:16:35.074 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.133+0000 7f701116c700 1 --2- 172.21.15.69:0/2505586852 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f700c10a3c0 0x7f700c19db30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7008011670 tx=0x7f7008011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:35.074 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.133+0000 7f70027fc700 1 -- 172.21.15.69:0/2505586852 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7008003a60 con 0x7f700c10a3c0 2024-04-25T11:16:35.074 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.133+0000 7f70027fc700 1 -- 172.21.15.69:0/2505586852 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f700801b8d0 con 0x7f700c10a3c0 2024-04-25T11:16:35.074 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.133+0000 7f70027fc700 1 -- 172.21.15.69:0/2505586852 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7008023750 con 0x7f700c10a3c0 2024-04-25T11:16:35.074 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.133+0000 7f70133d0700 1 -- 172.21.15.69:0/2505586852 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f700c19e300 con 0x7f700c10a3c0 2024-04-25T11:16:35.075 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.133+0000 7f70133d0700 1 -- 172.21.15.69:0/2505586852 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f700c1a0f10 con 0x7f700c10a3c0 2024-04-25T11:16:35.075 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.134+0000 7f70027fc700 1 -- 172.21.15.69:0/2505586852 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 11) ==== 49423+0+0 (secure 0 0 0) 0x7f7008023a50 con 0x7f700c10a3c0 2024-04-25T11:16:35.075 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.134+0000 7f70133d0700 1 -- 172.21.15.69:0/2505586852 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f700c064cf0 con 0x7f700c10a3c0 2024-04-25T11:16:35.075 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.134+0000 7f70027fc700 1 --2- 172.21.15.69:0/2505586852 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f6ff803d520 0x7f6ff803fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:35.075 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.134+0000 7f70027fc700 1 -- 172.21.15.69:0/2505586852 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7008052d80 con 0x7f700c10a3c0 2024-04-25T11:16:35.075 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.134+0000 7f701096b700 1 --2- 172.21.15.69:0/2505586852 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f6ff803d520 0x7f6ff803fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:35.075 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.135+0000 7f701096b700 1 --2- 172.21.15.69:0/2505586852 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f6ff803d520 0x7f6ff803fa30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc006fd0 tx=0x7f6ffc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:35.075 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.139+0000 7f70027fc700 1 -- 172.21.15.69:0/2505586852 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f700801f030 con 0x7f700c10a3c0 2024-04-25T11:16:35.075 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.297+0000 7f70027fc700 1 -- 172.21.15.69:0/2505586852 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mgrmap(e 12) ==== 49529+0+0 (secure 0 0 0) 0x7f700802c3f0 con 0x7f700c10a3c0 2024-04-25T11:16:35.075 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.351+0000 7f70133d0700 1 -- 172.21.15.69:0/2505586852 --> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f700c105f50 con 0x7f6ff803d520 2024-04-25T11:16:35.075 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.508+0000 7f70027fc700 1 -- 172.21.15.69:0/2505586852 <== mgr.14152 v2:172.21.15.69:6800/4220065409 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f700c105f50 con 0x7f6ff803d520 2024-04-25T11:16:35.075 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.510+0000 7f70133d0700 1 -- 172.21.15.69:0/2505586852 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f6ff803d520 msgr2=0x7f6ff803fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:35.075 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.510+0000 7f70133d0700 1 --2- 172.21.15.69:0/2505586852 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f6ff803d520 0x7f6ff803fa30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc006fd0 tx=0x7f6ffc006e40 comp rx=0 tx=0).stop 2024-04-25T11:16:35.076 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.510+0000 7f70133d0700 1 -- 172.21.15.69:0/2505586852 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f700c10a3c0 msgr2=0x7f700c19db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:35.076 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.510+0000 7f70133d0700 1 --2- 172.21.15.69:0/2505586852 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f700c10a3c0 0x7f700c19db30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7008011670 tx=0x7f7008011a30 comp rx=0 tx=0).stop 2024-04-25T11:16:35.076 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.511+0000 7f70133d0700 1 -- 172.21.15.69:0/2505586852 shutdown_connections 2024-04-25T11:16:35.076 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.511+0000 7f70133d0700 1 --2- 172.21.15.69:0/2505586852 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f6ff803d520 0x7f6ff803fa30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:35.076 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.511+0000 7f70133d0700 1 --2- 172.21.15.69:0/2505586852 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f700c10a3c0 0x7f700c19db30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:35.076 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.511+0000 7f70133d0700 1 -- 172.21.15.69:0/2505586852 >> 172.21.15.69:0/2505586852 conn(0x7f700c103ee0 msgr2=0x7f700c105840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:35.076 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.511+0000 7f70133d0700 1 -- 172.21.15.69:0/2505586852 shutdown_connections 2024-04-25T11:16:35.076 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:34.511+0000 7f70133d0700 1 -- 172.21.15.69:0/2505586852 wait complete. 2024-04-25T11:16:35.076 INFO:teuthology.orchestra.run.smithi069.stdout:Creating initial admin user... 2024-04-25T11:16:35.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:35 smithi069 ceph-mon[29188]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:16:36.656 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$hGdTXFf6t81qXS6DlfI7y.kAu7TmeF.IjHa9TGZMXYzboSR99Anh.", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714043795, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-25T11:16:36.656 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.490+0000 7f59cf7ce700 1 Processor -- start 2024-04-25T11:16:36.656 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.491+0000 7f59cf7ce700 1 -- start start 2024-04-25T11:16:36.656 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.491+0000 7f59cf7ce700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59c807ccf0 0x7f59c810c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:36.657 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.491+0000 7f59cf7ce700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f59c810ca60 con 0x7f59c807ccf0 2024-04-25T11:16:36.657 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.491+0000 7f59cd56a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59c807ccf0 0x7f59c810c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:36.657 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.491+0000 7f59cd56a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59c807ccf0 0x7f59c810c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47322/0 (socket says 172.21.15.69:47322) 2024-04-25T11:16:36.657 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.491+0000 7f59cd56a700 1 -- 172.21.15.69:0/3774821304 learned_addr learned my addr 172.21.15.69:0/3774821304 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:16:36.657 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.491+0000 7f59cd56a700 1 -- 172.21.15.69:0/3774821304 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c810cbe0 con 0x7f59c807ccf0 2024-04-25T11:16:36.657 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.492+0000 7f59cd56a700 1 --2- 172.21.15.69:0/3774821304 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59c807ccf0 0x7f59c810c450 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f59c40098a0 tx=0x7f59c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27e68751ffe2bfee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:36.657 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.492+0000 7f59bffff700 1 -- 172.21.15.69:0/3774821304 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59c4004160 con 0x7f59c807ccf0 2024-04-25T11:16:36.657 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.492+0000 7f59bffff700 1 -- 172.21.15.69:0/3774821304 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59c4004300 con 0x7f59c807ccf0 2024-04-25T11:16:36.657 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.492+0000 7f59bffff700 1 -- 172.21.15.69:0/3774821304 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59c4011960 con 0x7f59c807ccf0 2024-04-25T11:16:36.657 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.493+0000 7f59cf7ce700 1 -- 172.21.15.69:0/3774821304 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59c807ccf0 msgr2=0x7f59c810c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:36.657 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.493+0000 7f59cf7ce700 1 --2- 172.21.15.69:0/3774821304 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59c807ccf0 0x7f59c810c450 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f59c40098a0 tx=0x7f59c4009bb0 comp rx=0 tx=0).stop 2024-04-25T11:16:36.657 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.493+0000 7f59cf7ce700 1 -- 172.21.15.69:0/3774821304 shutdown_connections 2024-04-25T11:16:36.657 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.493+0000 7f59cf7ce700 1 --2- 172.21.15.69:0/3774821304 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59c807ccf0 0x7f59c810c450 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:36.657 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.493+0000 7f59cf7ce700 1 -- 172.21.15.69:0/3774821304 >> 172.21.15.69:0/3774821304 conn(0x7f59c8103f20 msgr2=0x7f59c8106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:36.658 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.493+0000 7f59cf7ce700 1 -- 172.21.15.69:0/3774821304 shutdown_connections 2024-04-25T11:16:36.658 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.493+0000 7f59cf7ce700 1 -- 172.21.15.69:0/3774821304 wait complete. 2024-04-25T11:16:36.658 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.494+0000 7f59cf7ce700 1 Processor -- start 2024-04-25T11:16:36.658 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.494+0000 7f59cf7ce700 1 -- start start 2024-04-25T11:16:36.658 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.494+0000 7f59cf7ce700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59c8109b40 0x7f59c8108350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:36.658 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.494+0000 7f59cf7ce700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f59c810dc40 con 0x7f59c8109b40 2024-04-25T11:16:36.658 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.494+0000 7f59cd56a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59c8109b40 0x7f59c8108350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:36.658 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.494+0000 7f59cd56a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59c8109b40 0x7f59c8108350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47336/0 (socket says 172.21.15.69:47336) 2024-04-25T11:16:36.658 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.494+0000 7f59cd56a700 1 -- 172.21.15.69:0/2370743642 learned_addr learned my addr 172.21.15.69:0/2370743642 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:16:36.658 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.494+0000 7f59cd56a700 1 -- 172.21.15.69:0/2370743642 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c8109fb0 con 0x7f59c8109b40 2024-04-25T11:16:36.658 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.495+0000 7f59cd56a700 1 --2- 172.21.15.69:0/2370743642 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59c8109b40 0x7f59c8108350 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f59c40098a0 tx=0x7f59c4012f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:36.658 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.495+0000 7f59be7fc700 1 -- 172.21.15.69:0/2370743642 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59c4011900 con 0x7f59c8109b40 2024-04-25T11:16:36.658 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.495+0000 7f59be7fc700 1 -- 172.21.15.69:0/2370743642 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59c4003b30 con 0x7f59c8109b40 2024-04-25T11:16:36.658 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.495+0000 7f59be7fc700 1 -- 172.21.15.69:0/2370743642 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59c401bbf0 con 0x7f59c8109b40 2024-04-25T11:16:36.659 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.495+0000 7f59cf7ce700 1 -- 172.21.15.69:0/2370743642 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59c810a240 con 0x7f59c8109b40 2024-04-25T11:16:36.659 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.495+0000 7f59cf7ce700 1 -- 172.21.15.69:0/2370743642 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59c8108ba0 con 0x7f59c8109b40 2024-04-25T11:16:36.659 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.496+0000 7f59cf7ce700 1 -- 172.21.15.69:0/2370743642 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59c8064cf0 con 0x7f59c8109b40 2024-04-25T11:16:36.659 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.496+0000 7f59be7fc700 1 -- 172.21.15.69:0/2370743642 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 12) ==== 49529+0+0 (secure 0 0 0) 0x7f59c40036a0 con 0x7f59c8109b40 2024-04-25T11:16:36.659 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.496+0000 7f59be7fc700 1 --2- 172.21.15.69:0/2370743642 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f59b403d6e0 0x7f59b403fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:36.659 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.497+0000 7f59ccd69700 1 --2- 172.21.15.69:0/2370743642 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f59b403d6e0 0x7f59b403fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:36.659 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.497+0000 7f59be7fc700 1 -- 172.21.15.69:0/2370743642 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f59c401bd90 con 0x7f59c8109b40 2024-04-25T11:16:36.659 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.497+0000 7f59ccd69700 1 --2- 172.21.15.69:0/2370743642 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f59b403d6e0 0x7f59b403fbf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f59b8006fd0 tx=0x7f59b8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:36.659 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.504+0000 7f59be7fc700 1 -- 172.21.15.69:0/2370743642 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f59c4023820 con 0x7f59c8109b40 2024-04-25T11:16:36.659 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.726+0000 7f59cf7ce700 1 -- 172.21.15.69:0/2370743642 --> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f59c8108e20 con 0x7f59b403d6e0 2024-04-25T11:16:36.659 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.953+0000 7f59be7fc700 1 -- 172.21.15.69:0/2370743642 <== mgr.14152 v2:172.21.15.69:6800/4220065409 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f59c8108e20 con 0x7f59b403d6e0 2024-04-25T11:16:36.659 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.955+0000 7f59cf7ce700 1 -- 172.21.15.69:0/2370743642 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f59b403d6e0 msgr2=0x7f59b403fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:36.659 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.955+0000 7f59cf7ce700 1 --2- 172.21.15.69:0/2370743642 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f59b403d6e0 0x7f59b403fbf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f59b8006fd0 tx=0x7f59b8006e40 comp rx=0 tx=0).stop 2024-04-25T11:16:36.660 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.955+0000 7f59cf7ce700 1 -- 172.21.15.69:0/2370743642 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59c8109b40 msgr2=0x7f59c8108350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:36.660 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.955+0000 7f59cf7ce700 1 --2- 172.21.15.69:0/2370743642 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59c8109b40 0x7f59c8108350 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f59c40098a0 tx=0x7f59c4012f00 comp rx=0 tx=0).stop 2024-04-25T11:16:36.660 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.955+0000 7f59cf7ce700 1 -- 172.21.15.69:0/2370743642 shutdown_connections 2024-04-25T11:16:36.660 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.955+0000 7f59cf7ce700 1 --2- 172.21.15.69:0/2370743642 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f59b403d6e0 0x7f59b403fbf0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:36.660 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.955+0000 7f59cf7ce700 1 --2- 172.21.15.69:0/2370743642 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59c8109b40 0x7f59c8108350 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:36.660 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.955+0000 7f59cf7ce700 1 -- 172.21.15.69:0/2370743642 >> 172.21.15.69:0/2370743642 conn(0x7f59c8103f20 msgr2=0x7f59c8105990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:36.660 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.955+0000 7f59cf7ce700 1 -- 172.21.15.69:0/2370743642 shutdown_connections 2024-04-25T11:16:36.660 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:35.955+0000 7f59cf7ce700 1 -- 172.21.15.69:0/2370743642 wait complete. 2024-04-25T11:16:36.660 INFO:teuthology.orchestra.run.smithi069.stdout:Fetching dashboard port number... 2024-04-25T11:16:36.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:36 smithi069 ceph-mon[29188]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:16:36.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:36 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:16:38.039 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:37 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/3960725515' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-25T11:16:38.331 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 8443 2024-04-25T11:16:38.331 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.513+0000 7f00a1619700 1 Processor -- start 2024-04-25T11:16:38.331 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.513+0000 7f00a1619700 1 -- start start 2024-04-25T11:16:38.331 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.513+0000 7f00a1619700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f009c1087e0 0x7f009c108bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:38.331 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.513+0000 7f00a1619700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f009c1091e0 con 0x7f009c1087e0 2024-04-25T11:16:38.331 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.514+0000 7f009affd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f009c1087e0 0x7f009c108bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:38.331 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.514+0000 7f009affd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f009c1087e0 0x7f009c108bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47348/0 (socket says 172.21.15.69:47348) 2024-04-25T11:16:38.332 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.514+0000 7f009affd700 1 -- 172.21.15.69:0/3309545905 learned_addr learned my addr 172.21.15.69:0/3309545905 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:16:38.332 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.514+0000 7f009affd700 1 -- 172.21.15.69:0/3309545905 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f009c109910 con 0x7f009c1087e0 2024-04-25T11:16:38.332 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.514+0000 7f009affd700 1 --2- 172.21.15.69:0/3309545905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f009c1087e0 0x7f009c108bf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f008c009e80 tx=0x7f008c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37f57c94cbb6fec5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:38.332 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.514+0000 7f009a7fc700 1 -- 172.21.15.69:0/3309545905 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f008c017070 con 0x7f009c1087e0 2024-04-25T11:16:38.332 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.514+0000 7f009a7fc700 1 -- 172.21.15.69:0/3309545905 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f008c00b8c0 con 0x7f009c1087e0 2024-04-25T11:16:38.332 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.514+0000 7f009a7fc700 1 -- 172.21.15.69:0/3309545905 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f008c004420 con 0x7f009c1087e0 2024-04-25T11:16:38.332 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.515+0000 7f00a1619700 1 -- 172.21.15.69:0/3309545905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f009c1087e0 msgr2=0x7f009c108bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:38.332 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.515+0000 7f00a1619700 1 --2- 172.21.15.69:0/3309545905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f009c1087e0 0x7f009c108bf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f008c009e80 tx=0x7f008c00b330 comp rx=0 tx=0).stop 2024-04-25T11:16:38.332 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.515+0000 7f00a1619700 1 -- 172.21.15.69:0/3309545905 shutdown_connections 2024-04-25T11:16:38.332 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.515+0000 7f00a1619700 1 --2- 172.21.15.69:0/3309545905 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f009c1087e0 0x7f009c108bf0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:38.332 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.515+0000 7f00a1619700 1 -- 172.21.15.69:0/3309545905 >> 172.21.15.69:0/3309545905 conn(0x7f009c103ee0 msgr2=0x7f009c106330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:38.332 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.515+0000 7f00a1619700 1 -- 172.21.15.69:0/3309545905 shutdown_connections 2024-04-25T11:16:38.332 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.515+0000 7f00a1619700 1 -- 172.21.15.69:0/3309545905 wait complete. 2024-04-25T11:16:38.332 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.516+0000 7f00a1619700 1 Processor -- start 2024-04-25T11:16:38.333 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.516+0000 7f00a1619700 1 -- start start 2024-04-25T11:16:38.333 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.516+0000 7f00a1619700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f009c1087e0 0x7f009c19cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:38.333 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.516+0000 7f00a1619700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f009c10ccc0 con 0x7f009c1087e0 2024-04-25T11:16:38.333 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.516+0000 7f009affd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f009c1087e0 0x7f009c19cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:38.333 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.516+0000 7f009affd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f009c1087e0 0x7f009c19cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47364/0 (socket says 172.21.15.69:47364) 2024-04-25T11:16:38.333 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.516+0000 7f009affd700 1 -- 172.21.15.69:0/3960725515 learned_addr learned my addr 172.21.15.69:0/3960725515 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:16:38.333 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.516+0000 7f009affd700 1 -- 172.21.15.69:0/3960725515 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f009c19d020 con 0x7f009c1087e0 2024-04-25T11:16:38.333 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.517+0000 7f009affd700 1 --2- 172.21.15.69:0/3960725515 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f009c1087e0 0x7f009c19cae0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f008c004070 tx=0x7f008c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:38.333 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.517+0000 7f0098ff9700 1 -- 172.21.15.69:0/3960725515 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f008c017040 con 0x7f009c1087e0 2024-04-25T11:16:38.333 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.517+0000 7f0098ff9700 1 -- 172.21.15.69:0/3960725515 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f008c022070 con 0x7f009c1087e0 2024-04-25T11:16:38.333 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.517+0000 7f0098ff9700 1 -- 172.21.15.69:0/3960725515 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f008c0125c0 con 0x7f009c1087e0 2024-04-25T11:16:38.333 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.517+0000 7f00a1619700 1 -- 172.21.15.69:0/3960725515 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f009c19d2b0 con 0x7f009c1087e0 2024-04-25T11:16:38.333 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.517+0000 7f00a1619700 1 -- 172.21.15.69:0/3960725515 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f009c19fec0 con 0x7f009c1087e0 2024-04-25T11:16:38.333 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.518+0000 7f0098ff9700 1 -- 172.21.15.69:0/3960725515 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 12) ==== 49529+0+0 (secure 0 0 0) 0x7f008c0127f0 con 0x7f009c1087e0 2024-04-25T11:16:38.333 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.518+0000 7f00a1619700 1 -- 172.21.15.69:0/3960725515 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f009c064cf0 con 0x7f009c1087e0 2024-04-25T11:16:38.334 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.518+0000 7f0098ff9700 1 --2- 172.21.15.69:0/3960725515 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f008803d690 0x7f008803fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:38.334 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.518+0000 7f0098ff9700 1 -- 172.21.15.69:0/3960725515 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f008c00f050 con 0x7f009c1087e0 2024-04-25T11:16:38.334 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.518+0000 7f00927fc700 1 --2- 172.21.15.69:0/3960725515 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f008803d690 0x7f008803fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:38.334 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.518+0000 7f00927fc700 1 --2- 172.21.15.69:0/3960725515 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f008803d690 0x7f008803fba0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0084006fd0 tx=0x7f0084006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:38.334 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.523+0000 7f0098ff9700 1 -- 172.21.15.69:0/3960725515 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f008c02d250 con 0x7f009c1087e0 2024-04-25T11:16:38.334 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.726+0000 7f00a1619700 1 -- 172.21.15.69:0/3960725515 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f009c1a0090 con 0x7f009c1087e0 2024-04-25T11:16:38.334 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.726+0000 7f0098ff9700 1 -- 172.21.15.69:0/3960725515 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f008c003d90 con 0x7f009c1087e0 2024-04-25T11:16:38.334 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.728+0000 7f00a1619700 1 -- 172.21.15.69:0/3960725515 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f008803d690 msgr2=0x7f008803fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:38.334 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.728+0000 7f00a1619700 1 --2- 172.21.15.69:0/3960725515 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f008803d690 0x7f008803fba0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0084006fd0 tx=0x7f0084006e40 comp rx=0 tx=0).stop 2024-04-25T11:16:38.334 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.728+0000 7f00a1619700 1 -- 172.21.15.69:0/3960725515 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f009c1087e0 msgr2=0x7f009c19cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:38.334 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.728+0000 7f00a1619700 1 --2- 172.21.15.69:0/3960725515 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f009c1087e0 0x7f009c19cae0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f008c004070 tx=0x7f008c00bd50 comp rx=0 tx=0).stop 2024-04-25T11:16:38.334 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.728+0000 7f00a1619700 1 -- 172.21.15.69:0/3960725515 shutdown_connections 2024-04-25T11:16:38.335 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.728+0000 7f00a1619700 1 --2- 172.21.15.69:0/3960725515 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f008803d690 0x7f008803fba0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:38.335 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.728+0000 7f00a1619700 1 --2- 172.21.15.69:0/3960725515 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f009c1087e0 0x7f009c19cae0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:38.335 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.728+0000 7f00a1619700 1 -- 172.21.15.69:0/3960725515 >> 172.21.15.69:0/3960725515 conn(0x7f009c103ee0 msgr2=0x7f009c1058c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:38.335 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.729+0000 7f00a1619700 1 -- 172.21.15.69:0/3960725515 shutdown_connections 2024-04-25T11:16:38.335 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:37.729+0000 7f00a1619700 1 -- 172.21.15.69:0/3960725515 wait complete. 2024-04-25T11:16:38.337 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T11:16:38.337 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout disabled 2024-04-25T11:16:38.344 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T11:16:38.344 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout inactive 2024-04-25T11:16:38.344 INFO:teuthology.orchestra.run.smithi069.stdout:firewalld.service is not enabled 2024-04-25T11:16:38.344 INFO:teuthology.orchestra.run.smithi069.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-25T11:16:38.346 INFO:teuthology.orchestra.run.smithi069.stdout:Ceph Dashboard is now available at: 2024-04-25T11:16:38.346 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:16:38.346 INFO:teuthology.orchestra.run.smithi069.stdout: URL: https://smithi069:8443/ 2024-04-25T11:16:38.346 INFO:teuthology.orchestra.run.smithi069.stdout: User: admin 2024-04-25T11:16:38.346 INFO:teuthology.orchestra.run.smithi069.stdout: Password: ay0tqby0gt 2024-04-25T11:16:38.346 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:16:38.347 INFO:teuthology.orchestra.run.smithi069.stdout:Saving cluster configuration to /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/config directory 2024-04-25T11:16:38.347 INFO:teuthology.orchestra.run.smithi069.stdout:Enabling autotune for osd_memory_target 2024-04-25T11:16:38.949 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:38 smithi069 ceph-mon[29188]: mgrmap e13: x(active, since 6s) 2024-04-25T11:16:39.654 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:38.865+0000 7f3eba448700 1 Processor -- start 2024-04-25T11:16:39.654 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:38.865+0000 7f3eba448700 1 -- start start 2024-04-25T11:16:39.654 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:38.866+0000 7f3eba448700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3eb4108820 0x7f3eb4108c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:39.654 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:38.866+0000 7f3eba448700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb4109220 con 0x7f3eb4108820 2024-04-25T11:16:39.654 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:38.866+0000 7f3eb3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3eb4108820 0x7f3eb4108c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:39.654 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:38.866+0000 7f3eb3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3eb4108820 0x7f3eb4108c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47380/0 (socket says 172.21.15.69:47380) 2024-04-25T11:16:39.654 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:38.866+0000 7f3eb3fff700 1 -- 172.21.15.69:0/3428159688 learned_addr learned my addr 172.21.15.69:0/3428159688 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:16:39.654 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:38.866+0000 7f3eb3fff700 1 -- 172.21.15.69:0/3428159688 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb4109950 con 0x7f3eb4108820 2024-04-25T11:16:39.654 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:38.866+0000 7f3eb3fff700 1 --2- 172.21.15.69:0/3428159688 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3eb4108820 0x7f3eb4108c30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3ea40098a0 tx=0x7f3ea4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8f8841d23751a8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:39.654 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:38.866+0000 7f3eb2ffd700 1 -- 172.21.15.69:0/3428159688 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ea4004160 con 0x7f3eb4108820 2024-04-25T11:16:39.655 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:38.867+0000 7f3eb2ffd700 1 -- 172.21.15.69:0/3428159688 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ea4004300 con 0x7f3eb4108820 2024-04-25T11:16:39.655 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:38.867+0000 7f3eb2ffd700 1 -- 172.21.15.69:0/3428159688 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ea4011960 con 0x7f3eb4108820 2024-04-25T11:16:39.655 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:38.867+0000 7f3eba448700 1 -- 172.21.15.69:0/3428159688 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3eb4108820 msgr2=0x7f3eb4108c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:39.655 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:38.867+0000 7f3eba448700 1 --2- 172.21.15.69:0/3428159688 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3eb4108820 0x7f3eb4108c30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3ea40098a0 tx=0x7f3ea4009bb0 comp rx=0 tx=0).stop 2024-04-25T11:16:39.655 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:38.867+0000 7f3eba448700 1 -- 172.21.15.69:0/3428159688 shutdown_connections 2024-04-25T11:16:39.655 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:38.867+0000 7f3eba448700 1 --2- 172.21.15.69:0/3428159688 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3eb4108820 0x7f3eb4108c30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:39.655 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:38.867+0000 7f3eba448700 1 -- 172.21.15.69:0/3428159688 >> 172.21.15.69:0/3428159688 conn(0x7f3eb4103f20 msgr2=0x7f3eb4106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:39.655 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:38.867+0000 7f3eba448700 1 -- 172.21.15.69:0/3428159688 shutdown_connections 2024-04-25T11:16:39.655 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:38.867+0000 7f3eba448700 1 -- 172.21.15.69:0/3428159688 wait complete. 2024-04-25T11:16:39.655 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:38.868+0000 7f3eba448700 1 Processor -- start 2024-04-25T11:16:39.655 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:38.868+0000 7f3eba448700 1 -- start start 2024-04-25T11:16:39.655 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:38.868+0000 7f3eba448700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3eb4108820 0x7f3eb419cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:39.655 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:38.868+0000 7f3eba448700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb410cd00 con 0x7f3eb4108820 2024-04-25T11:16:39.655 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:38.869+0000 7f3eb3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3eb4108820 0x7f3eb419cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:39.655 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:38.869+0000 7f3eb3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3eb4108820 0x7f3eb419cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47392/0 (socket says 172.21.15.69:47392) 2024-04-25T11:16:39.656 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:38.869+0000 7f3eb3fff700 1 -- 172.21.15.69:0/542143019 learned_addr learned my addr 172.21.15.69:0/542143019 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:16:39.656 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:38.869+0000 7f3eb3fff700 1 -- 172.21.15.69:0/542143019 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb419d000 con 0x7f3eb4108820 2024-04-25T11:16:39.656 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:38.869+0000 7f3eb3fff700 1 --2- 172.21.15.69:0/542143019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3eb4108820 0x7f3eb419cac0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3ea40119b0 tx=0x7f3ea4011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:39.656 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:38.869+0000 7f3eb17fa700 1 -- 172.21.15.69:0/542143019 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ea4003aa0 con 0x7f3eb4108820 2024-04-25T11:16:39.656 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:38.869+0000 7f3eb17fa700 1 -- 172.21.15.69:0/542143019 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ea401b8d0 con 0x7f3eb4108820 2024-04-25T11:16:39.656 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:38.869+0000 7f3eba448700 1 -- 172.21.15.69:0/542143019 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eb419d290 con 0x7f3eb4108820 2024-04-25T11:16:39.656 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:38.869+0000 7f3eb17fa700 1 -- 172.21.15.69:0/542143019 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ea4023860 con 0x7f3eb4108820 2024-04-25T11:16:39.656 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:38.869+0000 7f3eba448700 1 -- 172.21.15.69:0/542143019 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eb419fea0 con 0x7f3eb4108820 2024-04-25T11:16:39.656 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:38.870+0000 7f3eb17fa700 1 -- 172.21.15.69:0/542143019 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f3ea402c3f0 con 0x7f3eb4108820 2024-04-25T11:16:39.656 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:38.870+0000 7f3eba448700 1 -- 172.21.15.69:0/542143019 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3eb4064cf0 con 0x7f3eb4108820 2024-04-25T11:16:39.656 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:38.870+0000 7f3eb17fa700 1 --2- 172.21.15.69:0/542143019 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f3e9c03d730 0x7f3e9c03fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:39.656 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:38.870+0000 7f3eb17fa700 1 -- 172.21.15.69:0/542143019 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3ea4052e80 con 0x7f3eb4108820 2024-04-25T11:16:39.656 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:38.870+0000 7f3eb37fe700 1 --2- 172.21.15.69:0/542143019 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f3e9c03d730 0x7f3e9c03fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:39.656 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:38.871+0000 7f3eb37fe700 1 --2- 172.21.15.69:0/542143019 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f3e9c03d730 0x7f3e9c03fc40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8006fd0 tx=0x7f3ea8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:39.657 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:38.876+0000 7f3eb17fa700 1 -- 172.21.15.69:0/542143019 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3ea401f030 con 0x7f3eb4108820 2024-04-25T11:16:39.657 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:39.073+0000 7f3eba448700 1 -- 172.21.15.69:0/542143019 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f3eb41a0070 con 0x7f3eb4108820 2024-04-25T11:16:39.657 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:39.073+0000 7f3eb17fa700 1 -- 172.21.15.69:0/542143019 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f3eb41a0070 con 0x7f3eb4108820 2024-04-25T11:16:39.657 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:39.075+0000 7f3eba448700 1 -- 172.21.15.69:0/542143019 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f3e9c03d730 msgr2=0x7f3e9c03fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:39.657 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:39.075+0000 7f3eba448700 1 --2- 172.21.15.69:0/542143019 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f3e9c03d730 0x7f3e9c03fc40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8006fd0 tx=0x7f3ea8006e40 comp rx=0 tx=0).stop 2024-04-25T11:16:39.657 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:39.075+0000 7f3eba448700 1 -- 172.21.15.69:0/542143019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3eb4108820 msgr2=0x7f3eb419cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:39.657 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:39.075+0000 7f3eba448700 1 --2- 172.21.15.69:0/542143019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3eb4108820 0x7f3eb419cac0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3ea40119b0 tx=0x7f3ea4011a90 comp rx=0 tx=0).stop 2024-04-25T11:16:39.657 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:39.075+0000 7f3eba448700 1 -- 172.21.15.69:0/542143019 shutdown_connections 2024-04-25T11:16:39.657 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:39.075+0000 7f3eba448700 1 --2- 172.21.15.69:0/542143019 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f3e9c03d730 0x7f3e9c03fc40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:39.657 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:39.075+0000 7f3eba448700 1 --2- 172.21.15.69:0/542143019 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3eb4108820 0x7f3eb419cac0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:39.657 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:39.075+0000 7f3eba448700 1 -- 172.21.15.69:0/542143019 >> 172.21.15.69:0/542143019 conn(0x7f3eb4103f20 msgr2=0x7f3eb4105880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:39.657 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:39.075+0000 7f3eba448700 1 -- 172.21.15.69:0/542143019 shutdown_connections 2024-04-25T11:16:39.657 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:39.076+0000 7f3eba448700 1 -- 172.21.15.69:0/542143019 wait complete. 2024-04-25T11:16:40.985 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.105+0000 7f246a43b700 1 Processor -- start 2024-04-25T11:16:40.985 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.105+0000 7f246a43b700 1 -- start start 2024-04-25T11:16:40.985 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.105+0000 7f246a43b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f246407d580 0x7f246410c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:40.985 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.105+0000 7f246a43b700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f246410cae0 con 0x7f246407d580 2024-04-25T11:16:40.985 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.105+0000 7f2463fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f246407d580 0x7f246410c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:40.986 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.105+0000 7f2463fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f246407d580 0x7f246410c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47398/0 (socket says 172.21.15.69:47398) 2024-04-25T11:16:40.986 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.105+0000 7f2463fff700 1 -- 172.21.15.69:0/1272823617 learned_addr learned my addr 172.21.15.69:0/1272823617 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:16:40.986 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.106+0000 7f2463fff700 1 -- 172.21.15.69:0/1272823617 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f246410cc60 con 0x7f246407d580 2024-04-25T11:16:40.986 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.106+0000 7f2463fff700 1 --2- 172.21.15.69:0/1272823617 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f246407d580 0x7f246410c4d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f24540098a0 tx=0x7f2454009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f5d9171f2a38f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:40.986 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.106+0000 7f2462ffd700 1 -- 172.21.15.69:0/1272823617 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2454004160 con 0x7f246407d580 2024-04-25T11:16:40.986 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.106+0000 7f2462ffd700 1 -- 172.21.15.69:0/1272823617 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2454004300 con 0x7f246407d580 2024-04-25T11:16:40.986 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.106+0000 7f2462ffd700 1 -- 172.21.15.69:0/1272823617 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2454011960 con 0x7f246407d580 2024-04-25T11:16:40.986 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.107+0000 7f246a43b700 1 -- 172.21.15.69:0/1272823617 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f246407d580 msgr2=0x7f246410c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:40.986 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.107+0000 7f246a43b700 1 --2- 172.21.15.69:0/1272823617 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f246407d580 0x7f246410c4d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f24540098a0 tx=0x7f2454009bb0 comp rx=0 tx=0).stop 2024-04-25T11:16:40.986 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.107+0000 7f246a43b700 1 -- 172.21.15.69:0/1272823617 shutdown_connections 2024-04-25T11:16:40.986 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.107+0000 7f246a43b700 1 --2- 172.21.15.69:0/1272823617 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f246407d580 0x7f246410c4d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:40.987 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.107+0000 7f246a43b700 1 -- 172.21.15.69:0/1272823617 >> 172.21.15.69:0/1272823617 conn(0x7f2464103f40 msgr2=0x7f2464106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:40.987 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.107+0000 7f246a43b700 1 -- 172.21.15.69:0/1272823617 shutdown_connections 2024-04-25T11:16:40.987 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.107+0000 7f246a43b700 1 -- 172.21.15.69:0/1272823617 wait complete. 2024-04-25T11:16:40.987 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.107+0000 7f246a43b700 1 Processor -- start 2024-04-25T11:16:40.987 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.107+0000 7f246a43b700 1 -- start start 2024-04-25T11:16:40.987 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.108+0000 7f246a43b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f246407d580 0x7f246419db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:40.987 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.108+0000 7f246a43b700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f246410dcc0 con 0x7f246407d580 2024-04-25T11:16:40.987 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.108+0000 7f2463fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f246407d580 0x7f246419db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:40.987 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.108+0000 7f2463fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f246407d580 0x7f246419db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47402/0 (socket says 172.21.15.69:47402) 2024-04-25T11:16:40.987 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.108+0000 7f2463fff700 1 -- 172.21.15.69:0/1475537388 learned_addr learned my addr 172.21.15.69:0/1475537388 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:16:40.987 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.108+0000 7f2463fff700 1 -- 172.21.15.69:0/1475537388 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f246419e0a0 con 0x7f246407d580 2024-04-25T11:16:40.987 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.108+0000 7f2463fff700 1 --2- 172.21.15.69:0/1475537388 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f246407d580 0x7f246419db60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2454011670 tx=0x7f2454011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:40.987 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.108+0000 7f24617fa700 1 -- 172.21.15.69:0/1475537388 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2454003a60 con 0x7f246407d580 2024-04-25T11:16:40.987 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.109+0000 7f24617fa700 1 -- 172.21.15.69:0/1475537388 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f245401b8d0 con 0x7f246407d580 2024-04-25T11:16:40.988 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.109+0000 7f246a43b700 1 -- 172.21.15.69:0/1475537388 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f246419e330 con 0x7f246407d580 2024-04-25T11:16:40.988 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.109+0000 7f24617fa700 1 -- 172.21.15.69:0/1475537388 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2454023750 con 0x7f246407d580 2024-04-25T11:16:40.988 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.109+0000 7f246a43b700 1 -- 172.21.15.69:0/1475537388 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24641a0f40 con 0x7f246407d580 2024-04-25T11:16:40.988 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.109+0000 7f246a43b700 1 -- 172.21.15.69:0/1475537388 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2464064cf0 con 0x7f246407d580 2024-04-25T11:16:40.988 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.110+0000 7f24617fa700 1 -- 172.21.15.69:0/1475537388 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f2454023a00 con 0x7f246407d580 2024-04-25T11:16:40.988 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.110+0000 7f24617fa700 1 --2- 172.21.15.69:0/1475537388 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f244c03d730 0x7f244c03fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:40.988 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.110+0000 7f24617fa700 1 -- 172.21.15.69:0/1475537388 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2454052fc0 con 0x7f246407d580 2024-04-25T11:16:40.988 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.110+0000 7f24637fe700 1 --2- 172.21.15.69:0/1475537388 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f244c03d730 0x7f244c03fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:40.988 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.110+0000 7f24637fe700 1 --2- 172.21.15.69:0/1475537388 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f244c03d730 0x7f244c03fc40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2458006fd0 tx=0x7f2458006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:40.988 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.115+0000 7f24617fa700 1 -- 172.21.15.69:0/1475537388 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f245401f030 con 0x7f246407d580 2024-04-25T11:16:40.988 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.421+0000 7f246a43b700 1 -- 172.21.15.69:0/1475537388 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f24641a1110 con 0x7f246407d580 2024-04-25T11:16:40.988 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.424+0000 7f24617fa700 1 -- 172.21.15.69:0/1475537388 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f245400b3e0 con 0x7f246407d580 2024-04-25T11:16:40.988 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.427+0000 7f246a43b700 1 -- 172.21.15.69:0/1475537388 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f244c03d730 msgr2=0x7f244c03fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:40.988 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.427+0000 7f246a43b700 1 --2- 172.21.15.69:0/1475537388 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f244c03d730 0x7f244c03fc40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2458006fd0 tx=0x7f2458006e40 comp rx=0 tx=0).stop 2024-04-25T11:16:40.989 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.427+0000 7f246a43b700 1 -- 172.21.15.69:0/1475537388 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f246407d580 msgr2=0x7f246419db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:40.989 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.427+0000 7f246a43b700 1 --2- 172.21.15.69:0/1475537388 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f246407d580 0x7f246419db60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2454011670 tx=0x7f2454011a30 comp rx=0 tx=0).stop 2024-04-25T11:16:40.989 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.427+0000 7f246a43b700 1 -- 172.21.15.69:0/1475537388 shutdown_connections 2024-04-25T11:16:40.989 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.427+0000 7f246a43b700 1 --2- 172.21.15.69:0/1475537388 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f244c03d730 0x7f244c03fc40 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:40.989 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.427+0000 7f246a43b700 1 --2- 172.21.15.69:0/1475537388 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f246407d580 0x7f246419db60 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:40.989 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.427+0000 7f246a43b700 1 -- 172.21.15.69:0/1475537388 >> 172.21.15.69:0/1475537388 conn(0x7f2464103f40 msgr2=0x7f24641046c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:40.989 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.427+0000 7f246a43b700 1 -- 172.21.15.69:0/1475537388 shutdown_connections 2024-04-25T11:16:40.989 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-04-25T11:16:40.427+0000 7f246a43b700 1 -- 172.21.15.69:0/1475537388 wait complete. 2024-04-25T11:16:40.989 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-25T11:16:40.989 INFO:teuthology.orchestra.run.smithi069.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-25T11:16:40.989 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:16:40.989 INFO:teuthology.orchestra.run.smithi069.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-25T11:16:40.989 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:16:40.990 INFO:teuthology.orchestra.run.smithi069.stdout:Or, if you are only running a single cluster on this host: 2024-04-25T11:16:40.990 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:16:40.990 INFO:teuthology.orchestra.run.smithi069.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-25T11:16:40.990 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:16:40.990 INFO:teuthology.orchestra.run.smithi069.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-25T11:16:40.990 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:16:40.990 INFO:teuthology.orchestra.run.smithi069.stdout: ceph telemetry on 2024-04-25T11:16:40.990 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:16:40.990 INFO:teuthology.orchestra.run.smithi069.stdout:For more information see: 2024-04-25T11:16:40.990 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:16:40.990 INFO:teuthology.orchestra.run.smithi069.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-25T11:16:40.990 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:16:40.990 INFO:teuthology.orchestra.run.smithi069.stdout:Bootstrap complete. 2024-04-25T11:16:41.022 INFO:tasks.cephadm:Fetching config... 2024-04-25T11:16:41.022 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T11:16:41.022 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-25T11:16:41.038 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-25T11:16:41.038 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T11:16:41.038 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-25T11:16:41.093 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-25T11:16:41.093 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T11:16:41.093 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-25T11:16:41.157 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-25T11:16:41.157 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T11:16:41.157 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-25T11:16:41.213 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-25T11:16:41.213 DEBUG:teuthology.orchestra.run.smithi069:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCgJuPV2zxWIMKKqp4AJzUGITwFGTOK0sf/5Co4CWi/CbQOkM3oIl4igo3w/hWgAwHHyUVgSQEAoBMhDeLfLOZM9RSGL1Jiq1oFjdWv0hCGhSDQa6j3obZp5sk9HwNRz0HrQLxu/KAjZSddFt/NDfi13cSLBVPLrZIVq7Ep0DARQkxBnlhRlezB3OaIRVRPOiOiwjzeXRBja85nnwbt8QQf9P7imPMusSd8ef0fSDlFJJ8tc/SWO1/ZhsGMNnCdHAcOuhDd0Bu9Je+UJjEcFzOwxTPE8Bf/8l+Sw/FXt/bHsspDjDAIRzyzNpqaVyLYErHTfHa8wwGKtofmMuFS6n0FGuAh5wrnFkTS6oQhC+2cZkhjlbMIwcnyBel1LfyUpkZ1zJCDvRIRM1XGj5RFUDxfkt2sce6D3kBt4VX43EthJ5BHQQ3PsNchDk7UVlyuEzgOZqTfCVrRdWwsY+knUVRZbLDi+m/QRn0KmRV9KICLQsv8Of1PnBPcmXj8TRBVMuU= ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T11:16:41.285 INFO:teuthology.orchestra.run.smithi069.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCgJuPV2zxWIMKKqp4AJzUGITwFGTOK0sf/5Co4CWi/CbQOkM3oIl4igo3w/hWgAwHHyUVgSQEAoBMhDeLfLOZM9RSGL1Jiq1oFjdWv0hCGhSDQa6j3obZp5sk9HwNRz0HrQLxu/KAjZSddFt/NDfi13cSLBVPLrZIVq7Ep0DARQkxBnlhRlezB3OaIRVRPOiOiwjzeXRBja85nnwbt8QQf9P7imPMusSd8ef0fSDlFJJ8tc/SWO1/ZhsGMNnCdHAcOuhDd0Bu9Je+UJjEcFzOwxTPE8Bf/8l+Sw/FXt/bHsspDjDAIRzyzNpqaVyLYErHTfHa8wwGKtofmMuFS6n0FGuAh5wrnFkTS6oQhC+2cZkhjlbMIwcnyBel1LfyUpkZ1zJCDvRIRM1XGj5RFUDxfkt2sce6D3kBt4VX43EthJ5BHQQ3PsNchDk7UVlyuEzgOZqTfCVrRdWwsY+knUVRZbLDi+m/QRn0KmRV9KICLQsv8Of1PnBPcmXj8TRBVMuU= ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968 2024-04-25T11:16:41.296 DEBUG:teuthology.orchestra.run.smithi146:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCgJuPV2zxWIMKKqp4AJzUGITwFGTOK0sf/5Co4CWi/CbQOkM3oIl4igo3w/hWgAwHHyUVgSQEAoBMhDeLfLOZM9RSGL1Jiq1oFjdWv0hCGhSDQa6j3obZp5sk9HwNRz0HrQLxu/KAjZSddFt/NDfi13cSLBVPLrZIVq7Ep0DARQkxBnlhRlezB3OaIRVRPOiOiwjzeXRBja85nnwbt8QQf9P7imPMusSd8ef0fSDlFJJ8tc/SWO1/ZhsGMNnCdHAcOuhDd0Bu9Je+UJjEcFzOwxTPE8Bf/8l+Sw/FXt/bHsspDjDAIRzyzNpqaVyLYErHTfHa8wwGKtofmMuFS6n0FGuAh5wrnFkTS6oQhC+2cZkhjlbMIwcnyBel1LfyUpkZ1zJCDvRIRM1XGj5RFUDxfkt2sce6D3kBt4VX43EthJ5BHQQ3PsNchDk7UVlyuEzgOZqTfCVrRdWwsY+knUVRZbLDi+m/QRn0KmRV9KICLQsv8Of1PnBPcmXj8TRBVMuU= ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T11:16:41.335 INFO:teuthology.orchestra.run.smithi146.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCgJuPV2zxWIMKKqp4AJzUGITwFGTOK0sf/5Co4CWi/CbQOkM3oIl4igo3w/hWgAwHHyUVgSQEAoBMhDeLfLOZM9RSGL1Jiq1oFjdWv0hCGhSDQa6j3obZp5sk9HwNRz0HrQLxu/KAjZSddFt/NDfi13cSLBVPLrZIVq7Ep0DARQkxBnlhRlezB3OaIRVRPOiOiwjzeXRBja85nnwbt8QQf9P7imPMusSd8ef0fSDlFJJ8tc/SWO1/ZhsGMNnCdHAcOuhDd0Bu9Je+UJjEcFzOwxTPE8Bf/8l+Sw/FXt/bHsspDjDAIRzyzNpqaVyLYErHTfHa8wwGKtofmMuFS6n0FGuAh5wrnFkTS6oQhC+2cZkhjlbMIwcnyBel1LfyUpkZ1zJCDvRIRM1XGj5RFUDxfkt2sce6D3kBt4VX43EthJ5BHQQ3PsNchDk7UVlyuEzgOZqTfCVrRdWwsY+knUVRZbLDi+m/QRn0KmRV9KICLQsv8Of1PnBPcmXj8TRBVMuU= ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968 2024-04-25T11:16:41.347 DEBUG:teuthology.orchestra.run.smithi152:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCgJuPV2zxWIMKKqp4AJzUGITwFGTOK0sf/5Co4CWi/CbQOkM3oIl4igo3w/hWgAwHHyUVgSQEAoBMhDeLfLOZM9RSGL1Jiq1oFjdWv0hCGhSDQa6j3obZp5sk9HwNRz0HrQLxu/KAjZSddFt/NDfi13cSLBVPLrZIVq7Ep0DARQkxBnlhRlezB3OaIRVRPOiOiwjzeXRBja85nnwbt8QQf9P7imPMusSd8ef0fSDlFJJ8tc/SWO1/ZhsGMNnCdHAcOuhDd0Bu9Je+UJjEcFzOwxTPE8Bf/8l+Sw/FXt/bHsspDjDAIRzyzNpqaVyLYErHTfHa8wwGKtofmMuFS6n0FGuAh5wrnFkTS6oQhC+2cZkhjlbMIwcnyBel1LfyUpkZ1zJCDvRIRM1XGj5RFUDxfkt2sce6D3kBt4VX43EthJ5BHQQ3PsNchDk7UVlyuEzgOZqTfCVrRdWwsY+knUVRZbLDi+m/QRn0KmRV9KICLQsv8Of1PnBPcmXj8TRBVMuU= ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T11:16:41.386 INFO:teuthology.orchestra.run.smithi152.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCgJuPV2zxWIMKKqp4AJzUGITwFGTOK0sf/5Co4CWi/CbQOkM3oIl4igo3w/hWgAwHHyUVgSQEAoBMhDeLfLOZM9RSGL1Jiq1oFjdWv0hCGhSDQa6j3obZp5sk9HwNRz0HrQLxu/KAjZSddFt/NDfi13cSLBVPLrZIVq7Ep0DARQkxBnlhRlezB3OaIRVRPOiOiwjzeXRBja85nnwbt8QQf9P7imPMusSd8ef0fSDlFJJ8tc/SWO1/ZhsGMNnCdHAcOuhDd0Bu9Je+UJjEcFzOwxTPE8Bf/8l+Sw/FXt/bHsspDjDAIRzyzNpqaVyLYErHTfHa8wwGKtofmMuFS6n0FGuAh5wrnFkTS6oQhC+2cZkhjlbMIwcnyBel1LfyUpkZ1zJCDvRIRM1XGj5RFUDxfkt2sce6D3kBt4VX43EthJ5BHQQ3PsNchDk7UVlyuEzgOZqTfCVrRdWwsY+knUVRZbLDi+m/QRn0KmRV9KICLQsv8Of1PnBPcmXj8TRBVMuU= ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968 2024-04-25T11:16:41.397 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-25T11:16:41.635 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:16:41.700 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:41 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/1475537388' entity='client.admin' 2024-04-25T11:16:44.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:44.137+0000 7fbf3e10a700 1 -- 172.21.15.69:0/2535722283 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf381045c0 msgr2=0x7fbf381069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:44.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:44.137+0000 7fbf3e10a700 1 --2- 172.21.15.69:0/2535722283 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf381045c0 0x7fbf381069e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbf2800b0a0 tx=0x7fbf2800b3b0 comp rx=0 tx=0).stop 2024-04-25T11:16:44.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:44.138+0000 7fbf3e10a700 1 -- 172.21.15.69:0/2535722283 shutdown_connections 2024-04-25T11:16:44.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:44.138+0000 7fbf3e10a700 1 --2- 172.21.15.69:0/2535722283 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf381045c0 0x7fbf381069e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:44.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:44.138+0000 7fbf3e10a700 1 -- 172.21.15.69:0/2535722283 >> 172.21.15.69:0/2535722283 conn(0x7fbf380fdfe0 msgr2=0x7fbf381004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:44.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:44.138+0000 7fbf3e10a700 1 -- 172.21.15.69:0/2535722283 shutdown_connections 2024-04-25T11:16:44.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:44.138+0000 7fbf3e10a700 1 -- 172.21.15.69:0/2535722283 wait complete. 2024-04-25T11:16:44.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:44.138+0000 7fbf3e10a700 1 Processor -- start 2024-04-25T11:16:44.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:44.138+0000 7fbf3e10a700 1 -- start start 2024-04-25T11:16:44.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:44.139+0000 7fbf3e10a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf381045c0 0x7fbf381994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:44.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:44.139+0000 7fbf3e10a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbf38109930 con 0x7fbf381045c0 2024-04-25T11:16:44.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:44.139+0000 7fbf377fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf381045c0 0x7fbf381994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:44.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:44.139+0000 7fbf377fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf381045c0 0x7fbf381994b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54792/0 (socket says 172.21.15.69:54792) 2024-04-25T11:16:44.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:44.139+0000 7fbf377fe700 1 -- 172.21.15.69:0/187447420 learned_addr learned my addr 172.21.15.69:0/187447420 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:16:44.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:44.139+0000 7fbf377fe700 1 -- 172.21.15.69:0/187447420 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf381999f0 con 0x7fbf381045c0 2024-04-25T11:16:44.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:44.139+0000 7fbf377fe700 1 --2- 172.21.15.69:0/187447420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf381045c0 0x7fbf381994b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbf28015040 tx=0x7fbf2800bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:44.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:44.139+0000 7fbf34ff9700 1 -- 172.21.15.69:0/187447420 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf28017040 con 0x7fbf381045c0 2024-04-25T11:16:44.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:44.139+0000 7fbf3e10a700 1 -- 172.21.15.69:0/187447420 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf38199c80 con 0x7fbf381045c0 2024-04-25T11:16:44.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:44.139+0000 7fbf34ff9700 1 -- 172.21.15.69:0/187447420 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbf28022070 con 0x7fbf381045c0 2024-04-25T11:16:44.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:44.139+0000 7fbf34ff9700 1 -- 172.21.15.69:0/187447420 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf280125c0 con 0x7fbf381045c0 2024-04-25T11:16:44.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:44.139+0000 7fbf3e10a700 1 -- 172.21.15.69:0/187447420 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf3819c890 con 0x7fbf381045c0 2024-04-25T11:16:44.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:44.140+0000 7fbf3e10a700 1 -- 172.21.15.69:0/187447420 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf3810a0b0 con 0x7fbf381045c0 2024-04-25T11:16:44.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:44.140+0000 7fbf34ff9700 1 -- 172.21.15.69:0/187447420 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7fbf280128c0 con 0x7fbf381045c0 2024-04-25T11:16:44.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:44.141+0000 7fbf34ff9700 1 --2- 172.21.15.69:0/187447420 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fbf2003d610 0x7fbf2003fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:44.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:44.141+0000 7fbf34ff9700 1 -- 172.21.15.69:0/187447420 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbf2800f050 con 0x7fbf381045c0 2024-04-25T11:16:44.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:44.141+0000 7fbf36ffd700 1 --2- 172.21.15.69:0/187447420 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fbf2003d610 0x7fbf2003fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:44.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:44.141+0000 7fbf36ffd700 1 --2- 172.21.15.69:0/187447420 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fbf2003d610 0x7fbf2003fb20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c006fd0 tx=0x7fbf2c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:44.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:44.146+0000 7fbf34ff9700 1 -- 172.21.15.69:0/187447420 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbf280263c0 con 0x7fbf381045c0 2024-04-25T11:16:44.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:44.352+0000 7fbf3e10a700 1 -- 172.21.15.69:0/187447420 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fbf3802e6c0 con 0x7fbf381045c0 2024-04-25T11:16:44.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:44.355+0000 7fbf34ff9700 1 -- 172.21.15.69:0/187447420 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fbf28018250 con 0x7fbf381045c0 2024-04-25T11:16:44.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:44.357+0000 7fbf3e10a700 1 -- 172.21.15.69:0/187447420 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fbf2003d610 msgr2=0x7fbf2003fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:44.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:44.357+0000 7fbf3e10a700 1 --2- 172.21.15.69:0/187447420 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fbf2003d610 0x7fbf2003fb20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c006fd0 tx=0x7fbf2c006e40 comp rx=0 tx=0).stop 2024-04-25T11:16:44.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:44.357+0000 7fbf3e10a700 1 -- 172.21.15.69:0/187447420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf381045c0 msgr2=0x7fbf381994b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:44.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:44.357+0000 7fbf3e10a700 1 --2- 172.21.15.69:0/187447420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf381045c0 0x7fbf381994b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbf28015040 tx=0x7fbf2800bd40 comp rx=0 tx=0).stop 2024-04-25T11:16:44.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:44.357+0000 7fbf3e10a700 1 -- 172.21.15.69:0/187447420 shutdown_connections 2024-04-25T11:16:44.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:44.357+0000 7fbf3e10a700 1 --2- 172.21.15.69:0/187447420 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fbf2003d610 0x7fbf2003fb20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:44.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:44.357+0000 7fbf3e10a700 1 --2- 172.21.15.69:0/187447420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf381045c0 0x7fbf381994b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:44.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:44.357+0000 7fbf3e10a700 1 -- 172.21.15.69:0/187447420 >> 172.21.15.69:0/187447420 conn(0x7fbf380fdfe0 msgr2=0x7fbf380fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:44.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:44.358+0000 7fbf3e10a700 1 -- 172.21.15.69:0/187447420 shutdown_connections 2024-04-25T11:16:44.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:44.358+0000 7fbf3e10a700 1 -- 172.21.15.69:0/187447420 wait complete. 2024-04-25T11:16:45.486 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:45 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/187447420' entity='client.admin' 2024-04-25T11:16:45.486 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:45 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:16:45.487 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:45 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:16:45.487 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:45 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi069", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:16:45.487 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:45 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:16:45.487 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:45 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:16:45.487 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:45 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:16:45.488 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-25T11:16:45.489 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-25T11:16:45.722 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:16:47.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:47.296+0000 7efdf19fa700 1 -- 172.21.15.69:0/936229458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efdec103a90 msgr2=0x7efdec103ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:47.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:47.296+0000 7efdf19fa700 1 --2- 172.21.15.69:0/936229458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efdec103a90 0x7efdec103ea0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7efddc009910 tx=0x7efddc009c20 comp rx=0 tx=0).stop 2024-04-25T11:16:47.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:47.296+0000 7efdf19fa700 1 -- 172.21.15.69:0/936229458 shutdown_connections 2024-04-25T11:16:47.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:47.296+0000 7efdf19fa700 1 --2- 172.21.15.69:0/936229458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efdec103a90 0x7efdec103ea0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:47.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:47.296+0000 7efdf19fa700 1 -- 172.21.15.69:0/936229458 >> 172.21.15.69:0/936229458 conn(0x7efdec0ff0d0 msgr2=0x7efdec101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:47.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:47.297+0000 7efdf19fa700 1 -- 172.21.15.69:0/936229458 shutdown_connections 2024-04-25T11:16:47.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:47.297+0000 7efdf19fa700 1 -- 172.21.15.69:0/936229458 wait complete. 2024-04-25T11:16:47.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:47.297+0000 7efdf19fa700 1 Processor -- start 2024-04-25T11:16:47.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:47.297+0000 7efdf19fa700 1 -- start start 2024-04-25T11:16:47.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:47.298+0000 7efdf19fa700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efdec103a90 0x7efdec199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:47.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:47.298+0000 7efdf19fa700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efdec109900 con 0x7efdec103a90 2024-04-25T11:16:47.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:47.298+0000 7efdeaffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efdec103a90 0x7efdec199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:47.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:47.298+0000 7efdeaffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efdec103a90 0x7efdec199510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54798/0 (socket says 172.21.15.69:54798) 2024-04-25T11:16:47.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:47.298+0000 7efdeaffd700 1 -- 172.21.15.69:0/939958195 learned_addr learned my addr 172.21.15.69:0/939958195 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:16:47.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:47.298+0000 7efdeaffd700 1 -- 172.21.15.69:0/939958195 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdec199a50 con 0x7efdec103a90 2024-04-25T11:16:47.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:47.298+0000 7efdeaffd700 1 --2- 172.21.15.69:0/939958195 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efdec103a90 0x7efdec199510 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efddc005760 tx=0x7efddc004f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:47.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:47.298+0000 7efdf09f8700 1 -- 172.21.15.69:0/939958195 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efddc027070 con 0x7efdec103a90 2024-04-25T11:16:47.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:47.298+0000 7efdf09f8700 1 -- 172.21.15.69:0/939958195 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efddc00bdf0 con 0x7efdec103a90 2024-04-25T11:16:47.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:47.298+0000 7efdf19fa700 1 -- 172.21.15.69:0/939958195 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdec199ce0 con 0x7efdec103a90 2024-04-25T11:16:47.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:47.298+0000 7efdf09f8700 1 -- 172.21.15.69:0/939958195 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efddc00fd70 con 0x7efdec103a90 2024-04-25T11:16:47.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:47.298+0000 7efdf19fa700 1 -- 172.21.15.69:0/939958195 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdec19c8f0 con 0x7efdec103a90 2024-04-25T11:16:47.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:47.299+0000 7efdf19fa700 1 -- 172.21.15.69:0/939958195 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdec192750 con 0x7efdec103a90 2024-04-25T11:16:47.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:47.299+0000 7efdf09f8700 1 -- 172.21.15.69:0/939958195 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7efddc02a070 con 0x7efdec103a90 2024-04-25T11:16:47.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:47.300+0000 7efdf09f8700 1 --2- 172.21.15.69:0/939958195 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7efdd403d610 0x7efdd403fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:47.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:47.300+0000 7efdf09f8700 1 -- 172.21.15.69:0/939958195 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7efddc0527a0 con 0x7efdec103a90 2024-04-25T11:16:47.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:47.300+0000 7efdea7fc700 1 --2- 172.21.15.69:0/939958195 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7efdd403d610 0x7efdd403fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:47.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:47.300+0000 7efdea7fc700 1 --2- 172.21.15.69:0/939958195 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7efdd403d610 0x7efdd403fb20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7efde0006fd0 tx=0x7efde0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:47.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:47.305+0000 7efdf09f8700 1 -- 172.21.15.69:0/939958195 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efddc013070 con 0x7efdec103a90 2024-04-25T11:16:47.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:47.509+0000 7efdf19fa700 1 -- 172.21.15.69:0/939958195 --> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7efdec02e620 con 0x7efdd403d610 2024-04-25T11:16:47.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:47.513+0000 7efdf09f8700 1 -- 172.21.15.69:0/939958195 <== mgr.14152 v2:172.21.15.69:6800/4220065409 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efdec02e620 con 0x7efdd403d610 2024-04-25T11:16:47.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:47.515+0000 7efdf19fa700 1 -- 172.21.15.69:0/939958195 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7efdd403d610 msgr2=0x7efdd403fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:47.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:47.515+0000 7efdf19fa700 1 --2- 172.21.15.69:0/939958195 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7efdd403d610 0x7efdd403fb20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7efde0006fd0 tx=0x7efde0006e40 comp rx=0 tx=0).stop 2024-04-25T11:16:47.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:47.515+0000 7efdf19fa700 1 -- 172.21.15.69:0/939958195 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efdec103a90 msgr2=0x7efdec199510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:47.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:47.515+0000 7efdf19fa700 1 --2- 172.21.15.69:0/939958195 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efdec103a90 0x7efdec199510 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efddc005760 tx=0x7efddc004f60 comp rx=0 tx=0).stop 2024-04-25T11:16:47.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:47.515+0000 7efdf19fa700 1 -- 172.21.15.69:0/939958195 shutdown_connections 2024-04-25T11:16:47.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:47.515+0000 7efdf19fa700 1 --2- 172.21.15.69:0/939958195 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7efdd403d610 0x7efdd403fb20 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:47.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:47.515+0000 7efdf19fa700 1 --2- 172.21.15.69:0/939958195 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efdec103a90 0x7efdec199510 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:47.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:47.515+0000 7efdf19fa700 1 -- 172.21.15.69:0/939958195 >> 172.21.15.69:0/939958195 conn(0x7efdec0ff0d0 msgr2=0x7efdec0ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:47.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:47.515+0000 7efdf19fa700 1 -- 172.21.15.69:0/939958195 shutdown_connections 2024-04-25T11:16:47.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:47.515+0000 7efdf19fa700 1 -- 172.21.15.69:0/939958195 wait complete. 2024-04-25T11:16:48.265 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi146 2024-04-25T11:16:48.265 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T11:16:48.266 DEBUG:teuthology.orchestra.run.smithi146:> dd of=/etc/ceph/ceph.conf 2024-04-25T11:16:48.285 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T11:16:48.285 DEBUG:teuthology.orchestra.run.smithi146:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T11:16:48.343 INFO:tasks.cephadm:Adding host smithi146 to orchestrator... 2024-04-25T11:16:48.343 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph orch host add smithi146 2024-04-25T11:16:48.583 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:16:48.811 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:48 smithi069 ceph-mon[29188]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:16:48.812 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:48 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:16:48.812 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:48 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:16:48.812 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:48 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:16:48.812 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:48 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:16:48.812 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:48 smithi069 ceph-mon[29188]: Updating smithi069:/etc/ceph/ceph.conf 2024-04-25T11:16:48.812 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:48 smithi069 ceph-mon[29188]: Updating smithi069:/var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T11:16:48.812 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:48 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:16:48.812 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:48 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:16:48.812 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:48 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:16:49.768 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:49 smithi069 ceph-mon[29188]: Updating smithi069:/etc/ceph/ceph.client.admin.keyring 2024-04-25T11:16:49.768 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:49 smithi069 ceph-mon[29188]: Updating smithi069:/var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T11:16:50.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:50.102+0000 7fad82e1a700 1 -- 172.21.15.69:0/1697716662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad7c108940 msgr2=0x7fad7c108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:50.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:50.102+0000 7fad82e1a700 1 --2- 172.21.15.69:0/1697716662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad7c108940 0x7fad7c108d50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fad70009910 tx=0x7fad70009c20 comp rx=0 tx=0).stop 2024-04-25T11:16:50.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:50.102+0000 7fad82e1a700 1 -- 172.21.15.69:0/1697716662 shutdown_connections 2024-04-25T11:16:50.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:50.102+0000 7fad82e1a700 1 --2- 172.21.15.69:0/1697716662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad7c108940 0x7fad7c108d50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:50.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:50.102+0000 7fad82e1a700 1 -- 172.21.15.69:0/1697716662 >> 172.21.15.69:0/1697716662 conn(0x7fad7c078460 msgr2=0x7fad7c0788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:50.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:50.102+0000 7fad82e1a700 1 -- 172.21.15.69:0/1697716662 shutdown_connections 2024-04-25T11:16:50.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:50.102+0000 7fad82e1a700 1 -- 172.21.15.69:0/1697716662 wait complete. 2024-04-25T11:16:50.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:50.103+0000 7fad82e1a700 1 Processor -- start 2024-04-25T11:16:50.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:50.103+0000 7fad82e1a700 1 -- start start 2024-04-25T11:16:50.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:50.103+0000 7fad82e1a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad7c108940 0x7fad7c198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:50.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:50.103+0000 7fad82e1a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fad7c10cd40 con 0x7fad7c108940 2024-04-25T11:16:50.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:50.103+0000 7fad80bb6700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad7c108940 0x7fad7c198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:50.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:50.103+0000 7fad80bb6700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad7c108940 0x7fad7c198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54814/0 (socket says 172.21.15.69:54814) 2024-04-25T11:16:50.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:50.103+0000 7fad80bb6700 1 -- 172.21.15.69:0/256775387 learned_addr learned my addr 172.21.15.69:0/256775387 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:16:50.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:50.104+0000 7fad80bb6700 1 -- 172.21.15.69:0/256775387 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad7c198b60 con 0x7fad7c108940 2024-04-25T11:16:50.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:50.104+0000 7fad80bb6700 1 --2- 172.21.15.69:0/256775387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad7c108940 0x7fad7c198620 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fad70005660 tx=0x7fad70005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:50.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:50.104+0000 7fad79ffb700 1 -- 172.21.15.69:0/256775387 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad70026070 con 0x7fad7c108940 2024-04-25T11:16:50.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:50.104+0000 7fad79ffb700 1 -- 172.21.15.69:0/256775387 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad7000b6f0 con 0x7fad7c108940 2024-04-25T11:16:50.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:50.104+0000 7fad82e1a700 1 -- 172.21.15.69:0/256775387 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad7c198e50 con 0x7fad7c108940 2024-04-25T11:16:50.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:50.104+0000 7fad79ffb700 1 -- 172.21.15.69:0/256775387 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad700037d0 con 0x7fad7c108940 2024-04-25T11:16:50.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:50.104+0000 7fad82e1a700 1 -- 172.21.15.69:0/256775387 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad7c19ce50 con 0x7fad7c108940 2024-04-25T11:16:50.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:50.105+0000 7fad82e1a700 1 -- 172.21.15.69:0/256775387 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad7c104050 con 0x7fad7c108940 2024-04-25T11:16:50.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:50.105+0000 7fad79ffb700 1 -- 172.21.15.69:0/256775387 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7fad70029070 con 0x7fad7c108940 2024-04-25T11:16:50.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:50.105+0000 7fad79ffb700 1 --2- 172.21.15.69:0/256775387 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fad6c03d670 0x7fad6c03fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:50.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:50.105+0000 7fad79ffb700 1 -- 172.21.15.69:0/256775387 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fad7001e070 con 0x7fad7c108940 2024-04-25T11:16:50.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:50.106+0000 7fad7bfff700 1 --2- 172.21.15.69:0/256775387 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fad6c03d670 0x7fad6c03fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:50.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:50.106+0000 7fad7bfff700 1 --2- 172.21.15.69:0/256775387 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fad6c03d670 0x7fad6c03fb80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fad68006fd0 tx=0x7fad68006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:50.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:50.111+0000 7fad79ffb700 1 -- 172.21.15.69:0/256775387 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fad70013070 con 0x7fad7c108940 2024-04-25T11:16:50.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:50.313+0000 7fad82e1a700 1 -- 172.21.15.69:0/256775387 --> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi146", "target": ["mon-mgr", ""]}) -- 0x7fad7c199980 con 0x7fad6c03d670 2024-04-25T11:16:51.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:50 smithi069 ceph-mon[29188]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi146", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:16:52.949 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:52 smithi069 ceph-mon[29188]: Deploying cephadm binary to smithi146 2024-04-25T11:16:52.949 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:52 smithi069 ceph-mon[29188]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:16:54.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:54.600+0000 7fad79ffb700 1 -- 172.21.15.69:0/256775387 <== mgr.14152 v2:172.21.15.69:6800/4220065409 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fad7c199980 con 0x7fad6c03d670 2024-04-25T11:16:54.600 INFO:teuthology.orchestra.run.smithi069.stdout:Added host 'smithi146' with addr '172.21.15.146' 2024-04-25T11:16:54.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:54.602+0000 7fad82e1a700 1 -- 172.21.15.69:0/256775387 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fad6c03d670 msgr2=0x7fad6c03fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:54.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:54.602+0000 7fad82e1a700 1 --2- 172.21.15.69:0/256775387 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fad6c03d670 0x7fad6c03fb80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fad68006fd0 tx=0x7fad68006e40 comp rx=0 tx=0).stop 2024-04-25T11:16:54.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:54.602+0000 7fad82e1a700 1 -- 172.21.15.69:0/256775387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad7c108940 msgr2=0x7fad7c198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:54.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:54.602+0000 7fad82e1a700 1 --2- 172.21.15.69:0/256775387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad7c108940 0x7fad7c198620 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fad70005660 tx=0x7fad70005740 comp rx=0 tx=0).stop 2024-04-25T11:16:54.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:54.603+0000 7fad82e1a700 1 -- 172.21.15.69:0/256775387 shutdown_connections 2024-04-25T11:16:54.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:54.603+0000 7fad82e1a700 1 --2- 172.21.15.69:0/256775387 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fad6c03d670 0x7fad6c03fb80 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:54.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:54.603+0000 7fad82e1a700 1 --2- 172.21.15.69:0/256775387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad7c108940 0x7fad7c198620 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:54.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:54.603+0000 7fad82e1a700 1 -- 172.21.15.69:0/256775387 >> 172.21.15.69:0/256775387 conn(0x7fad7c078460 msgr2=0x7fad7c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:54.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:54.603+0000 7fad82e1a700 1 -- 172.21.15.69:0/256775387 shutdown_connections 2024-04-25T11:16:54.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:54.603+0000 7fad82e1a700 1 -- 172.21.15.69:0/256775387 wait complete. 2024-04-25T11:16:54.914 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:54 smithi069 ceph-mon[29188]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:16:54.915 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:54 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:16:54.915 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:54 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:16:55.206 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T11:16:55.441 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:16:55.834 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:55 smithi069 ceph-mon[29188]: Added host smithi146 2024-04-25T11:16:55.834 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:55 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:16:55.834 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:55 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:16:56.940 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:56 smithi069 ceph-mon[29188]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:16:57.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:57.051+0000 7f16feffc700 1 -- 172.21.15.69:0/1576129662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16f8108930 msgr2=0x7f16f8108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:57.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:57.051+0000 7f16feffc700 1 --2- 172.21.15.69:0/1576129662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16f8108930 0x7f16f8108d40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f16ec009910 tx=0x7f16ec009c20 comp rx=0 tx=0).stop 2024-04-25T11:16:57.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:57.051+0000 7f16feffc700 1 -- 172.21.15.69:0/1576129662 shutdown_connections 2024-04-25T11:16:57.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:57.051+0000 7f16feffc700 1 --2- 172.21.15.69:0/1576129662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16f8108930 0x7f16f8108d40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:57.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:57.051+0000 7f16feffc700 1 -- 172.21.15.69:0/1576129662 >> 172.21.15.69:0/1576129662 conn(0x7f16f80fdd00 msgr2=0x7f16f8100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:57.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:57.051+0000 7f16feffc700 1 -- 172.21.15.69:0/1576129662 shutdown_connections 2024-04-25T11:16:57.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:57.051+0000 7f16feffc700 1 -- 172.21.15.69:0/1576129662 wait complete. 2024-04-25T11:16:57.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:57.052+0000 7f16feffc700 1 Processor -- start 2024-04-25T11:16:57.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:57.052+0000 7f16feffc700 1 -- start start 2024-04-25T11:16:57.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:57.052+0000 7f16feffc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16f8108930 0x7f16f8199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:57.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:57.052+0000 7f16feffc700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f16f810cd30 con 0x7f16f8108930 2024-04-25T11:16:57.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:57.052+0000 7f16fcd98700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16f8108930 0x7f16f8199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:57.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:57.052+0000 7f16fcd98700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16f8108930 0x7f16f8199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53790/0 (socket says 172.21.15.69:53790) 2024-04-25T11:16:57.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:57.052+0000 7f16fcd98700 1 -- 172.21.15.69:0/3838458589 learned_addr learned my addr 172.21.15.69:0/3838458589 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:16:57.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:57.052+0000 7f16fcd98700 1 -- 172.21.15.69:0/3838458589 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16f8199c80 con 0x7f16f8108930 2024-04-25T11:16:57.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:57.052+0000 7f16fcd98700 1 --2- 172.21.15.69:0/3838458589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16f8108930 0x7f16f8199740 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f16ec005660 tx=0x7f16ec005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:57.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:57.053+0000 7f16f5ffb700 1 -- 172.21.15.69:0/3838458589 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16ec026070 con 0x7f16f8108930 2024-04-25T11:16:57.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:57.053+0000 7f16f5ffb700 1 -- 172.21.15.69:0/3838458589 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f16ec00b6f0 con 0x7f16f8108930 2024-04-25T11:16:57.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:57.053+0000 7f16feffc700 1 -- 172.21.15.69:0/3838458589 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16f8199f70 con 0x7f16f8108930 2024-04-25T11:16:57.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:57.053+0000 7f16f5ffb700 1 -- 172.21.15.69:0/3838458589 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16ec0037d0 con 0x7f16f8108930 2024-04-25T11:16:57.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:57.053+0000 7f16feffc700 1 -- 172.21.15.69:0/3838458589 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16f819df70 con 0x7f16f8108930 2024-04-25T11:16:57.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:57.054+0000 7f16feffc700 1 -- 172.21.15.69:0/3838458589 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16f8104180 con 0x7f16f8108930 2024-04-25T11:16:57.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:57.054+0000 7f16f5ffb700 1 -- 172.21.15.69:0/3838458589 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f16ec029070 con 0x7f16f8108930 2024-04-25T11:16:57.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:57.054+0000 7f16f5ffb700 1 --2- 172.21.15.69:0/3838458589 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f16e803d610 0x7f16e803fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:57.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:57.054+0000 7f16f5ffb700 1 -- 172.21.15.69:0/3838458589 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f16ec01e070 con 0x7f16f8108930 2024-04-25T11:16:57.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:57.054+0000 7f16f7fff700 1 --2- 172.21.15.69:0/3838458589 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f16e803d610 0x7f16e803fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:57.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:57.055+0000 7f16f7fff700 1 --2- 172.21.15.69:0/3838458589 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f16e803d610 0x7f16e803fb20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f16e4006fd0 tx=0x7f16e4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:57.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:57.059+0000 7f16f5ffb700 1 -- 172.21.15.69:0/3838458589 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f16ec013070 con 0x7f16f8108930 2024-04-25T11:16:57.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:57.261+0000 7f16feffc700 1 -- 172.21.15.69:0/3838458589 --> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f16f819aaa0 con 0x7f16e803d610 2024-04-25T11:16:57.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:57.263+0000 7f16f5ffb700 1 -- 172.21.15.69:0/3838458589 <== mgr.14152 v2:172.21.15.69:6800/4220065409 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f16f819aaa0 con 0x7f16e803d610 2024-04-25T11:16:57.263 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:16:57.263 INFO:teuthology.orchestra.run.smithi069.stdout:[{"addr": "172.21.15.69", "hostname": "smithi069", "labels": [], "status": ""}, {"addr": "172.21.15.146", "hostname": "smithi146", "labels": [], "status": ""}] 2024-04-25T11:16:57.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:57.265+0000 7f16feffc700 1 -- 172.21.15.69:0/3838458589 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f16e803d610 msgr2=0x7f16e803fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:57.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:57.265+0000 7f16feffc700 1 --2- 172.21.15.69:0/3838458589 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f16e803d610 0x7f16e803fb20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f16e4006fd0 tx=0x7f16e4006e40 comp rx=0 tx=0).stop 2024-04-25T11:16:57.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:57.265+0000 7f16feffc700 1 -- 172.21.15.69:0/3838458589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16f8108930 msgr2=0x7f16f8199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:57.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:57.265+0000 7f16feffc700 1 --2- 172.21.15.69:0/3838458589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16f8108930 0x7f16f8199740 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f16ec005660 tx=0x7f16ec005740 comp rx=0 tx=0).stop 2024-04-25T11:16:57.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:57.265+0000 7f16feffc700 1 -- 172.21.15.69:0/3838458589 shutdown_connections 2024-04-25T11:16:57.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:57.265+0000 7f16feffc700 1 --2- 172.21.15.69:0/3838458589 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f16e803d610 0x7f16e803fb20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:57.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:57.265+0000 7f16feffc700 1 --2- 172.21.15.69:0/3838458589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f16f8108930 0x7f16f8199740 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:57.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:57.265+0000 7f16feffc700 1 -- 172.21.15.69:0/3838458589 >> 172.21.15.69:0/3838458589 conn(0x7f16f80fdd00 msgr2=0x7f16f8101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:57.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:57.265+0000 7f16feffc700 1 -- 172.21.15.69:0/3838458589 shutdown_connections 2024-04-25T11:16:57.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:57.265+0000 7f16feffc700 1 -- 172.21.15.69:0/3838458589 wait complete. 2024-04-25T11:16:57.855 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi152 2024-04-25T11:16:57.855 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-25T11:16:57.855 DEBUG:teuthology.orchestra.run.smithi152:> dd of=/etc/ceph/ceph.conf 2024-04-25T11:16:57.874 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-25T11:16:57.874 DEBUG:teuthology.orchestra.run.smithi152:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T11:16:57.932 INFO:tasks.cephadm:Adding host smithi152 to orchestrator... 2024-04-25T11:16:57.933 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph orch host add smithi152 2024-04-25T11:16:58.165 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:16:58.449 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:58 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:16:58.450 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:58 smithi069 ceph-mon[29188]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T11:16:58.450 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:16:58 smithi069 ceph-mon[29188]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:16:59.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:59.580+0000 7fea64bc1700 1 -- 172.21.15.69:0/3271046605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea60108900 msgr2=0x7fea60108d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:59.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:59.580+0000 7fea64bc1700 1 --2- 172.21.15.69:0/3271046605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea60108900 0x7fea60108d10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fea50009910 tx=0x7fea50009c20 comp rx=0 tx=0).stop 2024-04-25T11:16:59.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:59.580+0000 7fea64bc1700 1 -- 172.21.15.69:0/3271046605 shutdown_connections 2024-04-25T11:16:59.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:59.580+0000 7fea64bc1700 1 --2- 172.21.15.69:0/3271046605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea60108900 0x7fea60108d10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:59.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:59.580+0000 7fea64bc1700 1 -- 172.21.15.69:0/3271046605 >> 172.21.15.69:0/3271046605 conn(0x7fea600fe410 msgr2=0x7fea60100860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:59.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:59.580+0000 7fea64bc1700 1 -- 172.21.15.69:0/3271046605 shutdown_connections 2024-04-25T11:16:59.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:59.580+0000 7fea64bc1700 1 -- 172.21.15.69:0/3271046605 wait complete. 2024-04-25T11:16:59.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:59.581+0000 7fea64bc1700 1 Processor -- start 2024-04-25T11:16:59.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:59.581+0000 7fea64bc1700 1 -- start start 2024-04-25T11:16:59.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:59.581+0000 7fea64bc1700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea60108900 0x7fea6019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:59.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:59.581+0000 7fea64bc1700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fea6010cd00 con 0x7fea60108900 2024-04-25T11:16:59.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:59.581+0000 7fea5e59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea60108900 0x7fea6019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:59.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:59.581+0000 7fea5e59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea60108900 0x7fea6019a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53818/0 (socket says 172.21.15.69:53818) 2024-04-25T11:16:59.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:59.581+0000 7fea5e59c700 1 -- 172.21.15.69:0/3332711556 learned_addr learned my addr 172.21.15.69:0/3332711556 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:16:59.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:59.582+0000 7fea5e59c700 1 -- 172.21.15.69:0/3332711556 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea6006c940 con 0x7fea60108900 2024-04-25T11:16:59.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:59.582+0000 7fea5e59c700 1 --2- 172.21.15.69:0/3332711556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea60108900 0x7fea6019a590 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fea50005660 tx=0x7fea50005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:59.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:59.582+0000 7fea4f7fe700 1 -- 172.21.15.69:0/3332711556 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea50026070 con 0x7fea60108900 2024-04-25T11:16:59.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:59.582+0000 7fea4f7fe700 1 -- 172.21.15.69:0/3332711556 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fea5000b750 con 0x7fea60108900 2024-04-25T11:16:59.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:59.582+0000 7fea64bc1700 1 -- 172.21.15.69:0/3332711556 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea6019ab90 con 0x7fea60108900 2024-04-25T11:16:59.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:59.582+0000 7fea4f7fe700 1 -- 172.21.15.69:0/3332711556 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea50003810 con 0x7fea60108900 2024-04-25T11:16:59.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:59.582+0000 7fea64bc1700 1 -- 172.21.15.69:0/3332711556 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea6019b050 con 0x7fea60108900 2024-04-25T11:16:59.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:59.583+0000 7fea4f7fe700 1 -- 172.21.15.69:0/3332711556 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7fea50029070 con 0x7fea60108900 2024-04-25T11:16:59.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:59.583+0000 7fea4f7fe700 1 --2- 172.21.15.69:0/3332711556 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fea4803d670 0x7fea4803fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:59.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:59.583+0000 7fea4f7fe700 1 -- 172.21.15.69:0/3332711556 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fea5001e070 con 0x7fea60108900 2024-04-25T11:16:59.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:59.583+0000 7fea64bc1700 1 -- 172.21.15.69:0/3332711556 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea60104010 con 0x7fea60108900 2024-04-25T11:16:59.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:59.583+0000 7fea5dd9b700 1 --2- 172.21.15.69:0/3332711556 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fea4803d670 0x7fea4803fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:59.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:59.583+0000 7fea5dd9b700 1 --2- 172.21.15.69:0/3332711556 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fea4803d670 0x7fea4803fb80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fea54006fd0 tx=0x7fea54006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:59.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:59.589+0000 7fea4f7fe700 1 -- 172.21.15.69:0/3332711556 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fea50030080 con 0x7fea60108900 2024-04-25T11:16:59.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:16:59.791+0000 7fea64bc1700 1 -- 172.21.15.69:0/3332711556 --> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi152", "target": ["mon-mgr", ""]}) -- 0x7fea6019b330 con 0x7fea4803d670 2024-04-25T11:17:00.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:00 smithi069 ceph-mon[29188]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:00.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:00 smithi069 ceph-mon[29188]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi152", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:17:00.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:00 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:00.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:00 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:00.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:00 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:00.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:00 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:00.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:00 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi146", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:17:00.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:00 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:17:00.951 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:00 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:17:01.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:01 smithi069 ceph-mon[29188]: Updating smithi146:/etc/ceph/ceph.conf 2024-04-25T11:17:01.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:01 smithi069 ceph-mon[29188]: Deploying cephadm binary to smithi152 2024-04-25T11:17:01.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:01 smithi069 ceph-mon[29188]: Updating smithi146:/var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T11:17:01.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:01 smithi069 ceph-mon[29188]: Updating smithi146:/etc/ceph/ceph.client.admin.keyring 2024-04-25T11:17:03.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:02 smithi069 ceph-mon[29188]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:03.894 INFO:teuthology.orchestra.run.smithi069.stdout:Added host 'smithi152' with addr '172.21.15.152' 2024-04-25T11:17:03.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:03.893+0000 7fea4f7fe700 1 -- 172.21.15.69:0/3332711556 <== mgr.14152 v2:172.21.15.69:6800/4220065409 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fea6019b330 con 0x7fea4803d670 2024-04-25T11:17:03.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:03.896+0000 7fea64bc1700 1 -- 172.21.15.69:0/3332711556 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fea4803d670 msgr2=0x7fea4803fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:03.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:03.896+0000 7fea64bc1700 1 --2- 172.21.15.69:0/3332711556 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fea4803d670 0x7fea4803fb80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fea54006fd0 tx=0x7fea54006e40 comp rx=0 tx=0).stop 2024-04-25T11:17:03.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:03.896+0000 7fea64bc1700 1 -- 172.21.15.69:0/3332711556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea60108900 msgr2=0x7fea6019a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:03.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:03.896+0000 7fea64bc1700 1 --2- 172.21.15.69:0/3332711556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea60108900 0x7fea6019a590 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fea50005660 tx=0x7fea50005740 comp rx=0 tx=0).stop 2024-04-25T11:17:03.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:03.896+0000 7fea64bc1700 1 -- 172.21.15.69:0/3332711556 shutdown_connections 2024-04-25T11:17:03.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:03.896+0000 7fea64bc1700 1 --2- 172.21.15.69:0/3332711556 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fea4803d670 0x7fea4803fb80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:03.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:03.896+0000 7fea64bc1700 1 --2- 172.21.15.69:0/3332711556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fea60108900 0x7fea6019a590 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:03.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:03.896+0000 7fea64bc1700 1 -- 172.21.15.69:0/3332711556 >> 172.21.15.69:0/3332711556 conn(0x7fea600fe410 msgr2=0x7fea600ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:17:03.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:03.896+0000 7fea64bc1700 1 -- 172.21.15.69:0/3332711556 shutdown_connections 2024-04-25T11:17:03.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:03.896+0000 7fea64bc1700 1 -- 172.21.15.69:0/3332711556 wait complete. 2024-04-25T11:17:04.465 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T11:17:04.698 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:17:04.821 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:04 smithi069 ceph-mon[29188]: Updating smithi146:/var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T11:17:04.821 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:04 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:04.821 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:04 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:04.821 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:04 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:04.821 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:04 smithi069 ceph-mon[29188]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:04.821 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:04 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:04.821 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:04 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:17:04.821 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:04 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:05.743 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:05 smithi069 ceph-mon[29188]: Added host smithi152 2024-04-25T11:17:05.743 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:05 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:06.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:06.256+0000 7f1c55554700 1 -- 172.21.15.69:0/2627198584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c501088f0 msgr2=0x7f1c50108d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:06.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:06.256+0000 7f1c55554700 1 --2- 172.21.15.69:0/2627198584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c501088f0 0x7f1c50108d00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1c40009910 tx=0x7f1c40009c20 comp rx=0 tx=0).stop 2024-04-25T11:17:06.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:06.257+0000 7f1c55554700 1 -- 172.21.15.69:0/2627198584 shutdown_connections 2024-04-25T11:17:06.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:06.257+0000 7f1c55554700 1 --2- 172.21.15.69:0/2627198584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c501088f0 0x7f1c50108d00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:06.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:06.257+0000 7f1c55554700 1 -- 172.21.15.69:0/2627198584 >> 172.21.15.69:0/2627198584 conn(0x7f1c500fe440 msgr2=0x7f1c50100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:17:06.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:06.257+0000 7f1c55554700 1 -- 172.21.15.69:0/2627198584 shutdown_connections 2024-04-25T11:17:06.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:06.257+0000 7f1c55554700 1 -- 172.21.15.69:0/2627198584 wait complete. 2024-04-25T11:17:06.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:06.257+0000 7f1c55554700 1 Processor -- start 2024-04-25T11:17:06.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:06.257+0000 7f1c55554700 1 -- start start 2024-04-25T11:17:06.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:06.258+0000 7f1c55554700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c501088f0 0x7f1c5019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:06.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:06.258+0000 7f1c55554700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5010ccf0 con 0x7f1c501088f0 2024-04-25T11:17:06.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:06.258+0000 7f1c4effd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c501088f0 0x7f1c5019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:06.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:06.258+0000 7f1c4effd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c501088f0 0x7f1c5019a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50372/0 (socket says 172.21.15.69:50372) 2024-04-25T11:17:06.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:06.258+0000 7f1c4effd700 1 -- 172.21.15.69:0/573032613 learned_addr learned my addr 172.21.15.69:0/573032613 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:17:06.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:06.258+0000 7f1c4effd700 1 -- 172.21.15.69:0/573032613 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c5006c600 con 0x7f1c501088f0 2024-04-25T11:17:06.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:06.258+0000 7f1c4effd700 1 --2- 172.21.15.69:0/573032613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c501088f0 0x7f1c5019a590 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1c40005660 tx=0x7f1c40005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:06.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:06.258+0000 7f1c37fff700 1 -- 172.21.15.69:0/573032613 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c40026070 con 0x7f1c501088f0 2024-04-25T11:17:06.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:06.258+0000 7f1c37fff700 1 -- 172.21.15.69:0/573032613 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1c4000b750 con 0x7f1c501088f0 2024-04-25T11:17:06.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:06.258+0000 7f1c55554700 1 -- 172.21.15.69:0/573032613 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c5019ab90 con 0x7f1c501088f0 2024-04-25T11:17:06.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:06.258+0000 7f1c37fff700 1 -- 172.21.15.69:0/573032613 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c40003810 con 0x7f1c501088f0 2024-04-25T11:17:06.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:06.258+0000 7f1c55554700 1 -- 172.21.15.69:0/573032613 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c5019b050 con 0x7f1c501088f0 2024-04-25T11:17:06.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:06.259+0000 7f1c37fff700 1 -- 172.21.15.69:0/573032613 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f1c40029070 con 0x7f1c501088f0 2024-04-25T11:17:06.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:06.259+0000 7f1c37fff700 1 --2- 172.21.15.69:0/573032613 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f1c3803d610 0x7f1c3803fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:06.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:06.259+0000 7f1c37fff700 1 -- 172.21.15.69:0/573032613 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1c4001e070 con 0x7f1c501088f0 2024-04-25T11:17:06.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:06.260+0000 7f1c4e7fc700 1 --2- 172.21.15.69:0/573032613 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f1c3803d610 0x7f1c3803fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:06.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:06.260+0000 7f1c55554700 1 -- 172.21.15.69:0/573032613 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c50104000 con 0x7f1c501088f0 2024-04-25T11:17:06.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:06.260+0000 7f1c4e7fc700 1 --2- 172.21.15.69:0/573032613 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f1c3803d610 0x7f1c3803fb20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1c44006fd0 tx=0x7f1c44006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:06.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:06.265+0000 7f1c37fff700 1 -- 172.21.15.69:0/573032613 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1c40013070 con 0x7f1c501088f0 2024-04-25T11:17:06.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:06.467+0000 7f1c55554700 1 -- 172.21.15.69:0/573032613 --> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1c5019b330 con 0x7f1c3803d610 2024-04-25T11:17:06.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:06.468+0000 7f1c37fff700 1 -- 172.21.15.69:0/573032613 <== mgr.14152 v2:172.21.15.69:6800/4220065409 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7f1c5019b330 con 0x7f1c3803d610 2024-04-25T11:17:06.468 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:17:06.468 INFO:teuthology.orchestra.run.smithi069.stdout:[{"addr": "172.21.15.69", "hostname": "smithi069", "labels": [], "status": ""}, {"addr": "172.21.15.146", "hostname": "smithi146", "labels": [], "status": ""}, {"addr": "172.21.15.152", "hostname": "smithi152", "labels": [], "status": ""}] 2024-04-25T11:17:06.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:06.470+0000 7f1c55554700 1 -- 172.21.15.69:0/573032613 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f1c3803d610 msgr2=0x7f1c3803fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:06.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:06.470+0000 7f1c55554700 1 --2- 172.21.15.69:0/573032613 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f1c3803d610 0x7f1c3803fb20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1c44006fd0 tx=0x7f1c44006e40 comp rx=0 tx=0).stop 2024-04-25T11:17:06.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:06.470+0000 7f1c55554700 1 -- 172.21.15.69:0/573032613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c501088f0 msgr2=0x7f1c5019a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:06.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:06.470+0000 7f1c55554700 1 --2- 172.21.15.69:0/573032613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c501088f0 0x7f1c5019a590 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1c40005660 tx=0x7f1c40005740 comp rx=0 tx=0).stop 2024-04-25T11:17:06.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:06.470+0000 7f1c55554700 1 -- 172.21.15.69:0/573032613 shutdown_connections 2024-04-25T11:17:06.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:06.470+0000 7f1c55554700 1 --2- 172.21.15.69:0/573032613 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f1c3803d610 0x7f1c3803fb20 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:06.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:06.470+0000 7f1c55554700 1 --2- 172.21.15.69:0/573032613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c501088f0 0x7f1c5019a590 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:06.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:06.470+0000 7f1c55554700 1 -- 172.21.15.69:0/573032613 >> 172.21.15.69:0/573032613 conn(0x7f1c500fe440 msgr2=0x7f1c500ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:17:06.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:06.470+0000 7f1c55554700 1 -- 172.21.15.69:0/573032613 shutdown_connections 2024-04-25T11:17:06.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:06.470+0000 7f1c55554700 1 -- 172.21.15.69:0/573032613 wait complete. 2024-04-25T11:17:06.790 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:06 smithi069 ceph-mon[29188]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:07.087 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-25T11:17:07.087 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-25T11:17:07.322 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:17:07.699 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:07 smithi069 ceph-mon[29188]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T11:17:07.700 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:07 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:08.699 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:08 smithi069 ceph-mon[29188]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:08.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:08.947+0000 7fc9b48ae700 1 -- 172.21.15.69:0/3355311081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9b0108940 msgr2=0x7fc9b0108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:08.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:08.947+0000 7fc9b48ae700 1 --2- 172.21.15.69:0/3355311081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9b0108940 0x7fc9b0108d50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0009910 tx=0x7fc9a0009c20 comp rx=0 tx=0).stop 2024-04-25T11:17:08.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:08.948+0000 7fc9b48ae700 1 -- 172.21.15.69:0/3355311081 shutdown_connections 2024-04-25T11:17:08.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:08.948+0000 7fc9b48ae700 1 --2- 172.21.15.69:0/3355311081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9b0108940 0x7fc9b0108d50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:08.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:08.948+0000 7fc9b48ae700 1 -- 172.21.15.69:0/3355311081 >> 172.21.15.69:0/3355311081 conn(0x7fc9b0078460 msgr2=0x7fc9b00788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:17:08.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:08.948+0000 7fc9b48ae700 1 -- 172.21.15.69:0/3355311081 shutdown_connections 2024-04-25T11:17:08.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:08.948+0000 7fc9b48ae700 1 -- 172.21.15.69:0/3355311081 wait complete. 2024-04-25T11:17:08.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:08.948+0000 7fc9b48ae700 1 Processor -- start 2024-04-25T11:17:08.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:08.948+0000 7fc9b48ae700 1 -- start start 2024-04-25T11:17:08.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:08.949+0000 7fc9b48ae700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9b0108940 0x7fc9b00ff810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:08.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:08.949+0000 7fc9b48ae700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b010cd40 con 0x7fc9b0108940 2024-04-25T11:17:08.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:08.949+0000 7fc9ae59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9b0108940 0x7fc9b00ff810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:08.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:08.949+0000 7fc9ae59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9b0108940 0x7fc9b00ff810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50390/0 (socket says 172.21.15.69:50390) 2024-04-25T11:17:08.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:08.949+0000 7fc9ae59c700 1 -- 172.21.15.69:0/2960795090 learned_addr learned my addr 172.21.15.69:0/2960795090 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:17:08.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:08.949+0000 7fc9ae59c700 1 -- 172.21.15.69:0/2960795090 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9b0101660 con 0x7fc9b0108940 2024-04-25T11:17:08.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:08.949+0000 7fc9ae59c700 1 --2- 172.21.15.69:0/2960795090 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9b0108940 0x7fc9b00ff810 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0005660 tx=0x7fc9a0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:08.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:08.949+0000 7fc99f7fe700 1 -- 172.21.15.69:0/2960795090 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9a0026070 con 0x7fc9b0108940 2024-04-25T11:17:08.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:08.949+0000 7fc99f7fe700 1 -- 172.21.15.69:0/2960795090 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc9a000b6f0 con 0x7fc9b0108940 2024-04-25T11:17:08.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:08.949+0000 7fc9b48ae700 1 -- 172.21.15.69:0/2960795090 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9b00ffd50 con 0x7fc9b0108940 2024-04-25T11:17:08.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:08.949+0000 7fc99f7fe700 1 -- 172.21.15.69:0/2960795090 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9a00037d0 con 0x7fc9b0108940 2024-04-25T11:17:08.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:08.950+0000 7fc9b48ae700 1 -- 172.21.15.69:0/2960795090 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9b0100210 con 0x7fc9b0108940 2024-04-25T11:17:08.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:08.950+0000 7fc99f7fe700 1 -- 172.21.15.69:0/2960795090 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7fc9a0029070 con 0x7fc9b0108940 2024-04-25T11:17:08.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:08.950+0000 7fc9b48ae700 1 -- 172.21.15.69:0/2960795090 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9b0104050 con 0x7fc9b0108940 2024-04-25T11:17:08.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:08.951+0000 7fc99f7fe700 1 --2- 172.21.15.69:0/2960795090 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc99803d610 0x7fc99803fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:08.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:08.951+0000 7fc99f7fe700 1 -- 172.21.15.69:0/2960795090 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc9a001e070 con 0x7fc9b0108940 2024-04-25T11:17:08.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:08.951+0000 7fc9add9b700 1 --2- 172.21.15.69:0/2960795090 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc99803d610 0x7fc99803fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:08.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:08.951+0000 7fc9add9b700 1 --2- 172.21.15.69:0/2960795090 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc99803d610 0x7fc99803fb20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4006fd0 tx=0x7fc9a4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:08.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:08.956+0000 7fc99f7fe700 1 -- 172.21.15.69:0/2960795090 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc9a0013070 con 0x7fc9b0108940 2024-04-25T11:17:09.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:09.155+0000 7fc9b48ae700 1 -- 172.21.15.69:0/2960795090 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fc9b0100d40 con 0x7fc9b0108940 2024-04-25T11:17:09.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:09.633+0000 7fc99f7fe700 1 -- 172.21.15.69:0/2960795090 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fc9a0029db0 con 0x7fc9b0108940 2024-04-25T11:17:09.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:09.635+0000 7fc9b48ae700 1 -- 172.21.15.69:0/2960795090 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc99803d610 msgr2=0x7fc99803fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:09.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:09.635+0000 7fc9b48ae700 1 --2- 172.21.15.69:0/2960795090 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc99803d610 0x7fc99803fb20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4006fd0 tx=0x7fc9a4006e40 comp rx=0 tx=0).stop 2024-04-25T11:17:09.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:09.635+0000 7fc9b48ae700 1 -- 172.21.15.69:0/2960795090 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9b0108940 msgr2=0x7fc9b00ff810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:09.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:09.635+0000 7fc9b48ae700 1 --2- 172.21.15.69:0/2960795090 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9b0108940 0x7fc9b00ff810 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0005660 tx=0x7fc9a0005740 comp rx=0 tx=0).stop 2024-04-25T11:17:09.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:09.635+0000 7fc9b48ae700 1 -- 172.21.15.69:0/2960795090 shutdown_connections 2024-04-25T11:17:09.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:09.635+0000 7fc9b48ae700 1 --2- 172.21.15.69:0/2960795090 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc99803d610 0x7fc99803fb20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:09.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:09.635+0000 7fc9b48ae700 1 --2- 172.21.15.69:0/2960795090 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9b0108940 0x7fc9b00ff810 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:09.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:09.635+0000 7fc9b48ae700 1 -- 172.21.15.69:0/2960795090 >> 172.21.15.69:0/2960795090 conn(0x7fc9b0078460 msgr2=0x7fc9b0106420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:17:09.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:09.636+0000 7fc9b48ae700 1 -- 172.21.15.69:0/2960795090 shutdown_connections 2024-04-25T11:17:09.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:09.636+0000 7fc9b48ae700 1 -- 172.21.15.69:0/2960795090 wait complete. 2024-04-25T11:17:09.637 INFO:teuthology.orchestra.run.smithi069.stderr:adjusted tunables profile to default 2024-04-25T11:17:09.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:09 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2960795090' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T11:17:10.313 INFO:tasks.cephadm:Adding mon.a on smithi069 2024-04-25T11:17:10.313 INFO:tasks.cephadm:Adding mon.b on smithi146 2024-04-25T11:17:10.313 INFO:tasks.cephadm:Adding mon.c on smithi152 2024-04-25T11:17:10.313 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi069:172.21.15.69=a;smithi146:172.21.15.146=b;smithi152:172.21.15.152=c' 2024-04-25T11:17:10.551 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T11:17:10.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:10 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2960795090' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T11:17:10.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:10 smithi069 ceph-mon[29188]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T11:17:10.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:10 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:10.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:10 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:10.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:10 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:10.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:10 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:10.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:10 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi152", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:17:10.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:10 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:17:10.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:10 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:17:10.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:10 smithi069 ceph-mon[29188]: Updating smithi152:/etc/ceph/ceph.conf 2024-04-25T11:17:10.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:10 smithi069 ceph-mon[29188]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:10.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:10 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:10.951 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:10 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:10.951 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:10 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:11.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:11.938+0000 7fc6eddb7700 1 -- 172.21.15.152:0/1933056496 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6e8108960 msgr2=0x7fc6e8108d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:11.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:11.938+0000 7fc6eddb7700 1 --2- 172.21.15.152:0/1933056496 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6e8108960 0x7fc6e8108d70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc6e000b0a0 tx=0x7fc6e000b3b0 comp rx=0 tx=0).stop 2024-04-25T11:17:11.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:11.938+0000 7fc6eddb7700 1 -- 172.21.15.152:0/1933056496 shutdown_connections 2024-04-25T11:17:11.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:11.938+0000 7fc6eddb7700 1 --2- 172.21.15.152:0/1933056496 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6e8108960 0x7fc6e8108d70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:11.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:11.938+0000 7fc6eddb7700 1 -- 172.21.15.152:0/1933056496 >> 172.21.15.152:0/1933056496 conn(0x7fc6e8078480 msgr2=0x7fc6e80788e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:17:11.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:11.938+0000 7fc6eddb7700 1 -- 172.21.15.152:0/1933056496 shutdown_connections 2024-04-25T11:17:11.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:11.938+0000 7fc6eddb7700 1 -- 172.21.15.152:0/1933056496 wait complete. 2024-04-25T11:17:11.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:11.939+0000 7fc6eddb7700 1 Processor -- start 2024-04-25T11:17:11.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:11.939+0000 7fc6eddb7700 1 -- start start 2024-04-25T11:17:11.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:11.939+0000 7fc6eddb7700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6e8108960 0x7fc6e8198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:11.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:11.939+0000 7fc6eddb7700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e810cd60 con 0x7fc6e8108960 2024-04-25T11:17:11.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:11.939+0000 7fc6e77fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6e8108960 0x7fc6e8198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:11.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:11.939+0000 7fc6e77fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6e8108960 0x7fc6e8198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.152:42402/0 (socket says 172.21.15.152:42402) 2024-04-25T11:17:11.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:11.940+0000 7fc6e77fe700 1 -- 172.21.15.152:0/364263523 learned_addr learned my addr 172.21.15.152:0/364263523 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-25T11:17:11.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:11.940+0000 7fc6e77fe700 1 -- 172.21.15.152:0/364263523 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6e8198b60 con 0x7fc6e8108960 2024-04-25T11:17:11.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:11.940+0000 7fc6e77fe700 1 --2- 172.21.15.152:0/364263523 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6e8108960 0x7fc6e8198620 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0015040 tx=0x7fc6e0007940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:11.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:11.940+0000 7fc6e4ff9700 1 -- 172.21.15.152:0/364263523 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6e000bda0 con 0x7fc6e8108960 2024-04-25T11:17:11.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:11.940+0000 7fc6eddb7700 1 -- 172.21.15.152:0/364263523 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6e8198df0 con 0x7fc6e8108960 2024-04-25T11:17:11.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:11.940+0000 7fc6e4ff9700 1 -- 172.21.15.152:0/364263523 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6e0004040 con 0x7fc6e8108960 2024-04-25T11:17:11.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:11.940+0000 7fc6e4ff9700 1 -- 172.21.15.152:0/364263523 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6e0024480 con 0x7fc6e8108960 2024-04-25T11:17:11.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:11.940+0000 7fc6eddb7700 1 -- 172.21.15.152:0/364263523 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6e819ce50 con 0x7fc6e8108960 2024-04-25T11:17:11.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:11.941+0000 7fc6eddb7700 1 -- 172.21.15.152:0/364263523 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6e8104070 con 0x7fc6e8108960 2024-04-25T11:17:11.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:11.941+0000 7fc6e4ff9700 1 -- 172.21.15.152:0/364263523 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7fc6e0024780 con 0x7fc6e8108960 2024-04-25T11:17:11.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:11.941+0000 7fc6e4ff9700 1 --2- 172.21.15.152:0/364263523 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc6d403d660 0x7fc6d403fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:11.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:11.941+0000 7fc6e4ff9700 1 -- 172.21.15.152:0/364263523 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc6e000f050 con 0x7fc6e8108960 2024-04-25T11:17:11.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:11.942+0000 7fc6e6ffd700 1 --2- 172.21.15.152:0/364263523 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc6d403d660 0x7fc6d403fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:11.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:11.942+0000 7fc6e6ffd700 1 --2- 172.21.15.152:0/364263523 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc6d403d660 0x7fc6d403fb70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc6d0006fd0 tx=0x7fc6d0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:11.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:11.947+0000 7fc6e4ff9700 1 -- 172.21.15.152:0/364263523 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc6e0022380 con 0x7fc6e8108960 2024-04-25T11:17:11.949 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:11 smithi069 ceph-mon[29188]: Updating smithi152:/var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T11:17:11.949 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:11 smithi069 ceph-mon[29188]: Updating smithi152:/etc/ceph/ceph.client.admin.keyring 2024-04-25T11:17:11.949 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:11 smithi069 ceph-mon[29188]: Updating smithi152:/var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T11:17:12.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:12.154+0000 7fc6eddb7700 1 -- 172.21.15.152:0/364263523 --> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi069:172.21.15.69=a;smithi146:172.21.15.146=b;smithi152:172.21.15.152=c", "target": ["mon-mgr", ""]}) -- 0x7fc6e81999f0 con 0x7fc6d403d660 2024-04-25T11:17:12.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:12.161+0000 7fc6e4ff9700 1 -- 172.21.15.152:0/364263523 <== mgr.14152 v2:172.21.15.69:6800/4220065409 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fc6e81999f0 con 0x7fc6d403d660 2024-04-25T11:17:12.162 INFO:teuthology.orchestra.run.smithi152.stdout:Scheduled mon update... 2024-04-25T11:17:12.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:12.163+0000 7fc6eddb7700 1 -- 172.21.15.152:0/364263523 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc6d403d660 msgr2=0x7fc6d403fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:12.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:12.163+0000 7fc6eddb7700 1 --2- 172.21.15.152:0/364263523 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc6d403d660 0x7fc6d403fb70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc6d0006fd0 tx=0x7fc6d0006e40 comp rx=0 tx=0).stop 2024-04-25T11:17:12.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:12.163+0000 7fc6eddb7700 1 -- 172.21.15.152:0/364263523 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6e8108960 msgr2=0x7fc6e8198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:12.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:12.163+0000 7fc6eddb7700 1 --2- 172.21.15.152:0/364263523 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6e8108960 0x7fc6e8198620 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0015040 tx=0x7fc6e0007940 comp rx=0 tx=0).stop 2024-04-25T11:17:12.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:12.164+0000 7fc6eddb7700 1 -- 172.21.15.152:0/364263523 shutdown_connections 2024-04-25T11:17:12.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:12.164+0000 7fc6eddb7700 1 --2- 172.21.15.152:0/364263523 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc6d403d660 0x7fc6d403fb70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:12.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:12.164+0000 7fc6eddb7700 1 --2- 172.21.15.152:0/364263523 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6e8108960 0x7fc6e8198620 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:12.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:12.164+0000 7fc6eddb7700 1 -- 172.21.15.152:0/364263523 >> 172.21.15.152:0/364263523 conn(0x7fc6e8078480 msgr2=0x7fc6e80ff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:17:12.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:12.164+0000 7fc6eddb7700 1 -- 172.21.15.152:0/364263523 shutdown_connections 2024-04-25T11:17:12.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:12.164+0000 7fc6eddb7700 1 -- 172.21.15.152:0/364263523 wait complete. 2024-04-25T11:17:12.755 DEBUG:teuthology.orchestra.run.smithi146:mon.b> sudo journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mon.b.service 2024-04-25T11:17:12.758 DEBUG:teuthology.orchestra.run.smithi152:mon.c> sudo journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mon.c.service 2024-04-25T11:17:12.760 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T11:17:12.760 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T11:17:12.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:12 smithi069 ceph-mon[29188]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:12.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:12 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:12.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:12 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:17:12.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:12 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:17:12.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:12 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:17:12.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:12 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:12.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:12 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T11:17:12.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:12 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:17:13.849 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T11:17:13.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:13 smithi069 ceph-mon[29188]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi069:172.21.15.69=a;smithi146:172.21.15.146=b;smithi152:172.21.15.152=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:17:13.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:13 smithi069 ceph-mon[29188]: Saving service mon spec with placement smithi069:172.21.15.69=a;smithi146:172.21.15.146=b;smithi152:172.21.15.152=c;count:3 2024-04-25T11:17:13.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:13 smithi069 ceph-mon[29188]: Deploying daemon mon.c on smithi152 2024-04-25T11:17:14.949 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:14 smithi069 ceph-mon[29188]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:16.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:16.563+0000 7f2f12e3a700 1 -- 172.21.15.152:0/4043923949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f0c109a20 msgr2=0x7f2f0c109e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:16.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:16.563+0000 7f2f12e3a700 1 --2- 172.21.15.152:0/4043923949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f0c109a20 0x7f2f0c109e30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2f00009910 tx=0x7f2f00009c20 comp rx=0 tx=0).stop 2024-04-25T11:17:16.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:16.563+0000 7f2f12e3a700 1 -- 172.21.15.152:0/4043923949 shutdown_connections 2024-04-25T11:17:16.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:16.563+0000 7f2f12e3a700 1 --2- 172.21.15.152:0/4043923949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f0c109a20 0x7f2f0c109e30 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:16.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:16.563+0000 7f2f12e3a700 1 -- 172.21.15.152:0/4043923949 >> 172.21.15.152:0/4043923949 conn(0x7f2f0c0ff530 msgr2=0x7f2f0c101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:17:16.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:16.563+0000 7f2f12e3a700 1 -- 172.21.15.152:0/4043923949 shutdown_connections 2024-04-25T11:17:16.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:16.563+0000 7f2f12e3a700 1 -- 172.21.15.152:0/4043923949 wait complete. 2024-04-25T11:17:16.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:16.564+0000 7f2f12e3a700 1 Processor -- start 2024-04-25T11:17:16.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:16.564+0000 7f2f12e3a700 1 -- start start 2024-04-25T11:17:16.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:16.564+0000 7f2f12e3a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f0c109a20 0x7f2f0c199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:16.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:16.564+0000 7f2f12e3a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0c10de20 con 0x7f2f0c109a20 2024-04-25T11:17:16.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:16.564+0000 7f2f10bd6700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f0c109a20 0x7f2f0c199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:16.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:16.565+0000 7f2f10bd6700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f0c109a20 0x7f2f0c199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.152:56808/0 (socket says 172.21.15.152:56808) 2024-04-25T11:17:16.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:16.565+0000 7f2f10bd6700 1 -- 172.21.15.152:0/2945302902 learned_addr learned my addr 172.21.15.152:0/2945302902 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-25T11:17:16.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:16.565+0000 7f2f10bd6700 1 -- 172.21.15.152:0/2945302902 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f0c199c80 con 0x7f2f0c109a20 2024-04-25T11:17:16.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:16.565+0000 7f2f10bd6700 1 --2- 172.21.15.152:0/2945302902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f0c109a20 0x7f2f0c199740 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2f00005660 tx=0x7f2f00005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:16.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:16.565+0000 7f2f09ffb700 1 -- 172.21.15.152:0/2945302902 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f00026070 con 0x7f2f0c109a20 2024-04-25T11:17:16.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:16.565+0000 7f2f12e3a700 1 -- 172.21.15.152:0/2945302902 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f0c199f70 con 0x7f2f0c109a20 2024-04-25T11:17:16.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:16.566+0000 7f2f12e3a700 1 -- 172.21.15.152:0/2945302902 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f0c19df70 con 0x7f2f0c109a20 2024-04-25T11:17:16.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:16.566+0000 7f2f09ffb700 1 -- 172.21.15.152:0/2945302902 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f0000b6f0 con 0x7f2f0c109a20 2024-04-25T11:17:16.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:16.566+0000 7f2f09ffb700 1 -- 172.21.15.152:0/2945302902 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f000037f0 con 0x7f2f0c109a20 2024-04-25T11:17:16.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:16.567+0000 7f2f09ffb700 1 -- 172.21.15.152:0/2945302902 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f2f00029070 con 0x7f2f0c109a20 2024-04-25T11:17:16.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:16.567+0000 7f2f12e3a700 1 -- 172.21.15.152:0/2945302902 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ef0005330 con 0x7f2f0c109a20 2024-04-25T11:17:16.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:16.567+0000 7f2f09ffb700 1 --2- 172.21.15.152:0/2945302902 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f2efc03d670 0x7f2efc03fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:16.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:16.567+0000 7f2f09ffb700 1 -- 172.21.15.152:0/2945302902 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2f0001e070 con 0x7f2f0c109a20 2024-04-25T11:17:16.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:16.568+0000 7f2f0bfff700 1 --2- 172.21.15.152:0/2945302902 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f2efc03d670 0x7f2efc03fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:16.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:16.568+0000 7f2f0bfff700 1 --2- 172.21.15.152:0/2945302902 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f2efc03d670 0x7f2efc03fb80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8006fd0 tx=0x7f2ef8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:16.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:16.573+0000 7f2f09ffb700 1 -- 172.21.15.152:0/2945302902 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2f00013070 con 0x7f2f0c109a20 2024-04-25T11:17:16.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:16.892+0000 7f2f12e3a700 1 -- 172.21.15.152:0/2945302902 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f2ef0005620 con 0x7f2f0c109a20 2024-04-25T11:17:16.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:16.893+0000 7f2f09ffb700 1 -- 172.21.15.152:0/2945302902 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f2f00030300 con 0x7f2f0c109a20 2024-04-25T11:17:16.894 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-25T11:17:16.894 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":1,"fsid":"f7ed73dc-02f4-11ef-bc93-c7b262605968","modified":"2024-04-25T11:15:35.052240Z","created":"2024-04-25T11:15:35.052240Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:3300","nonce":0},{"type":"v1","addr":"172.21.15.69:6789","nonce":0}]},"addr":"172.21.15.69:6789/0","public_addr":"172.21.15.69:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T11:17:16.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:16.895+0000 7f2f12e3a700 1 -- 172.21.15.152:0/2945302902 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f2efc03d670 msgr2=0x7f2efc03fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:16.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:16.895+0000 7f2f12e3a700 1 --2- 172.21.15.152:0/2945302902 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f2efc03d670 0x7f2efc03fb80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8006fd0 tx=0x7f2ef8006e40 comp rx=0 tx=0).stop 2024-04-25T11:17:16.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:16.895+0000 7f2f12e3a700 1 -- 172.21.15.152:0/2945302902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f0c109a20 msgr2=0x7f2f0c199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:16.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:16.895+0000 7f2f12e3a700 1 --2- 172.21.15.152:0/2945302902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f0c109a20 0x7f2f0c199740 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2f00005660 tx=0x7f2f00005740 comp rx=0 tx=0).stop 2024-04-25T11:17:16.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:16.895+0000 7f2f12e3a700 1 -- 172.21.15.152:0/2945302902 shutdown_connections 2024-04-25T11:17:16.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:16.895+0000 7f2f12e3a700 1 --2- 172.21.15.152:0/2945302902 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f2efc03d670 0x7f2efc03fb80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:16.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:16.895+0000 7f2f12e3a700 1 --2- 172.21.15.152:0/2945302902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f0c109a20 0x7f2f0c199740 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:16.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:16.895+0000 7f2f12e3a700 1 -- 172.21.15.152:0/2945302902 >> 172.21.15.152:0/2945302902 conn(0x7f2f0c0ff530 msgr2=0x7f2f0c100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:17:16.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:16.895+0000 7f2f12e3a700 1 -- 172.21.15.152:0/2945302902 shutdown_connections 2024-04-25T11:17:16.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:16.895+0000 7f2f12e3a700 1 -- 172.21.15.152:0/2945302902 wait complete. 2024-04-25T11:17:16.896 INFO:teuthology.orchestra.run.smithi152.stderr:dumped monmap epoch 1 2024-04-25T11:17:17.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:16 smithi069 ceph-mon[29188]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:18.682 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T11:17:18.682 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T11:17:18.927 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T11:17:22.450 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:22 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T11:17:22.450 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:22 smithi069 ceph-mon[29188]: mon.a calling monitor election 2024-04-25T11:17:22.450 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:22 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:17:22.450 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:22 smithi069 ceph-mon[29188]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:22.451 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:22 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:17:22.451 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:22 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:17:22.451 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:22 smithi069 ceph-mon[29188]: mon.c calling monitor election 2024-04-25T11:17:22.451 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:22 smithi069 ceph-mon[29188]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:22.451 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:22 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:17:22.451 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:22 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:17:22.451 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:22 smithi069 ceph-mon[29188]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:22.451 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:22 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:17:22.451 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:22 smithi069 ceph-mon[29188]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T11:17:22.451 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:22 smithi069 ceph-mon[29188]: monmap epoch 2 2024-04-25T11:17:22.451 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:22 smithi069 ceph-mon[29188]: fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 2024-04-25T11:17:22.451 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:22 smithi069 ceph-mon[29188]: last_changed 2024-04-25T11:17:17.035845+0000 2024-04-25T11:17:22.451 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:22 smithi069 ceph-mon[29188]: created 2024-04-25T11:15:35.052240+0000 2024-04-25T11:17:22.452 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:22 smithi069 ceph-mon[29188]: min_mon_release 19 (squid) 2024-04-25T11:17:22.452 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:22 smithi069 ceph-mon[29188]: election_strategy: 1 2024-04-25T11:17:22.452 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:22 smithi069 ceph-mon[29188]: 0: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.a 2024-04-25T11:17:22.452 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:22 smithi069 ceph-mon[29188]: 1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.c 2024-04-25T11:17:22.452 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:22 smithi069 ceph-mon[29188]: fsmap 2024-04-25T11:17:22.452 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:22 smithi069 ceph-mon[29188]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T11:17:22.452 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:22 smithi069 ceph-mon[29188]: mgrmap e13: x(active, since 50s) 2024-04-25T11:17:22.452 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:22 smithi069 ceph-mon[29188]: overall HEALTH_OK 2024-04-25T11:17:22.452 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:22 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:22.452 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:22 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:22.452 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:22 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:22.452 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:22 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T11:17:22.452 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:22 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:17:22.487 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:22 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:17:23.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:23 smithi152 ceph-mon[32701]: Deploying daemon mon.b on smithi146 2024-04-25T11:17:23.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:23 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:17:23.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.337+0000 7f0ac8b31700 1 -- 172.21.15.152:0/89722564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0aac0035c0 msgr2=0x7f0aac005aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:23.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.337+0000 7f0ac8b31700 1 --2- 172.21.15.152:0/89722564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0aac0035c0 0x7f0aac005aa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0ac406b910 tx=0x7f0ab400fe60 comp rx=0 tx=0).stop 2024-04-25T11:17:23.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.337+0000 7f0ac8b31700 1 -- 172.21.15.152:0/89722564 shutdown_connections 2024-04-25T11:17:23.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.337+0000 7f0ac8b31700 1 --2- 172.21.15.152:0/89722564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0aac0035c0 0x7f0aac005aa0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:23.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.337+0000 7f0ac8b31700 1 -- 172.21.15.152:0/89722564 >> 172.21.15.152:0/89722564 conn(0x7f0ac4077ce0 msgr2=0x7f0ac4078120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:17:23.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.337+0000 7f0ac8b31700 1 -- 172.21.15.152:0/89722564 shutdown_connections 2024-04-25T11:17:23.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.337+0000 7f0ac8b31700 1 -- 172.21.15.152:0/89722564 wait complete. 2024-04-25T11:17:23.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.338+0000 7f0ac8b31700 1 Processor -- start 2024-04-25T11:17:23.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.338+0000 7f0ac8b31700 1 -- start start 2024-04-25T11:17:23.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.339+0000 7f0ac8b31700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0aac0035c0 0x7f0ac419f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:23.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.339+0000 7f0ac8b31700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ac419faa0 0x7f0ac41997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:23.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.339+0000 7f0ac8b31700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac410bad0 con 0x7f0ac419faa0 2024-04-25T11:17:23.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.339+0000 7f0ac8b31700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac410b950 con 0x7f0aac0035c0 2024-04-25T11:17:23.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.339+0000 7f0ac259c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0aac0035c0 0x7f0ac419f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:23.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.339+0000 7f0ac259c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0aac0035c0 0x7f0ac419f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49010/0 (socket says 172.21.15.152:49010) 2024-04-25T11:17:23.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.339+0000 7f0ac259c700 1 -- 172.21.15.152:0/2402418965 learned_addr learned my addr 172.21.15.152:0/2402418965 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-25T11:17:23.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.339+0000 7f0ac259c700 1 -- 172.21.15.152:0/2402418965 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0aac0035c0 msgr2=0x7f0ac419f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-25T11:17:23.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.339+0000 7f0ac259c700 1 -- 172.21.15.152:0/2402418965 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0aac0035c0 msgr2=0x7f0ac419f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T11:17:23.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.339+0000 7f0ac259c700 1 --2- 172.21.15.152:0/2402418965 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0aac0035c0 0x7f0ac419f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:17:23.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.339+0000 7f0ac1d9b700 1 --2- 172.21.15.152:0/2402418965 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ac419faa0 0x7f0ac41997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:23.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.339+0000 7f0ac259c700 1 --2- 172.21.15.152:0/2402418965 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0aac0035c0 0x7f0ac419f560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T11:17:23.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.340+0000 7f0ac1d9b700 1 -- 172.21.15.152:0/2402418965 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0aac0035c0 msgr2=0x7f0ac419f560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:17:23.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.340+0000 7f0ac1d9b700 1 --2- 172.21.15.152:0/2402418965 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0aac0035c0 0x7f0ac419f560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:23.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.340+0000 7f0ac1d9b700 1 -- 172.21.15.152:0/2402418965 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac41a0090 con 0x7f0ac419faa0 2024-04-25T11:17:23.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.340+0000 7f0ac1d9b700 1 --2- 172.21.15.152:0/2402418965 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ac419faa0 0x7f0ac41997a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0ab800b4d0 tx=0x7f0ab800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:23.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.340+0000 7f0ab37fe700 1 -- 172.21.15.152:0/2402418965 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f0ab8011630 con 0x7f0ac419faa0 2024-04-25T11:17:23.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.340+0000 7f0ab37fe700 1 -- 172.21.15.152:0/2402418965 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0ab8011c60 con 0x7f0ac419faa0 2024-04-25T11:17:23.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.340+0000 7f0ab37fe700 1 -- 172.21.15.152:0/2402418965 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f0ab8010c90 con 0x7f0ac419faa0 2024-04-25T11:17:23.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.340+0000 7f0ac8b31700 1 -- 172.21.15.152:0/2402418965 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ac4199da0 con 0x7f0ac419faa0 2024-04-25T11:17:23.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.340+0000 7f0ac8b31700 1 -- 172.21.15.152:0/2402418965 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ac419a330 con 0x7f0ac419faa0 2024-04-25T11:17:23.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.341+0000 7f0ac8b31700 1 -- 172.21.15.152:0/2402418965 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ac419a640 con 0x7f0ac419faa0 2024-04-25T11:17:23.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.342+0000 7f0ab37fe700 1 -- 172.21.15.152:0/2402418965 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f0ab8018020 con 0x7f0ac419faa0 2024-04-25T11:17:23.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.342+0000 7f0ab37fe700 1 --2- 172.21.15.152:0/2402418965 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f0aa403d420 0x7f0aa403f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:23.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.342+0000 7f0ab37fe700 1 -- 172.21.15.152:0/2402418965 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0ab8051090 con 0x7f0ac419faa0 2024-04-25T11:17:23.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.342+0000 7f0ac259c700 1 --2- 172.21.15.152:0/2402418965 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f0aa403d420 0x7f0aa403f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:23.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.343+0000 7f0ac259c700 1 --2- 172.21.15.152:0/2402418965 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f0aa403d420 0x7f0aa403f930 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0ab400f5a0 tx=0x7f0ab4020040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:23.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.348+0000 7f0ab37fe700 1 -- 172.21.15.152:0/2402418965 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0ab801f020 con 0x7f0ac419faa0 2024-04-25T11:17:23.450 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:23 smithi069 ceph-mon[29188]: Deploying daemon mon.b on smithi146 2024-04-25T11:17:23.450 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:23 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:17:23.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.638+0000 7f0ac8b31700 1 -- 172.21.15.152:0/2402418965 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f0ac419acb0 con 0x7f0ac419faa0 2024-04-25T11:17:23.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.639+0000 7f0ab37fe700 1 -- 172.21.15.152:0/2402418965 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f0ab80187b0 con 0x7f0ac419faa0 2024-04-25T11:17:23.640 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-25T11:17:23.641 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":2,"fsid":"f7ed73dc-02f4-11ef-bc93-c7b262605968","modified":"2024-04-25T11:17:17.035845Z","created":"2024-04-25T11:15:35.052240Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:3300","nonce":0},{"type":"v1","addr":"172.21.15.69:6789","nonce":0}]},"addr":"172.21.15.69:6789/0","public_addr":"172.21.15.69:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T11:17:23.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.641+0000 7f0ac8b31700 1 -- 172.21.15.152:0/2402418965 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f0aa403d420 msgr2=0x7f0aa403f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:23.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.641+0000 7f0ac8b31700 1 --2- 172.21.15.152:0/2402418965 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f0aa403d420 0x7f0aa403f930 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0ab400f5a0 tx=0x7f0ab4020040 comp rx=0 tx=0).stop 2024-04-25T11:17:23.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.641+0000 7f0ac8b31700 1 -- 172.21.15.152:0/2402418965 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ac419faa0 msgr2=0x7f0ac41997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:23.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.641+0000 7f0ac8b31700 1 --2- 172.21.15.152:0/2402418965 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ac419faa0 0x7f0ac41997a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0ab800b4d0 tx=0x7f0ab800b890 comp rx=0 tx=0).stop 2024-04-25T11:17:23.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.641+0000 7f0ac8b31700 1 -- 172.21.15.152:0/2402418965 shutdown_connections 2024-04-25T11:17:23.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.641+0000 7f0ac8b31700 1 --2- 172.21.15.152:0/2402418965 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f0aa403d420 0x7f0aa403f930 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:23.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.641+0000 7f0ac8b31700 1 --2- 172.21.15.152:0/2402418965 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0aac0035c0 0x7f0ac419f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:23.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.641+0000 7f0ac8b31700 1 --2- 172.21.15.152:0/2402418965 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ac419faa0 0x7f0ac41997a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:23.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.641+0000 7f0ac8b31700 1 -- 172.21.15.152:0/2402418965 >> 172.21.15.152:0/2402418965 conn(0x7f0ac4077ce0 msgr2=0x7f0ac4100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:17:23.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.641+0000 7f0ac8b31700 1 -- 172.21.15.152:0/2402418965 shutdown_connections 2024-04-25T11:17:23.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:23.641+0000 7f0ac8b31700 1 -- 172.21.15.152:0/2402418965 wait complete. 2024-04-25T11:17:23.644 INFO:teuthology.orchestra.run.smithi152.stderr:dumped monmap epoch 2 2024-04-25T11:17:24.194 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:24 smithi152 ceph-mon[32701]: from='client.? 172.21.15.152:0/2402418965' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T11:17:24.194 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:24 smithi152 ceph-mon[32701]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:24.450 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:24 smithi069 ceph-mon[29188]: from='client.? 172.21.15.152:0/2402418965' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T11:17:24.450 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:24 smithi069 ceph-mon[29188]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:25.222 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T11:17:25.223 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T11:17:25.466 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T11:17:26.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:26 smithi146 systemd[1]: Started Ceph mon.b for f7ed73dc-02f4-11ef-bc93-c7b262605968. 2024-04-25T11:17:31.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.348+0000 7f292dd31700 1 -- 172.21.15.152:0/156022 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29100057d0 msgr2=0x7f2910007cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:31.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.348+0000 7f292dd31700 1 --2- 172.21.15.152:0/156022 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29100057d0 0x7f2910007cb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2928102940 tx=0x7f291800fe60 comp rx=0 tx=0).stop 2024-04-25T11:17:31.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.348+0000 7f292dd31700 1 -- 172.21.15.152:0/156022 shutdown_connections 2024-04-25T11:17:31.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.348+0000 7f292dd31700 1 --2- 172.21.15.152:0/156022 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29100057d0 0x7f2910007cb0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:31.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.348+0000 7f292dd31700 1 -- 172.21.15.152:0/156022 >> 172.21.15.152:0/156022 conn(0x7f29280796b0 msgr2=0x7f2928079af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:17:31.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.348+0000 7f292dd31700 1 -- 172.21.15.152:0/156022 shutdown_connections 2024-04-25T11:17:31.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.348+0000 7f292dd31700 1 -- 172.21.15.152:0/156022 wait complete. 2024-04-25T11:17:31.351 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:31 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T11:17:31.351 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:31 smithi152 ceph-mon[32701]: mon.a calling monitor election 2024-04-25T11:17:31.351 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:31 smithi152 ceph-mon[32701]: mon.c calling monitor election 2024-04-25T11:17:31.351 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:31 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:17:31.351 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:31 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:17:31.351 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:31 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:17:31.351 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:31 smithi152 ceph-mon[32701]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:31.351 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:31 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:17:31.351 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:31 smithi152 ceph-mon[32701]: mon.b calling monitor election 2024-04-25T11:17:31.352 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:31 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:17:31.352 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:31 smithi152 ceph-mon[32701]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:31.352 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:31 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:17:31.352 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:31 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:17:31.352 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:31 smithi152 ceph-mon[32701]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T11:17:31.352 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:31 smithi152 ceph-mon[32701]: monmap epoch 3 2024-04-25T11:17:31.352 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:31 smithi152 ceph-mon[32701]: fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 2024-04-25T11:17:31.352 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:31 smithi152 ceph-mon[32701]: last_changed 2024-04-25T11:17:26.030502+0000 2024-04-25T11:17:31.352 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:31 smithi152 ceph-mon[32701]: created 2024-04-25T11:15:35.052240+0000 2024-04-25T11:17:31.352 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:31 smithi152 ceph-mon[32701]: min_mon_release 19 (squid) 2024-04-25T11:17:31.352 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:31 smithi152 ceph-mon[32701]: election_strategy: 1 2024-04-25T11:17:31.352 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:31 smithi152 ceph-mon[32701]: 0: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.a 2024-04-25T11:17:31.352 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:31 smithi152 ceph-mon[32701]: 1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.c 2024-04-25T11:17:31.352 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:31 smithi152 ceph-mon[32701]: 2: [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] mon.b 2024-04-25T11:17:31.353 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:31 smithi152 ceph-mon[32701]: fsmap 2024-04-25T11:17:31.353 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:31 smithi152 ceph-mon[32701]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T11:17:31.353 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:31 smithi152 ceph-mon[32701]: mgrmap e13: x(active, since 59s) 2024-04-25T11:17:31.353 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:31 smithi152 ceph-mon[32701]: overall HEALTH_OK 2024-04-25T11:17:31.353 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:31 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:31.353 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:31 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:31.353 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:31 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:31.353 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:31 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:31.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.349+0000 7f292dd31700 1 Processor -- start 2024-04-25T11:17:31.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.349+0000 7f292dd31700 1 -- start start 2024-04-25T11:17:31.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.349+0000 7f292dd31700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29100057d0 0x7f292819d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:31.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.349+0000 7f292dd31700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f292819da90 0x7f29281976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:31.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.349+0000 7f292dd31700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2928197c20 0x7f2928198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:31.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.349+0000 7f292dd31700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f29281098b0 con 0x7f29100057d0 2024-04-25T11:17:31.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.349+0000 7f292dd31700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f292806c9d0 con 0x7f292819da90 2024-04-25T11:17:31.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.350+0000 7f292dd31700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2928109a30 con 0x7f2928197c20 2024-04-25T11:17:31.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.350+0000 7f2927fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2928197c20 0x7f2928198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:31.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.350+0000 7f2927fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2928197c20 0x7f2928198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36874/0 (socket says 172.21.15.152:36874) 2024-04-25T11:17:31.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.350+0000 7f2927fff700 1 -- 172.21.15.152:0/1020280238 learned_addr learned my addr 172.21.15.152:0/1020280238 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-25T11:17:31.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.350+0000 7f29277fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29100057d0 0x7f292819d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:31.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.350+0000 7f2926ffd700 1 --2- 172.21.15.152:0/1020280238 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f292819da90 0x7f29281976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:31.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.350+0000 7f2927fff700 1 -- 172.21.15.152:0/1020280238 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2928197c20 msgr2=0x7f2928198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-04-25T11:17:31.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.350+0000 7f2927fff700 1 -- 172.21.15.152:0/1020280238 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2928197c20 msgr2=0x7f2928198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T11:17:31.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.350+0000 7f2927fff700 1 --2- 172.21.15.152:0/1020280238 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2928197c20 0x7f2928198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:17:31.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.350+0000 7f2926ffd700 1 -- 172.21.15.152:0/1020280238 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f292819da90 msgr2=0x7f29281976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-25T11:17:31.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.350+0000 7f2926ffd700 1 -- 172.21.15.152:0/1020280238 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f292819da90 msgr2=0x7f29281976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T11:17:31.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.350+0000 7f2926ffd700 1 --2- 172.21.15.152:0/1020280238 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f292819da90 0x7f29281976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:17:31.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.350+0000 7f2926ffd700 1 --2- 172.21.15.152:0/1020280238 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f292819da90 0x7f29281976e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T11:17:31.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.350+0000 7f2927fff700 1 --2- 172.21.15.152:0/1020280238 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2928197c20 0x7f2928198030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T11:17:31.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.350+0000 7f29277fe700 1 -- 172.21.15.152:0/1020280238 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2928197c20 msgr2=0x7f2928198030 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:17:31.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.350+0000 7f29277fe700 1 --2- 172.21.15.152:0/1020280238 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2928197c20 0x7f2928198030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:31.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.350+0000 7f29277fe700 1 -- 172.21.15.152:0/1020280238 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f292819da90 msgr2=0x7f29281976e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:17:31.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.350+0000 7f29277fe700 1 --2- 172.21.15.152:0/1020280238 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f292819da90 0x7f29281976e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:31.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.350+0000 7f29277fe700 1 -- 172.21.15.152:0/1020280238 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29281a4300 con 0x7f29100057d0 2024-04-25T11:17:31.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.350+0000 7f29277fe700 1 --2- 172.21.15.152:0/1020280238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29100057d0 0x7f292819d550 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2928102940 tx=0x7f2918014fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:31.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.351+0000 7f2924ff9700 1 -- 172.21.15.152:0/1020280238 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2918022040 con 0x7f29100057d0 2024-04-25T11:17:31.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.351+0000 7f2924ff9700 1 -- 172.21.15.152:0/1020280238 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f291802c070 con 0x7f29100057d0 2024-04-25T11:17:31.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.351+0000 7f292dd31700 1 -- 172.21.15.152:0/1020280238 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29281a4590 con 0x7f29100057d0 2024-04-25T11:17:31.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.351+0000 7f2924ff9700 1 -- 172.21.15.152:0/1020280238 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2918003b00 con 0x7f29100057d0 2024-04-25T11:17:31.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.351+0000 7f292dd31700 1 -- 172.21.15.152:0/1020280238 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29281a4af0 con 0x7f29100057d0 2024-04-25T11:17:31.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.352+0000 7f292dd31700 1 -- 172.21.15.152:0/1020280238 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2928104150 con 0x7f29100057d0 2024-04-25T11:17:31.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.352+0000 7f2924ff9700 1 -- 172.21.15.152:0/1020280238 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f2918031460 con 0x7f29100057d0 2024-04-25T11:17:31.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.352+0000 7f2924ff9700 1 --2- 172.21.15.152:0/1020280238 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f291403daa0 0x7f291403ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:31.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.352+0000 7f2924ff9700 1 -- 172.21.15.152:0/1020280238 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2918053ed0 con 0x7f29100057d0 2024-04-25T11:17:31.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.353+0000 7f2926ffd700 1 --2- 172.21.15.152:0/1020280238 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f291403daa0 0x7f291403ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:31.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.353+0000 7f2926ffd700 1 --2- 172.21.15.152:0/1020280238 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f291403daa0 0x7f291403ffb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2928198e80 tx=0x7f291c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:31.359 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:31 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T11:17:31.359 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:31 smithi146 ceph-mon[32821]: mon.a calling monitor election 2024-04-25T11:17:31.359 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:31 smithi146 ceph-mon[32821]: mon.c calling monitor election 2024-04-25T11:17:31.359 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:31 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:17:31.359 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:31 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:17:31.360 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:31 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:17:31.360 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:31 smithi146 ceph-mon[32821]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:31.360 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:31 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:17:31.360 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:31 smithi146 ceph-mon[32821]: mon.b calling monitor election 2024-04-25T11:17:31.360 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:31 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:17:31.360 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:31 smithi146 ceph-mon[32821]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:31.360 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:31 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:17:31.360 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:31 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:17:31.360 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:31 smithi146 ceph-mon[32821]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T11:17:31.360 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:31 smithi146 ceph-mon[32821]: monmap epoch 3 2024-04-25T11:17:31.360 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:31 smithi146 ceph-mon[32821]: fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 2024-04-25T11:17:31.360 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:31 smithi146 ceph-mon[32821]: last_changed 2024-04-25T11:17:26.030502+0000 2024-04-25T11:17:31.360 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:31 smithi146 ceph-mon[32821]: created 2024-04-25T11:15:35.052240+0000 2024-04-25T11:17:31.360 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:31 smithi146 ceph-mon[32821]: min_mon_release 19 (squid) 2024-04-25T11:17:31.361 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:31 smithi146 ceph-mon[32821]: election_strategy: 1 2024-04-25T11:17:31.361 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:31 smithi146 ceph-mon[32821]: 0: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.a 2024-04-25T11:17:31.361 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:31 smithi146 ceph-mon[32821]: 1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.c 2024-04-25T11:17:31.361 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:31 smithi146 ceph-mon[32821]: 2: [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] mon.b 2024-04-25T11:17:31.361 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:31 smithi146 ceph-mon[32821]: fsmap 2024-04-25T11:17:31.361 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:31 smithi146 ceph-mon[32821]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T11:17:31.361 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:31 smithi146 ceph-mon[32821]: mgrmap e13: x(active, since 59s) 2024-04-25T11:17:31.361 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:31 smithi146 ceph-mon[32821]: overall HEALTH_OK 2024-04-25T11:17:31.361 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:31 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:31.361 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:31 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:31.361 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:31 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:31.361 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:31 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:31.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.358+0000 7f2924ff9700 1 -- 172.21.15.152:0/1020280238 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2918026c30 con 0x7f29100057d0 2024-04-25T11:17:31.450 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:31 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T11:17:31.450 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:31 smithi069 ceph-mon[29188]: mon.a calling monitor election 2024-04-25T11:17:31.451 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:31 smithi069 ceph-mon[29188]: mon.c calling monitor election 2024-04-25T11:17:31.451 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:31 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:17:31.451 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:31 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:17:31.451 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:31 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:17:31.451 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:31 smithi069 ceph-mon[29188]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:31.451 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:31 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:17:31.451 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:31 smithi069 ceph-mon[29188]: mon.b calling monitor election 2024-04-25T11:17:31.451 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:31 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:17:31.451 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:31 smithi069 ceph-mon[29188]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:31.451 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:31 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:17:31.452 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:31 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:17:31.452 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:31 smithi069 ceph-mon[29188]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T11:17:31.452 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:31 smithi069 ceph-mon[29188]: monmap epoch 3 2024-04-25T11:17:31.452 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:31 smithi069 ceph-mon[29188]: fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 2024-04-25T11:17:31.452 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:31 smithi069 ceph-mon[29188]: last_changed 2024-04-25T11:17:26.030502+0000 2024-04-25T11:17:31.452 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:31 smithi069 ceph-mon[29188]: created 2024-04-25T11:15:35.052240+0000 2024-04-25T11:17:31.452 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:31 smithi069 ceph-mon[29188]: min_mon_release 19 (squid) 2024-04-25T11:17:31.452 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:31 smithi069 ceph-mon[29188]: election_strategy: 1 2024-04-25T11:17:31.452 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:31 smithi069 ceph-mon[29188]: 0: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.a 2024-04-25T11:17:31.452 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:31 smithi069 ceph-mon[29188]: 1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.c 2024-04-25T11:17:31.452 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:31 smithi069 ceph-mon[29188]: 2: [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] mon.b 2024-04-25T11:17:31.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:31 smithi069 ceph-mon[29188]: fsmap 2024-04-25T11:17:31.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:31 smithi069 ceph-mon[29188]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T11:17:31.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:31 smithi069 ceph-mon[29188]: mgrmap e13: x(active, since 59s) 2024-04-25T11:17:31.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:31 smithi069 ceph-mon[29188]: overall HEALTH_OK 2024-04-25T11:17:31.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:31 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:31.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:31 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:31.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:31 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:31.453 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:31 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:31.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.645+0000 7f292dd31700 1 -- 172.21.15.152:0/1020280238 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f292802e510 con 0x7f29100057d0 2024-04-25T11:17:31.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.646+0000 7f2924ff9700 1 -- 172.21.15.152:0/1020280238 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1285 (secure 0 0 0) 0x7f2918055020 con 0x7f29100057d0 2024-04-25T11:17:31.648 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-04-25T11:17:31.648 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":3,"fsid":"f7ed73dc-02f4-11ef-bc93-c7b262605968","modified":"2024-04-25T11:17:26.030502Z","created":"2024-04-25T11:15:35.052240Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:3300","nonce":0},{"type":"v1","addr":"172.21.15.69:6789","nonce":0}]},"addr":"172.21.15.69:6789/0","public_addr":"172.21.15.69:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:3300","nonce":0},{"type":"v1","addr":"172.21.15.146:6789","nonce":0}]},"addr":"172.21.15.146:6789/0","public_addr":"172.21.15.146:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-04-25T11:17:31.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.648+0000 7f292dd31700 1 -- 172.21.15.152:0/1020280238 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f291403daa0 msgr2=0x7f291403ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:31.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.648+0000 7f292dd31700 1 --2- 172.21.15.152:0/1020280238 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f291403daa0 0x7f291403ffb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2928198e80 tx=0x7f291c008040 comp rx=0 tx=0).stop 2024-04-25T11:17:31.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.648+0000 7f292dd31700 1 -- 172.21.15.152:0/1020280238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29100057d0 msgr2=0x7f292819d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:31.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.648+0000 7f292dd31700 1 --2- 172.21.15.152:0/1020280238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29100057d0 0x7f292819d550 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2928102940 tx=0x7f2918014fc0 comp rx=0 tx=0).stop 2024-04-25T11:17:31.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.648+0000 7f292dd31700 1 -- 172.21.15.152:0/1020280238 shutdown_connections 2024-04-25T11:17:31.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.648+0000 7f292dd31700 1 --2- 172.21.15.152:0/1020280238 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f291403daa0 0x7f291403ffb0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:31.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.648+0000 7f292dd31700 1 --2- 172.21.15.152:0/1020280238 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2928197c20 0x7f2928198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:31.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.648+0000 7f292dd31700 1 --2- 172.21.15.152:0/1020280238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29100057d0 0x7f292819d550 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:31.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.648+0000 7f292dd31700 1 --2- 172.21.15.152:0/1020280238 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f292819da90 0x7f29281976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:31.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.648+0000 7f292dd31700 1 -- 172.21.15.152:0/1020280238 >> 172.21.15.152:0/1020280238 conn(0x7f29280796b0 msgr2=0x7f2928102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:17:31.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.648+0000 7f292dd31700 1 -- 172.21.15.152:0/1020280238 shutdown_connections 2024-04-25T11:17:31.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:31.648+0000 7f292dd31700 1 -- 172.21.15.152:0/1020280238 wait complete. 2024-04-25T11:17:31.651 INFO:teuthology.orchestra.run.smithi152.stderr:dumped monmap epoch 3 2024-04-25T11:17:32.214 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:32 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:17:32.214 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:32 smithi152 ceph-mon[32701]: from='client.? 172.21.15.152:0/1020280238' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T11:17:32.214 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:32 smithi152 ceph-mon[32701]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:32.214 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:32 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:32.214 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:32 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:17:32.241 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-25T11:17:32.241 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-25T11:17:32.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:32 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:17:32.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:32 smithi146 ceph-mon[32821]: from='client.? 172.21.15.152:0/1020280238' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T11:17:32.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:32 smithi146 ceph-mon[32821]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:32.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:32 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:32.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:32 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:17:32.450 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:32 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:17:32.450 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:32 smithi069 ceph-mon[29188]: from='client.? 172.21.15.152:0/1020280238' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T11:17:32.450 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:32 smithi069 ceph-mon[29188]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:32.450 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:32 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:32.450 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:32 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:17:32.483 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:17:33.699 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:33 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:33.700 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:33 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:33.700 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:33 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:33.700 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:33 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:33.700 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:33 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:17:33.700 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:33 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:17:33.737 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:33 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:33.737 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:33 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:33.737 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:33 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:33.737 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:33 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:33.737 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:33 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:17:33.737 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:33 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:17:33.855 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:33 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:33.856 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:33 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:33.856 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:33 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:33.856 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:33 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:33.856 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:33 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:17:33.856 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:33 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:17:34.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.051+0000 7f6940f7e700 1 -- 172.21.15.69:0/1516793458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f693c108940 msgr2=0x7f693c108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:34.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.051+0000 7f6940f7e700 1 --2- 172.21.15.69:0/1516793458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f693c108940 0x7f693c108d50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f692c009910 tx=0x7f692c009c20 comp rx=0 tx=0).stop 2024-04-25T11:17:34.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.051+0000 7f6940f7e700 1 -- 172.21.15.69:0/1516793458 shutdown_connections 2024-04-25T11:17:34.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.051+0000 7f6940f7e700 1 --2- 172.21.15.69:0/1516793458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f693c108940 0x7f693c108d50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:34.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.051+0000 7f6940f7e700 1 -- 172.21.15.69:0/1516793458 >> 172.21.15.69:0/1516793458 conn(0x7f693c078460 msgr2=0x7f693c0788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:17:34.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.052+0000 7f6940f7e700 1 -- 172.21.15.69:0/1516793458 shutdown_connections 2024-04-25T11:17:34.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.052+0000 7f6940f7e700 1 -- 172.21.15.69:0/1516793458 wait complete. 2024-04-25T11:17:34.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.052+0000 7f6940f7e700 1 Processor -- start 2024-04-25T11:17:34.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.052+0000 7f6940f7e700 1 -- start start 2024-04-25T11:17:34.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.053+0000 7f6940f7e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f693c108940 0x7f693c19a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:34.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.053+0000 7f6940f7e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f693c19ae40 0x7f693c19b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:34.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.053+0000 7f6940f7e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f693c1a3970 0x7f693c1a5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:34.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.053+0000 7f6940f7e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f693c10cd00 con 0x7f693c108940 2024-04-25T11:17:34.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.053+0000 7f6940f7e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f693c10cb80 con 0x7f693c1a3970 2024-04-25T11:17:34.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.053+0000 7f6940f7e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f693c10ce80 con 0x7f693c19ae40 2024-04-25T11:17:34.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.053+0000 7f693a59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f693c108940 0x7f693c19a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:34.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.053+0000 7f693a59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f693c108940 0x7f693c19a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43004/0 (socket says 172.21.15.69:43004) 2024-04-25T11:17:34.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.053+0000 7f693a59c700 1 -- 172.21.15.69:0/2151299014 learned_addr learned my addr 172.21.15.69:0/2151299014 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:17:34.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.053+0000 7f6939d9b700 1 --2- 172.21.15.69:0/2151299014 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f693c19ae40 0x7f693c19b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:34.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.053+0000 7f693ad9d700 1 --2- 172.21.15.69:0/2151299014 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f693c1a3970 0x7f693c1a5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:34.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.053+0000 7f693a59c700 1 -- 172.21.15.69:0/2151299014 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f693c19ae40 msgr2=0x7f693c19b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:34.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.053+0000 7f693a59c700 1 --2- 172.21.15.69:0/2151299014 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f693c19ae40 0x7f693c19b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:34.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.053+0000 7f693a59c700 1 -- 172.21.15.69:0/2151299014 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f693c1a3970 msgr2=0x7f693c1a5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:34.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.053+0000 7f693a59c700 1 --2- 172.21.15.69:0/2151299014 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f693c1a3970 0x7f693c1a5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:34.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.053+0000 7f693a59c700 1 -- 172.21.15.69:0/2151299014 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f693c1a6520 con 0x7f693c108940 2024-04-25T11:17:34.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.053+0000 7f693ad9d700 1 --2- 172.21.15.69:0/2151299014 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f693c1a3970 0x7f693c1a5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:17:34.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.053+0000 7f693a59c700 1 --2- 172.21.15.69:0/2151299014 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f693c108940 0x7f693c19a900 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f692c000c00 tx=0x7f692c00f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:34.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.054+0000 7f692b7fe700 1 -- 172.21.15.69:0/2151299014 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c027070 con 0x7f693c108940 2024-04-25T11:17:34.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.054+0000 7f692b7fe700 1 -- 172.21.15.69:0/2151299014 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f692c005da0 con 0x7f693c108940 2024-04-25T11:17:34.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.054+0000 7f6940f7e700 1 -- 172.21.15.69:0/2151299014 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f693c1a6810 con 0x7f693c108940 2024-04-25T11:17:34.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.054+0000 7f692b7fe700 1 -- 172.21.15.69:0/2151299014 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c017440 con 0x7f693c108940 2024-04-25T11:17:34.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.054+0000 7f6940f7e700 1 -- 172.21.15.69:0/2151299014 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f693c1a6da0 con 0x7f693c108940 2024-04-25T11:17:34.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.055+0000 7f6940f7e700 1 -- 172.21.15.69:0/2151299014 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f693c104050 con 0x7f693c108940 2024-04-25T11:17:34.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.055+0000 7f692b7fe700 1 -- 172.21.15.69:0/2151299014 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f692c02a070 con 0x7f693c108940 2024-04-25T11:17:34.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.055+0000 7f692b7fe700 1 --2- 172.21.15.69:0/2151299014 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f692403daa0 0x7f692403ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:34.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.055+0000 7f692b7fe700 1 -- 172.21.15.69:0/2151299014 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f692c052620 con 0x7f693c108940 2024-04-25T11:17:34.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.055+0000 7f6939d9b700 1 --2- 172.21.15.69:0/2151299014 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f692403daa0 0x7f692403ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:34.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.056+0000 7f6939d9b700 1 --2- 172.21.15.69:0/2151299014 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f692403daa0 0x7f692403ffb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f693c10b010 tx=0x7f6930008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:34.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.061+0000 7f692b7fe700 1 -- 172.21.15.69:0/2151299014 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f692c013070 con 0x7f693c108940 2024-04-25T11:17:34.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.266+0000 7f6940f7e700 1 -- 172.21.15.69:0/2151299014 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f693c064cf0 con 0x7f693c108940 2024-04-25T11:17:34.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.267+0000 7f692b7fe700 1 -- 172.21.15.69:0/2151299014 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f692c0142b0 con 0x7f693c108940 2024-04-25T11:17:34.267 INFO:teuthology.orchestra.run.smithi069.stdout:# minimal ceph.conf for f7ed73dc-02f4-11ef-bc93-c7b262605968 2024-04-25T11:17:34.268 INFO:teuthology.orchestra.run.smithi069.stdout:[global] 2024-04-25T11:17:34.268 INFO:teuthology.orchestra.run.smithi069.stdout: fsid = f7ed73dc-02f4-11ef-bc93-c7b262605968 2024-04-25T11:17:34.268 INFO:teuthology.orchestra.run.smithi069.stdout: mon_host = [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] 2024-04-25T11:17:34.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.269+0000 7f6940f7e700 1 -- 172.21.15.69:0/2151299014 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f692403daa0 msgr2=0x7f692403ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:34.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.269+0000 7f6940f7e700 1 --2- 172.21.15.69:0/2151299014 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f692403daa0 0x7f692403ffb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f693c10b010 tx=0x7f6930008040 comp rx=0 tx=0).stop 2024-04-25T11:17:34.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.269+0000 7f6940f7e700 1 -- 172.21.15.69:0/2151299014 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f693c108940 msgr2=0x7f693c19a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:34.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.269+0000 7f6940f7e700 1 --2- 172.21.15.69:0/2151299014 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f693c108940 0x7f693c19a900 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f692c000c00 tx=0x7f692c00f740 comp rx=0 tx=0).stop 2024-04-25T11:17:34.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.269+0000 7f6940f7e700 1 -- 172.21.15.69:0/2151299014 shutdown_connections 2024-04-25T11:17:34.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.269+0000 7f6940f7e700 1 --2- 172.21.15.69:0/2151299014 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f692403daa0 0x7f692403ffb0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:34.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.269+0000 7f6940f7e700 1 --2- 172.21.15.69:0/2151299014 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f693c1a3970 0x7f693c1a5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:34.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.269+0000 7f6940f7e700 1 --2- 172.21.15.69:0/2151299014 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f693c108940 0x7f693c19a900 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:34.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.269+0000 7f6940f7e700 1 --2- 172.21.15.69:0/2151299014 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f693c19ae40 0x7f693c19b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:34.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.269+0000 7f6940f7e700 1 -- 172.21.15.69:0/2151299014 >> 172.21.15.69:0/2151299014 conn(0x7f693c078460 msgr2=0x7f693c0ff920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:17:34.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.269+0000 7f6940f7e700 1 -- 172.21.15.69:0/2151299014 shutdown_connections 2024-04-25T11:17:34.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:34.269+0000 7f6940f7e700 1 -- 172.21.15.69:0/2151299014 wait complete. 2024-04-25T11:17:34.666 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:34 smithi069 ceph-mon[29188]: Updating smithi069:/etc/ceph/ceph.conf 2024-04-25T11:17:34.666 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:34 smithi069 ceph-mon[29188]: Updating smithi146:/etc/ceph/ceph.conf 2024-04-25T11:17:34.666 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:34 smithi069 ceph-mon[29188]: Updating smithi152:/etc/ceph/ceph.conf 2024-04-25T11:17:34.667 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:34 smithi069 ceph-mon[29188]: Updating smithi069:/var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T11:17:34.667 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:34 smithi069 ceph-mon[29188]: Updating smithi146:/var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T11:17:34.667 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:34 smithi069 ceph-mon[29188]: Updating smithi152:/var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T11:17:34.667 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:34 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:34.667 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:34 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:34.667 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:34 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:34.667 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:34 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:34.667 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:34 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:34.667 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:34 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:34.667 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:34 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:34.667 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:34 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:34.667 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:34 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:34.667 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:34 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:34.667 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:34 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:34.668 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:34 smithi069 ceph-mon[29188]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T11:17:34.668 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:34 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T11:17:34.668 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:34 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T11:17:34.668 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:34 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:17:34.668 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:34 smithi069 ceph-mon[29188]: Reconfiguring daemon mon.a on smithi069 2024-04-25T11:17:34.668 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:34 smithi069 ceph-mon[29188]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:34.668 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:34 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2151299014' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:17:34.737 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:34 smithi152 ceph-mon[32701]: Updating smithi069:/etc/ceph/ceph.conf 2024-04-25T11:17:34.737 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:34 smithi152 ceph-mon[32701]: Updating smithi146:/etc/ceph/ceph.conf 2024-04-25T11:17:34.737 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:34 smithi152 ceph-mon[32701]: Updating smithi152:/etc/ceph/ceph.conf 2024-04-25T11:17:34.738 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:34 smithi152 ceph-mon[32701]: Updating smithi069:/var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T11:17:34.738 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:34 smithi152 ceph-mon[32701]: Updating smithi146:/var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T11:17:34.738 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:34 smithi152 ceph-mon[32701]: Updating smithi152:/var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T11:17:34.738 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:34 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:34.738 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:34 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:34.738 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:34 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:34.738 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:34 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:34.738 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:34 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:34.738 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:34 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:34.738 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:34 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:34.738 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:34 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:34.738 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:34 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:34.738 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:34 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:34.738 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:34 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:34.739 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:34 smithi152 ceph-mon[32701]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T11:17:34.739 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:34 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T11:17:34.739 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:34 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T11:17:34.739 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:34 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:17:34.739 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:34 smithi152 ceph-mon[32701]: Reconfiguring daemon mon.a on smithi069 2024-04-25T11:17:34.739 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:34 smithi152 ceph-mon[32701]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:34.739 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:34 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/2151299014' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:17:34.856 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:34 smithi146 ceph-mon[32821]: Updating smithi069:/etc/ceph/ceph.conf 2024-04-25T11:17:34.856 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:34 smithi146 ceph-mon[32821]: Updating smithi146:/etc/ceph/ceph.conf 2024-04-25T11:17:34.856 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:34 smithi146 ceph-mon[32821]: Updating smithi152:/etc/ceph/ceph.conf 2024-04-25T11:17:34.856 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:34 smithi146 ceph-mon[32821]: Updating smithi069:/var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T11:17:34.856 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:34 smithi146 ceph-mon[32821]: Updating smithi146:/var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T11:17:34.856 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:34 smithi146 ceph-mon[32821]: Updating smithi152:/var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T11:17:34.856 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:34 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:34.857 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:34 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:34.857 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:34 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:34.857 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:34 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:34.857 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:34 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:34.857 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:34 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:34.857 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:34 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:34.857 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:34 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:34.857 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:34 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:34.858 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:34 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:34.858 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:34 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:34.858 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:34 smithi146 ceph-mon[32821]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T11:17:34.858 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:34 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T11:17:34.858 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:34 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T11:17:34.858 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:34 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:17:34.858 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:34 smithi146 ceph-mon[32821]: Reconfiguring daemon mon.a on smithi069 2024-04-25T11:17:34.858 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:34 smithi146 ceph-mon[32821]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:34.858 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:34 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/2151299014' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:17:35.721 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-25T11:17:35.721 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T11:17:35.721 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T11:17:35.750 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T11:17:35.750 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T11:17:35.815 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T11:17:35.815 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T11:17:35.844 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T11:17:35.844 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T11:17:35.909 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-25T11:17:35.909 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T11:17:35.939 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-25T11:17:35.940 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T11:17:36.007 INFO:tasks.cephadm:Adding mgr.x on smithi069 2024-04-25T11:17:36.007 INFO:tasks.cephadm:Adding mgr.y on smithi146 2024-04-25T11:17:36.007 INFO:tasks.cephadm:Adding mgr.z on smithi152 2024-04-25T11:17:36.007 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi069=x;smithi146=y;smithi152=z' 2024-04-25T11:17:36.280 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T11:17:36.619 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:36 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:36.619 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:36 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:36.619 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:36 smithi152 ceph-mon[32701]: Reconfiguring mon.b (monmap changed)... 2024-04-25T11:17:36.619 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:36 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T11:17:36.619 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:36 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T11:17:36.619 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:36 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:17:36.619 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:36 smithi152 ceph-mon[32701]: Reconfiguring daemon mon.b on smithi146 2024-04-25T11:17:36.619 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:36 smithi152 ceph-mon[32701]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:36.700 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:36 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:36.700 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:36 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:36.700 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:36 smithi069 ceph-mon[29188]: Reconfiguring mon.b (monmap changed)... 2024-04-25T11:17:36.700 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:36 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T11:17:36.700 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:36 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T11:17:36.700 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:36 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:17:36.700 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:36 smithi069 ceph-mon[29188]: Reconfiguring daemon mon.b on smithi146 2024-04-25T11:17:36.700 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:36 smithi069 ceph-mon[29188]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:36.856 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:36 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:36.856 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:36 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:36.856 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:36 smithi146 ceph-mon[32821]: Reconfiguring mon.b (monmap changed)... 2024-04-25T11:17:36.856 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:36 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T11:17:36.856 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:36 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T11:17:36.856 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:36 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:17:36.856 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:36 smithi146 ceph-mon[32821]: Reconfiguring daemon mon.b on smithi146 2024-04-25T11:17:36.857 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:36 smithi146 ceph-mon[32821]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:37.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:37.813+0000 7fe5f7d9b700 1 -- 172.21.15.152:0/3423723632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5f0103bc0 msgr2=0x7fe5f0103fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:37.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:37.813+0000 7fe5f7d9b700 1 --2- 172.21.15.152:0/3423723632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5f0103bc0 0x7fe5f0103fd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec009910 tx=0x7fe5ec009c20 comp rx=0 tx=0).stop 2024-04-25T11:17:37.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:37.813+0000 7fe5f7d9b700 1 -- 172.21.15.152:0/3423723632 shutdown_connections 2024-04-25T11:17:37.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:37.813+0000 7fe5f7d9b700 1 --2- 172.21.15.152:0/3423723632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5f0103bc0 0x7fe5f0103fd0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:37.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:37.813+0000 7fe5f7d9b700 1 -- 172.21.15.152:0/3423723632 >> 172.21.15.152:0/3423723632 conn(0x7fe5f00ff1e0 msgr2=0x7fe5f01016b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:17:37.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:37.813+0000 7fe5f7d9b700 1 -- 172.21.15.152:0/3423723632 shutdown_connections 2024-04-25T11:17:37.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:37.813+0000 7fe5f7d9b700 1 -- 172.21.15.152:0/3423723632 wait complete. 2024-04-25T11:17:37.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:37.814+0000 7fe5f7d9b700 1 Processor -- start 2024-04-25T11:17:37.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:37.814+0000 7fe5f7d9b700 1 -- start start 2024-04-25T11:17:37.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:37.814+0000 7fe5f7d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe5f0103bc0 0x7fe5f0199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:37.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:37.814+0000 7fe5f7d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5f0199cd0 0x7fe5f019ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:37.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:37.814+0000 7fe5f7d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5f019f1a0 0x7fe5f01a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:37.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:37.814+0000 7fe5f7d9b700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f0109a30 con 0x7fe5f019f1a0 2024-04-25T11:17:37.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:37.814+0000 7fe5f7d9b700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f01098b0 con 0x7fe5f0103bc0 2024-04-25T11:17:37.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:37.814+0000 7fe5f7d9b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f0109bb0 con 0x7fe5f0199cd0 2024-04-25T11:17:37.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:37.814+0000 7fe5f5336700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5f0199cd0 0x7fe5f019ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:37.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:37.814+0000 7fe5f5336700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5f0199cd0 0x7fe5f019ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38472/0 (socket says 172.21.15.152:38472) 2024-04-25T11:17:37.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:37.814+0000 7fe5f5336700 1 -- 172.21.15.152:0/1139191641 learned_addr learned my addr 172.21.15.152:0/1139191641 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-25T11:17:37.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:37.815+0000 7fe5f6338700 1 --2- 172.21.15.152:0/1139191641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5f019f1a0 0x7fe5f01a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:37.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:37.815+0000 7fe5f5b37700 1 --2- 172.21.15.152:0/1139191641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe5f0103bc0 0x7fe5f0199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:37.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:37.815+0000 7fe5f5336700 1 -- 172.21.15.152:0/1139191641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe5f0103bc0 msgr2=0x7fe5f0199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:37.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:37.815+0000 7fe5f5336700 1 --2- 172.21.15.152:0/1139191641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe5f0103bc0 0x7fe5f0199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:37.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:37.815+0000 7fe5f5336700 1 -- 172.21.15.152:0/1139191641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5f019f1a0 msgr2=0x7fe5f01a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:37.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:37.815+0000 7fe5f5336700 1 --2- 172.21.15.152:0/1139191641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5f019f1a0 0x7fe5f01a15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:37.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:37.815+0000 7fe5f5336700 1 -- 172.21.15.152:0/1139191641 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5f01a1b30 con 0x7fe5f0199cd0 2024-04-25T11:17:37.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:37.815+0000 7fe5f5336700 1 --2- 172.21.15.152:0/1139191641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5f0199cd0 0x7fe5f019ec60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe5e000b780 tx=0x7fe5e000ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:37.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:37.815+0000 7fe5e6ffd700 1 -- 172.21.15.152:0/1139191641 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e0004230 con 0x7fe5f0199cd0 2024-04-25T11:17:37.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:37.815+0000 7fe5e6ffd700 1 -- 172.21.15.152:0/1139191641 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe5e00043d0 con 0x7fe5f0199cd0 2024-04-25T11:17:37.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:37.815+0000 7fe5f7d9b700 1 -- 172.21.15.152:0/1139191641 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5f01a1e20 con 0x7fe5f0199cd0 2024-04-25T11:17:37.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:37.815+0000 7fe5e6ffd700 1 -- 172.21.15.152:0/1139191641 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e0011a60 con 0x7fe5f0199cd0 2024-04-25T11:17:37.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:37.815+0000 7fe5f7d9b700 1 -- 172.21.15.152:0/1139191641 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5f01a22f0 con 0x7fe5f0199cd0 2024-04-25T11:17:37.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:37.816+0000 7fe5e6ffd700 1 -- 172.21.15.152:0/1139191641 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7fe5e0011c00 con 0x7fe5f0199cd0 2024-04-25T11:17:37.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:37.816+0000 7fe5f7d9b700 1 -- 172.21.15.152:0/1139191641 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5f01a2580 con 0x7fe5f0199cd0 2024-04-25T11:17:37.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:37.817+0000 7fe5e6ffd700 1 --2- 172.21.15.152:0/1139191641 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fe5dc03daa0 0x7fe5dc03ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:37.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:37.817+0000 7fe5e6ffd700 1 -- 172.21.15.152:0/1139191641 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe5e0051220 con 0x7fe5f0199cd0 2024-04-25T11:17:37.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:37.817+0000 7fe5f5b37700 1 --2- 172.21.15.152:0/1139191641 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fe5dc03daa0 0x7fe5dc03ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:37.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:37.817+0000 7fe5f5b37700 1 --2- 172.21.15.152:0/1139191641 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fe5dc03daa0 0x7fe5dc03ffb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec004690 tx=0x7fe5ec01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:37.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:37.822+0000 7fe5e6ffd700 1 -- 172.21.15.152:0/1139191641 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe5e000fcf0 con 0x7fe5f0199cd0 2024-04-25T11:17:38.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:38.040+0000 7fe5f7d9b700 1 -- 172.21.15.152:0/1139191641 --> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi069=x;smithi146=y;smithi152=z", "target": ["mon-mgr", ""]}) -- 0x7fe5f0101110 con 0x7fe5dc03daa0 2024-04-25T11:17:38.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:38.047+0000 7fe5e6ffd700 1 -- 172.21.15.152:0/1139191641 <== mgr.14152 v2:172.21.15.69:6800/4220065409 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fe5f0101110 con 0x7fe5dc03daa0 2024-04-25T11:17:38.047 INFO:teuthology.orchestra.run.smithi152.stdout:Scheduled mgr update... 2024-04-25T11:17:38.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:38.049+0000 7fe5f7d9b700 1 -- 172.21.15.152:0/1139191641 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fe5dc03daa0 msgr2=0x7fe5dc03ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:38.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:38.049+0000 7fe5f7d9b700 1 --2- 172.21.15.152:0/1139191641 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fe5dc03daa0 0x7fe5dc03ffb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec004690 tx=0x7fe5ec01a040 comp rx=0 tx=0).stop 2024-04-25T11:17:38.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:38.049+0000 7fe5f7d9b700 1 -- 172.21.15.152:0/1139191641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5f0199cd0 msgr2=0x7fe5f019ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:38.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:38.049+0000 7fe5f7d9b700 1 --2- 172.21.15.152:0/1139191641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5f0199cd0 0x7fe5f019ec60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe5e000b780 tx=0x7fe5e000ba90 comp rx=0 tx=0).stop 2024-04-25T11:17:38.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:38.049+0000 7fe5f7d9b700 1 -- 172.21.15.152:0/1139191641 shutdown_connections 2024-04-25T11:17:38.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:38.049+0000 7fe5f7d9b700 1 --2- 172.21.15.152:0/1139191641 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fe5dc03daa0 0x7fe5dc03ffb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:38.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:38.049+0000 7fe5f7d9b700 1 --2- 172.21.15.152:0/1139191641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5f019f1a0 0x7fe5f01a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:38.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:38.049+0000 7fe5f7d9b700 1 --2- 172.21.15.152:0/1139191641 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe5f0103bc0 0x7fe5f0199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:38.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:38.049+0000 7fe5f7d9b700 1 --2- 172.21.15.152:0/1139191641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5f0199cd0 0x7fe5f019ec60 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:38.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:38.049+0000 7fe5f7d9b700 1 -- 172.21.15.152:0/1139191641 >> 172.21.15.152:0/1139191641 conn(0x7fe5f00ff1e0 msgr2=0x7fe5f0100a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:17:38.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:38.049+0000 7fe5f7d9b700 1 -- 172.21.15.152:0/1139191641 shutdown_connections 2024-04-25T11:17:38.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:17:38.049+0000 7fe5f7d9b700 1 -- 172.21.15.152:0/1139191641 wait complete. 2024-04-25T11:17:38.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:38 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:38.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:38 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:38.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:38 smithi146 ceph-mon[32821]: Reconfiguring mon.c (monmap changed)... 2024-04-25T11:17:38.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:38 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T11:17:38.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:38 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T11:17:38.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:38 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:17:38.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:38 smithi146 ceph-mon[32821]: Reconfiguring daemon mon.c on smithi152 2024-04-25T11:17:38.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:38 smithi146 ceph-mon[32821]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:38.450 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:38 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:38.450 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:38 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:38.450 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:38 smithi069 ceph-mon[29188]: Reconfiguring mon.c (monmap changed)... 2024-04-25T11:17:38.450 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:38 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T11:17:38.450 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:38 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T11:17:38.450 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:38 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:17:38.450 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:38 smithi069 ceph-mon[29188]: Reconfiguring daemon mon.c on smithi152 2024-04-25T11:17:38.450 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:38 smithi069 ceph-mon[29188]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:38.487 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:38 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:38.487 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:38 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:38.487 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:38 smithi152 ceph-mon[32701]: Reconfiguring mon.c (monmap changed)... 2024-04-25T11:17:38.487 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:38 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T11:17:38.487 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:38 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T11:17:38.487 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:38 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:17:38.488 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:38 smithi152 ceph-mon[32701]: Reconfiguring daemon mon.c on smithi152 2024-04-25T11:17:38.488 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:38 smithi152 ceph-mon[32701]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:39.191 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:39 smithi152 ceph-mon[32701]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi069=x;smithi146=y;smithi152=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:17:39.191 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:39 smithi152 ceph-mon[32701]: Saving service mgr spec with placement smithi069=x;smithi146=y;smithi152=z;count:3 2024-04-25T11:17:39.192 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:39 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:39.192 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:39 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:39.192 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:39 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:39.192 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:39 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:17:39.355 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:39 smithi146 ceph-mon[32821]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi069=x;smithi146=y;smithi152=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:17:39.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:39 smithi146 ceph-mon[32821]: Saving service mgr spec with placement smithi069=x;smithi146=y;smithi152=z;count:3 2024-04-25T11:17:39.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:39 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:39.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:39 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:39.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:39 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:39.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:39 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:17:39.449 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:39 smithi069 ceph-mon[29188]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi069=x;smithi146=y;smithi152=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:17:39.450 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:39 smithi069 ceph-mon[29188]: Saving service mgr spec with placement smithi069=x;smithi146=y;smithi152=z;count:3 2024-04-25T11:17:39.450 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:39 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:39.450 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:39 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:39.450 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:39 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:39.450 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:39 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:17:39.507 DEBUG:teuthology.orchestra.run.smithi146:mgr.y> sudo journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mgr.y.service 2024-04-25T11:17:39.510 DEBUG:teuthology.orchestra.run.smithi152:mgr.z> sudo journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mgr.z.service 2024-04-25T11:17:39.512 INFO:tasks.cephadm:Deploying OSDs... 2024-04-25T11:17:39.512 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T11:17:39.512 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T11:17:39.527 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T11:17:39.528 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_1 2024-04-25T11:17:39.583 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T11:17:39.583 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:17:39.583 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-25T11:17:39.583 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:17:39.583 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:17:39.583 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2024-04-25 11:16:44.183136430 +0000 2024-04-25T11:17:39.584 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2024-04-25 11:12:10.669619206 +0000 2024-04-25T11:17:39.584 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2024-04-25 11:12:10.669619206 +0000 2024-04-25T11:17:39.584 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2024-04-25 11:12:10.669619206 +0000 2024-04-25T11:17:39.584 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T11:17:39.645 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2024-04-25T11:17:39.645 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2024-04-25T11:17:39.645 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000268401 s, 1.9 MB/s 2024-04-25T11:17:39.646 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T11:17:39.702 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_2 2024-04-25T11:17:39.758 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T11:17:39.758 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:17:39.758 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-25T11:17:39.758 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:17:39.758 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:17:39.758 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2024-04-25 11:16:44.217135702 +0000 2024-04-25T11:17:39.758 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2024-04-25 11:12:10.988612020 +0000 2024-04-25T11:17:39.758 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2024-04-25 11:12:10.988612020 +0000 2024-04-25T11:17:39.758 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2024-04-25 11:12:10.988612020 +0000 2024-04-25T11:17:39.759 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T11:17:39.820 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2024-04-25T11:17:39.820 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2024-04-25T11:17:39.820 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000192662 s, 2.7 MB/s 2024-04-25T11:17:39.821 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T11:17:39.876 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_3 2024-04-25T11:17:39.932 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T11:17:39.932 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:17:39.932 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-25T11:17:39.932 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:17:39.932 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:17:39.932 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2024-04-25 11:16:44.248135039 +0000 2024-04-25T11:17:39.932 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2024-04-25 11:12:11.335604203 +0000 2024-04-25T11:17:39.932 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2024-04-25 11:12:11.335604203 +0000 2024-04-25T11:17:39.933 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2024-04-25 11:12:11.335604203 +0000 2024-04-25T11:17:39.933 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T11:17:39.994 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2024-04-25T11:17:39.994 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2024-04-25T11:17:39.994 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000215314 s, 2.4 MB/s 2024-04-25T11:17:39.995 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T11:17:40.052 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_4 2024-04-25T11:17:40.107 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T11:17:40.107 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:17:40.107 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-25T11:17:40.107 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:17:40.107 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:17:40.108 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2024-04-25 11:16:44.280134353 +0000 2024-04-25T11:17:40.108 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2024-04-25 11:12:11.680596431 +0000 2024-04-25T11:17:40.108 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2024-04-25 11:12:11.680596431 +0000 2024-04-25T11:17:40.108 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2024-04-25 11:12:11.680596431 +0000 2024-04-25T11:17:40.108 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T11:17:40.169 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2024-04-25T11:17:40.169 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2024-04-25T11:17:40.169 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000180569 s, 2.8 MB/s 2024-04-25T11:17:40.171 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T11:17:40.226 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-04-25T11:17:40.227 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T11:17:40.245 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T11:17:40.246 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_1 2024-04-25T11:17:40.304 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T11:17:40.304 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:17:40.304 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-25T11:17:40.304 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:17:40.305 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:17:40.305 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2024-04-25 11:17:25.485070505 +0000 2024-04-25T11:17:40.305 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2024-04-25 11:12:08.532803152 +0000 2024-04-25T11:17:40.305 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2024-04-25 11:12:08.532803152 +0000 2024-04-25T11:17:40.305 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: 2024-04-25 11:12:08.532803152 +0000 2024-04-25T11:17:40.305 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T11:17:40.311 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:40 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:17:40.311 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:40 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:17:40.311 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:40 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:40.311 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:40 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T11:17:40.311 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:40 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T11:17:40.311 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:40 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T11:17:40.311 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:40 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:17:40.311 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:40 smithi152 ceph-mon[32701]: Deploying daemon mgr.z on smithi152 2024-04-25T11:17:40.311 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:40 smithi152 ceph-mon[32701]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:40.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:40 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:17:40.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:40 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:17:40.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:40 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:40.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:40 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T11:17:40.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:40 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T11:17:40.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:40 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T11:17:40.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:40 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:17:40.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:40 smithi146 ceph-mon[32821]: Deploying daemon mgr.z on smithi152 2024-04-25T11:17:40.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:40 smithi146 ceph-mon[32821]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:40.372 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2024-04-25T11:17:40.372 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2024-04-25T11:17:40.372 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000235239 s, 2.2 MB/s 2024-04-25T11:17:40.374 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T11:17:40.432 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_2 2024-04-25T11:17:40.449 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:40 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:17:40.449 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:40 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:17:40.449 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:40 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:40.450 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:40 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T11:17:40.450 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:40 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T11:17:40.450 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:40 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T11:17:40.450 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:40 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:17:40.450 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:40 smithi069 ceph-mon[29188]: Deploying daemon mgr.z on smithi152 2024-04-25T11:17:40.450 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:40 smithi069 ceph-mon[29188]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:40.488 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T11:17:40.489 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:17:40.489 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-25T11:17:40.489 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:17:40.489 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:17:40.489 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2024-04-25 11:17:25.486070490 +0000 2024-04-25T11:17:40.489 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2024-04-25 11:12:08.901797306 +0000 2024-04-25T11:17:40.489 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2024-04-25 11:12:08.901797306 +0000 2024-04-25T11:17:40.489 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: 2024-04-25 11:12:08.901797306 +0000 2024-04-25T11:17:40.489 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T11:17:40.555 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2024-04-25T11:17:40.555 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2024-04-25T11:17:40.555 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000194545 s, 2.6 MB/s 2024-04-25T11:17:40.556 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T11:17:40.614 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_3 2024-04-25T11:17:40.671 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T11:17:40.671 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:17:40.671 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-25T11:17:40.671 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:17:40.671 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:17:40.671 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2024-04-25 11:17:25.486070490 +0000 2024-04-25T11:17:40.671 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2024-04-25 11:12:09.230792095 +0000 2024-04-25T11:17:40.671 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2024-04-25 11:12:09.230792095 +0000 2024-04-25T11:17:40.671 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: 2024-04-25 11:12:09.230792095 +0000 2024-04-25T11:17:40.672 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T11:17:40.734 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2024-04-25T11:17:40.734 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2024-04-25T11:17:40.734 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000220396 s, 2.3 MB/s 2024-04-25T11:17:40.736 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T11:17:40.792 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_4 2024-04-25T11:17:40.848 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T11:17:40.848 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:17:40.848 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-25T11:17:40.848 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:17:40.848 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:17:40.849 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2024-04-25 11:17:25.486070490 +0000 2024-04-25T11:17:40.849 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2024-04-25 11:12:09.577786598 +0000 2024-04-25T11:17:40.849 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2024-04-25 11:12:09.577786598 +0000 2024-04-25T11:17:40.849 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: 2024-04-25 11:12:09.577786598 +0000 2024-04-25T11:17:40.849 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T11:17:40.911 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2024-04-25T11:17:40.911 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2024-04-25T11:17:40.911 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000197146 s, 2.6 MB/s 2024-04-25T11:17:40.912 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T11:17:40.969 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-04-25T11:17:40.969 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T11:17:40.986 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T11:17:40.986 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_1 2024-04-25T11:17:41.047 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T11:17:41.047 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:17:41.048 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-25T11:17:41.048 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:17:41.048 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:17:41.048 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2024-04-25 11:17:16.447062449 +0000 2024-04-25T11:17:41.048 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2024-04-25 11:12:19.466897676 +0000 2024-04-25T11:17:41.048 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2024-04-25 11:12:19.466897676 +0000 2024-04-25T11:17:41.048 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: 2024-04-25 11:12:19.466897676 +0000 2024-04-25T11:17:41.048 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T11:17:41.206 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-04-25T11:17:41.207 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-04-25T11:17:41.207 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000272958 s, 1.9 MB/s 2024-04-25T11:17:41.208 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T11:17:41.223 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_2 2024-04-25T11:17:41.285 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T11:17:41.285 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:17:41.285 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-25T11:17:41.286 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:17:41.286 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:17:41.286 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2024-04-25 11:17:41.160753217 +0000 2024-04-25T11:17:41.286 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2024-04-25 11:12:19.812893224 +0000 2024-04-25T11:17:41.286 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2024-04-25 11:12:19.812893224 +0000 2024-04-25T11:17:41.286 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: 2024-04-25 11:12:19.812893224 +0000 2024-04-25T11:17:41.286 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T11:17:41.394 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-04-25T11:17:41.394 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-04-25T11:17:41.394 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000190839 s, 2.7 MB/s 2024-04-25T11:17:41.395 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T11:17:41.410 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_3 2024-04-25T11:17:41.467 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T11:17:41.467 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:17:41.467 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-25T11:17:41.467 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:17:41.467 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:17:41.467 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2024-04-25 11:17:41.349750852 +0000 2024-04-25T11:17:41.467 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2024-04-25 11:12:20.139889017 +0000 2024-04-25T11:17:41.467 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2024-04-25 11:12:20.139889017 +0000 2024-04-25T11:17:41.467 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: 2024-04-25 11:12:20.139889017 +0000 2024-04-25T11:17:41.467 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T11:17:41.487 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Apr 25 11:17:41 smithi152 systemd[1]: Starting Ceph mgr.z for f7ed73dc-02f4-11ef-bc93-c7b262605968... 2024-04-25T11:17:41.512 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-04-25T11:17:41.512 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-04-25T11:17:41.512 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000298376 s, 1.7 MB/s 2024-04-25T11:17:41.513 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T11:17:41.569 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_4 2024-04-25T11:17:41.624 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T11:17:41.624 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:17:41.624 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-25T11:17:41.624 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:17:41.625 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:17:41.625 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2024-04-25 11:17:41.349750852 +0000 2024-04-25T11:17:41.625 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2024-04-25 11:12:20.479884643 +0000 2024-04-25T11:17:41.625 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2024-04-25 11:12:20.479884643 +0000 2024-04-25T11:17:41.625 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: 2024-04-25 11:12:20.479884643 +0000 2024-04-25T11:17:41.625 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T11:17:41.687 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-04-25T11:17:41.688 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-04-25T11:17:41.688 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000200234 s, 2.6 MB/s 2024-04-25T11:17:41.689 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T11:17:41.744 INFO:tasks.cephadm:Deploying osd.0 on smithi069 with /dev/vg_nvme/lv_4... 2024-04-25T11:17:41.744 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T11:17:41.780 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Apr 25 11:17:41 smithi152 podman[34470]: 2024-04-25 11:17:41.538228439 +0000 UTC m=+0.018026327 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T11:17:41.780 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Apr 25 11:17:41 smithi152 podman[34470]: 2024-04-25 11:17:41.658124131 +0000 UTC m=+0.137922009 container create 36ffa8d398c7360af331d9793c19b9653e3b24df639ff6fe0ce740174f1209cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-25T11:17:41.984 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:17:42.058 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Apr 25 11:17:41 smithi152 podman[34470]: 2024-04-25 11:17:41.885539786 +0000 UTC m=+0.365337664 container init 36ffa8d398c7360af331d9793c19b9653e3b24df639ff6fe0ce740174f1209cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-z, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T11:17:42.058 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Apr 25 11:17:41 smithi152 podman[34470]: 2024-04-25 11:17:41.888804832 +0000 UTC m=+0.368602710 container start 36ffa8d398c7360af331d9793c19b9653e3b24df639ff6fe0ce740174f1209cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-z, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-25T11:17:42.058 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Apr 25 11:17:41 smithi152 ceph-mgr[34561]: -- 172.21.15.152:0/365783566 <== mon.1 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561ff276c5a0 con 0x561ff74f6900 2024-04-25T11:17:42.058 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Apr 25 11:17:41 smithi152 bash[34470]: 36ffa8d398c7360af331d9793c19b9653e3b24df639ff6fe0ce740174f1209cd 2024-04-25T11:17:42.059 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Apr 25 11:17:41 smithi152 systemd[1]: Started Ceph mgr.z for f7ed73dc-02f4-11ef-bc93-c7b262605968. 2024-04-25T11:17:42.322 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Apr 25 11:17:42 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-z[34557]: 2024-04-25T11:17:42.057+0000 7f11bde40200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T11:17:42.323 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Apr 25 11:17:42 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-z[34557]: 2024-04-25T11:17:42.248+0000 7f11bde40200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:17:42.682 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Apr 25 11:17:42 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-z[34557]: 2024-04-25T11:17:42.356+0000 7f11bde40200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T11:17:42.683 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Apr 25 11:17:42 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-z[34557]: 2024-04-25T11:17:42.682+0000 7f11bde40200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T11:17:42.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:42 smithi146 ceph-mon[32821]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:42.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:42 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:42.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:42 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:42.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:42 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:42.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:42 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T11:17:42.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:42 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T11:17:42.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:42 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T11:17:42.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:42 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:17:42.987 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Apr 25 11:17:42 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-z[34557]: 2024-04-25T11:17:42.770+0000 7f11bde40200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T11:17:42.987 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:42 smithi152 ceph-mon[32701]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:42.987 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:42 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:42.988 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:42 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:42.988 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:42 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:42.988 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:42 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T11:17:42.988 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:42 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T11:17:42.988 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:42 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T11:17:42.988 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:42 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:17:43.052 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:42 smithi069 ceph-mon[29188]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:43.052 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:42 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:43.052 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:42 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:43.052 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:42 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:43.052 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:42 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T11:17:43.052 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:42 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T11:17:43.052 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:42 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T11:17:43.053 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:42 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:17:43.867 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T11:17:43.868 INFO:teuthology.orchestra.run.smithi069.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T11:17:43.868 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10+0 records in 2024-04-25T11:17:43.868 INFO:teuthology.orchestra.run.smithi069.stderr:10+0 records out 2024-04-25T11:17:43.868 INFO:teuthology.orchestra.run.smithi069.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0127583 s, 822 MB/s 2024-04-25T11:17:43.868 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 2024-04-25T11:17:43.868 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping successful for: 2024-04-25T11:17:44.051 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:43 smithi146 ceph-mon[32821]: Deploying daemon mgr.y on smithi146 2024-04-25T11:17:44.051 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Apr 25 11:17:43 smithi146 systemd[1]: Starting Ceph mgr.y for f7ed73dc-02f4-11ef-bc93-c7b262605968... 2024-04-25T11:17:44.051 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Apr 25 11:17:44 smithi146 podman[34189]: 2024-04-25 11:17:44.050952854 +0000 UTC m=+0.104842824 container create 91e9d14228743e7ef9d70c163a4beb80f5e7240e8ed76761a1d4e0c5e879881e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-y, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-25T11:17:44.150 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:43 smithi152 ceph-mon[32701]: Deploying daemon mgr.y on smithi146 2024-04-25T11:17:44.150 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Apr 25 11:17:43 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-z[34557]: 2024-04-25T11:17:43.982+0000 7f11bde40200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T11:17:44.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:43 smithi069 ceph-mon[29188]: Deploying daemon mgr.y on smithi146 2024-04-25T11:17:44.311 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Apr 25 11:17:44 smithi146 podman[34189]: 2024-04-25 11:17:43.964111078 +0000 UTC m=+0.018001058 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T11:17:44.311 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Apr 25 11:17:44 smithi146 podman[34189]: 2024-04-25 11:17:44.253244844 +0000 UTC m=+0.307134812 container init 91e9d14228743e7ef9d70c163a4beb80f5e7240e8ed76761a1d4e0c5e879881e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-y, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T11:17:44.311 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Apr 25 11:17:44 smithi146 podman[34189]: 2024-04-25 11:17:44.256078938 +0000 UTC m=+0.309968906 container start 91e9d14228743e7ef9d70c163a4beb80f5e7240e8ed76761a1d4e0c5e879881e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-y, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T11:17:44.311 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Apr 25 11:17:44 smithi146 ceph-mgr[34209]: -- 172.21.15.146:0/2209445013 <== mon.2 v2:172.21.15.146:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d450b485a0 con 0x55d4558d2480 2024-04-25T11:17:44.487 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Apr 25 11:17:44 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-z[34557]: 2024-04-25T11:17:44.149+0000 7f11bde40200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T11:17:44.487 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Apr 25 11:17:44 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-z[34557]: 2024-04-25T11:17:44.319+0000 7f11bde40200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T11:17:44.560 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi069:vg_nvme/lv_4 2024-04-25T11:17:44.577 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Apr 25 11:17:44 smithi146 bash[34189]: 91e9d14228743e7ef9d70c163a4beb80f5e7240e8ed76761a1d4e0c5e879881e 2024-04-25T11:17:44.577 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Apr 25 11:17:44 smithi146 systemd[1]: Started Ceph mgr.y for f7ed73dc-02f4-11ef-bc93-c7b262605968. 2024-04-25T11:17:44.577 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Apr 25 11:17:44 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-y[34204]: 2024-04-25T11:17:44.432+0000 7f5554635200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T11:17:44.794 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:17:44.856 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:44 smithi146 ceph-mon[32821]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:44.856 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:44 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:44.856 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:44 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:44.856 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:44 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:44.856 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:44 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:44.856 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:44 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:17:44.856 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Apr 25 11:17:44 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-y[34204]: 2024-04-25T11:17:44.633+0000 7f5554635200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:17:44.856 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Apr 25 11:17:44 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-y[34204]: 2024-04-25T11:17:44.744+0000 7f5554635200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T11:17:44.909 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:44 smithi069 ceph-mon[29188]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:44.909 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:44 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:44.909 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:44 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:44.909 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:44 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:44.909 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:44 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:44.909 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:44 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:17:44.919 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Apr 25 11:17:44 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-z[34557]: 2024-04-25T11:17:44.610+0000 7f11bde40200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T11:17:44.919 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Apr 25 11:17:44 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-z[34557]: 2024-04-25T11:17:44.706+0000 7f11bde40200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T11:17:44.919 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Apr 25 11:17:44 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-z[34557]: 2024-04-25T11:17:44.918+0000 7f11bde40200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T11:17:44.919 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:44 smithi152 ceph-mon[32701]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:44.919 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:44 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:44.919 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:44 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:44.919 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:44 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:44.919 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:44 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:44.919 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:44 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:17:45.237 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Apr 25 11:17:45 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-z[34557]: 2024-04-25T11:17:45.129+0000 7f11bde40200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:17:45.356 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Apr 25 11:17:45 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-y[34204]: 2024-04-25T11:17:45.095+0000 7f5554635200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T11:17:45.356 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Apr 25 11:17:45 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-y[34204]: 2024-04-25T11:17:45.197+0000 7f5554635200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T11:17:46.187 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Apr 25 11:17:45 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-z[34557]: 2024-04-25T11:17:45.756+0000 7f11bde40200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T11:17:46.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:46.291+0000 7f79564e0700 1 -- 172.21.15.69:0/1063077212 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79501041e0 msgr2=0x7f79501045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:46.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:46.291+0000 7f79564e0700 1 --2- 172.21.15.69:0/1063077212 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79501041e0 0x7f79501045f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7944009990 tx=0x7f7944009ca0 comp rx=0 tx=0).stop 2024-04-25T11:17:46.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:46.291+0000 7f79564e0700 1 -- 172.21.15.69:0/1063077212 shutdown_connections 2024-04-25T11:17:46.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:46.291+0000 7f79564e0700 1 --2- 172.21.15.69:0/1063077212 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7950105cb0 0x7f795010c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:46.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:46.291+0000 7f79564e0700 1 --2- 172.21.15.69:0/1063077212 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79501041e0 0x7f79501045f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:46.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:46.291+0000 7f79564e0700 1 --2- 172.21.15.69:0/1063077212 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7950105310 0x7f7950105720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:46.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:46.291+0000 7f79564e0700 1 -- 172.21.15.69:0/1063077212 >> 172.21.15.69:0/1063077212 conn(0x7f79500ff840 msgr2=0x7f7950101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:17:46.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:46.291+0000 7f79564e0700 1 -- 172.21.15.69:0/1063077212 shutdown_connections 2024-04-25T11:17:46.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:46.291+0000 7f79564e0700 1 -- 172.21.15.69:0/1063077212 wait complete. 2024-04-25T11:17:46.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:46.292+0000 7f79564e0700 1 Processor -- start 2024-04-25T11:17:46.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:46.292+0000 7f79564e0700 1 -- start start 2024-04-25T11:17:46.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:46.292+0000 7f79564e0700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79501041e0 0x7f7950077330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:46.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:46.292+0000 7f79564e0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7950105310 0x7f79500759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:46.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:46.292+0000 7f79564e0700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7950105cb0 0x7f7950075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:46.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:46.292+0000 7f79564e0700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f795010fae0 con 0x7f7950105310 2024-04-25T11:17:46.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:46.292+0000 7f79564e0700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f795010f960 con 0x7f7950105cb0 2024-04-25T11:17:46.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:46.292+0000 7f79564e0700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f795010fc60 con 0x7f79501041e0 2024-04-25T11:17:46.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:46.292+0000 7f794f7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7950105310 0x7f79500759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:46.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:46.292+0000 7f794ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79501041e0 0x7f7950077330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:46.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:46.292+0000 7f794f7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7950105310 0x7f79500759e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49212/0 (socket says 172.21.15.69:49212) 2024-04-25T11:17:46.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:46.292+0000 7f794f7fe700 1 -- 172.21.15.69:0/1627418322 learned_addr learned my addr 172.21.15.69:0/1627418322 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:17:46.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:46.292+0000 7f7954a7d700 1 --2- 172.21.15.69:0/1627418322 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7950105cb0 0x7f7950075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:46.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:46.293+0000 7f794f7fe700 1 -- 172.21.15.69:0/1627418322 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79501041e0 msgr2=0x7f7950077330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:46.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:46.293+0000 7f794f7fe700 1 --2- 172.21.15.69:0/1627418322 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79501041e0 0x7f7950077330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:46.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:46.293+0000 7f794f7fe700 1 -- 172.21.15.69:0/1627418322 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7950105cb0 msgr2=0x7f7950075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:46.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:46.293+0000 7f794f7fe700 1 --2- 172.21.15.69:0/1627418322 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7950105cb0 0x7f7950075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:46.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:46.293+0000 7f794f7fe700 1 -- 172.21.15.69:0/1627418322 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7950076680 con 0x7f7950105310 2024-04-25T11:17:46.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:46.293+0000 7f794f7fe700 1 --2- 172.21.15.69:0/1627418322 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7950105310 0x7f79500759e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f794000d980 tx=0x7f794000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:46.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:46.293+0000 7f794d7fa700 1 -- 172.21.15.69:0/1627418322 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7940016070 con 0x7f7950105310 2024-04-25T11:17:46.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:46.293+0000 7f794d7fa700 1 -- 172.21.15.69:0/1627418322 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7940004830 con 0x7f7950105310 2024-04-25T11:17:46.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:46.293+0000 7f794d7fa700 1 -- 172.21.15.69:0/1627418322 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79400054c0 con 0x7f7950105310 2024-04-25T11:17:46.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:46.293+0000 7f79564e0700 1 -- 172.21.15.69:0/1627418322 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79501a47b0 con 0x7f7950105310 2024-04-25T11:17:46.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:46.293+0000 7f79564e0700 1 -- 172.21.15.69:0/1627418322 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79501a4ce0 con 0x7f7950105310 2024-04-25T11:17:46.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:46.294+0000 7f794d7fa700 1 -- 172.21.15.69:0/1627418322 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 49572+0+0 (secure 0 0 0) 0x7f794000f690 con 0x7f7950105310 2024-04-25T11:17:46.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:46.294+0000 7f794d7fa700 1 --2- 172.21.15.69:0/1627418322 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f793803d850 0x7f793803fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:46.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:46.294+0000 7f794d7fa700 1 -- 172.21.15.69:0/1627418322 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f79400515c0 con 0x7f7950105310 2024-04-25T11:17:46.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:46.294+0000 7f79564e0700 1 -- 172.21.15.69:0/1627418322 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7950051190 con 0x7f7950105310 2024-04-25T11:17:46.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:46.294+0000 7f794ffff700 1 --2- 172.21.15.69:0/1627418322 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f793803d850 0x7f793803fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:46.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:46.295+0000 7f794ffff700 1 --2- 172.21.15.69:0/1627418322 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f793803d850 0x7f793803fd60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7944006010 tx=0x7f794401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:46.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:46.300+0000 7f794d7fa700 1 -- 172.21.15.69:0/1627418322 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f794001f020 con 0x7f7950105310 2024-04-25T11:17:46.481 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Apr 25 11:17:46 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-z[34557]: 2024-04-25T11:17:46.186+0000 7f11bde40200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T11:17:46.482 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Apr 25 11:17:46 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-z[34557]: 2024-04-25T11:17:46.281+0000 7f11bde40200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T11:17:46.482 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Apr 25 11:17:46 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-z[34557]: 2024-04-25T11:17:46.480+0000 7f11bde40200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T11:17:46.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:46.537+0000 7f79564e0700 1 -- 172.21.15.69:0/1627418322 --> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f7950063ad0 con 0x7f793803d850 2024-04-25T11:17:46.766 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Apr 25 11:17:46 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-y[34204]: 2024-04-25T11:17:46.451+0000 7f5554635200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T11:17:46.767 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Apr 25 11:17:46 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-y[34204]: 2024-04-25T11:17:46.622+0000 7f5554635200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T11:17:46.795 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:46 smithi069 ceph-mon[29188]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:46.795 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:46 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:46.795 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:46 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:46.795 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:46 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:46.795 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:46 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:46.795 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:46 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:17:46.795 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:46 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:17:46.795 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:46 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:46.795 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:46 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T11:17:46.795 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:46 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T11:17:46.795 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:46 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:17:46.795 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:46 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:17:46.795 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:46 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:17:46.795 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:46 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:17:46.796 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:46 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:47.082 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Apr 25 11:17:46 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-y[34204]: 2024-04-25T11:17:46.795+0000 7f5554635200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T11:17:47.082 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:46 smithi146 ceph-mon[32821]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:47.082 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:46 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:47.082 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:46 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:47.082 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:46 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:47.082 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:46 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:47.082 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:46 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:17:47.082 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:46 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:17:47.082 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:46 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:47.082 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:46 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T11:17:47.083 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:46 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T11:17:47.083 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:46 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:17:47.083 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:46 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:17:47.083 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:46 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:17:47.083 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:46 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:17:47.083 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:46 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:47.100 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Apr 25 11:17:47 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-z[34557]: 2024-04-25T11:17:47.008+0000 7f11bde40200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T11:17:47.100 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:46 smithi152 ceph-mon[32701]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:47.100 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:46 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:47.100 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:46 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:47.100 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:46 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:47.100 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:46 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:47.100 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:46 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:17:47.100 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:46 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:17:47.100 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:46 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:47.101 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:46 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T11:17:47.101 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:46 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T11:17:47.101 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:46 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:17:47.101 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:46 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:17:47.101 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:46 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:17:47.101 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:46 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:17:47.101 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:46 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:47.355 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Apr 25 11:17:47 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-y[34204]: 2024-04-25T11:17:47.081+0000 7f5554635200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T11:17:47.356 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Apr 25 11:17:47 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-y[34204]: 2024-04-25T11:17:47.173+0000 7f5554635200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T11:17:47.473 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Apr 25 11:17:47 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-z[34557]: 2024-04-25T11:17:47.099+0000 7f11bde40200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T11:17:47.473 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Apr 25 11:17:47 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-z[34557]: 2024-04-25T11:17:47.341+0000 7f11bde40200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T11:17:47.473 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Apr 25 11:17:47 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-z[34557]: 2024-04-25T11:17:47.473+0000 7f11bde40200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T11:17:47.737 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Apr 25 11:17:47 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-z[34557]: 2024-04-25T11:17:47.567+0000 7f11bde40200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T11:17:47.737 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Apr 25 11:17:47 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-z[34557]: 2024-04-25T11:17:47.682+0000 7f11bde40200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T11:17:47.770 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Apr 25 11:17:47 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-y[34204]: 2024-04-25T11:17:47.376+0000 7f5554635200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T11:17:47.770 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Apr 25 11:17:47 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-y[34204]: 2024-04-25T11:17:47.569+0000 7f5554635200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:17:48.042 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:47 smithi069 ceph-mon[29188]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T11:17:48.043 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:47 smithi069 ceph-mon[29188]: Reconfiguring daemon mgr.x on smithi069 2024-04-25T11:17:48.043 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:47 smithi069 ceph-mon[29188]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:17:48.043 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:47 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:48.043 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:47 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:48.043 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:47 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:17:48.043 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:47 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:17:48.043 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:47 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:17:48.043 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:47 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:48.106 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:47 smithi146 ceph-mon[32821]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T11:17:48.106 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:47 smithi146 ceph-mon[32821]: Reconfiguring daemon mgr.x on smithi069 2024-04-25T11:17:48.106 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:47 smithi146 ceph-mon[32821]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:17:48.106 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:47 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:48.106 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:47 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:48.106 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:47 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:17:48.106 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:47 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:17:48.106 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:47 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:17:48.106 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:47 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:48.237 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Apr 25 11:17:47 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-z[34557]: 2024-04-25T11:17:47.987+0000 7f11bde40200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T11:17:48.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:47 smithi152 ceph-mon[32701]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T11:17:48.238 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:47 smithi152 ceph-mon[32701]: Reconfiguring daemon mgr.x on smithi069 2024-04-25T11:17:48.238 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:47 smithi152 ceph-mon[32701]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:17:48.238 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:47 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:48.238 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:47 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:48.238 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:47 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:17:48.238 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:47 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:17:48.238 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:47 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:17:48.238 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:47 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:17:48.596 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Apr 25 11:17:48 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-y[34204]: 2024-04-25T11:17:48.174+0000 7f5554635200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T11:17:48.859 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:48 smithi146 ceph-mon[32821]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:48.860 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Apr 25 11:17:48 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-y[34204]: 2024-04-25T11:17:48.595+0000 7f5554635200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T11:17:48.860 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Apr 25 11:17:48 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-y[34204]: 2024-04-25T11:17:48.691+0000 7f5554635200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T11:17:48.987 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:48 smithi152 ceph-mon[32701]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:48.987 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Apr 25 11:17:48 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-z[34557]: 2024-04-25T11:17:48.577+0000 7f11bde40200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T11:17:48.988 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Apr 25 11:17:48 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-z[34557]: 2024-04-25T11:17:48.792+0000 7f11bde40200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T11:17:49.025 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:48 smithi069 ceph-mon[29188]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:49.356 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Apr 25 11:17:48 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-y[34204]: 2024-04-25T11:17:48.891+0000 7f5554635200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T11:17:49.755 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Apr 25 11:17:49 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-y[34204]: 2024-04-25T11:17:49.417+0000 7f5554635200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T11:17:49.755 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Apr 25 11:17:49 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-y[34204]: 2024-04-25T11:17:49.511+0000 7f5554635200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T11:17:49.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:49.781+0000 7f794d7fa700 1 -- 172.21.15.69:0/1627418322 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mgrmap(e 14) ==== 98518+0+0 (secure 0 0 0) 0x7f79400193b0 con 0x7f7950105310 2024-04-25T11:17:50.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:49 smithi146 ceph-mon[32821]: from='mgr.? 172.21.15.152:0/3714862293' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T11:17:50.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:49 smithi146 ceph-mon[32821]: Standby manager daemon z started 2024-04-25T11:17:50.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:49 smithi146 ceph-mon[32821]: from='mgr.? 172.21.15.152:0/3714862293' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T11:17:50.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:49 smithi146 ceph-mon[32821]: from='mgr.? 172.21.15.152:0/3714862293' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T11:17:50.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:49 smithi146 ceph-mon[32821]: from='mgr.? 172.21.15.152:0/3714862293' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T11:17:50.101 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Apr 25 11:17:49 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-y[34204]: 2024-04-25T11:17:49.754+0000 7f5554635200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T11:17:50.101 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Apr 25 11:17:49 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-y[34204]: 2024-04-25T11:17:49.890+0000 7f5554635200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T11:17:50.101 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Apr 25 11:17:49 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-y[34204]: 2024-04-25T11:17:49.985+0000 7f5554635200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T11:17:50.101 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Apr 25 11:17:50 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-y[34204]: 2024-04-25T11:17:50.100+0000 7f5554635200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T11:17:50.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:49 smithi069 ceph-mon[29188]: from='mgr.? 172.21.15.152:0/3714862293' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T11:17:50.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:49 smithi069 ceph-mon[29188]: Standby manager daemon z started 2024-04-25T11:17:50.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:49 smithi069 ceph-mon[29188]: from='mgr.? 172.21.15.152:0/3714862293' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T11:17:50.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:49 smithi069 ceph-mon[29188]: from='mgr.? 172.21.15.152:0/3714862293' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T11:17:50.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:49 smithi069 ceph-mon[29188]: from='mgr.? 172.21.15.152:0/3714862293' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T11:17:50.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:49 smithi152 ceph-mon[32701]: from='mgr.? 172.21.15.152:0/3714862293' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T11:17:50.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:49 smithi152 ceph-mon[32701]: Standby manager daemon z started 2024-04-25T11:17:50.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:49 smithi152 ceph-mon[32701]: from='mgr.? 172.21.15.152:0/3714862293' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T11:17:50.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:49 smithi152 ceph-mon[32701]: from='mgr.? 172.21.15.152:0/3714862293' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T11:17:50.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:49 smithi152 ceph-mon[32701]: from='mgr.? 172.21.15.152:0/3714862293' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T11:17:50.780 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Apr 25 11:17:50 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-y[34204]: 2024-04-25T11:17:50.403+0000 7f5554635200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T11:17:51.106 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Apr 25 11:17:51 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-y[34204]: 2024-04-25T11:17:51.004+0000 7f5554635200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T11:17:51.106 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:50 smithi146 ceph-mon[32821]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:51.106 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:50 smithi146 ceph-mon[32821]: mgrmap e14: x(active, since 78s), standbys: z 2024-04-25T11:17:51.106 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:50 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T11:17:51.106 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:50 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/4192784169' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6a0d179-4998-4754-bf82-2ea3dac54e12"}]: dispatch 2024-04-25T11:17:51.106 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:50 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/4192784169' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a6a0d179-4998-4754-bf82-2ea3dac54e12"}]': finished 2024-04-25T11:17:51.106 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:50 smithi146 ceph-mon[32821]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T11:17:51.106 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:50 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:17:51.107 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:50 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/1575674728' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:17:51.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:50 smithi069 ceph-mon[29188]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:51.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:50 smithi069 ceph-mon[29188]: mgrmap e14: x(active, since 78s), standbys: z 2024-04-25T11:17:51.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:50 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T11:17:51.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:50 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/4192784169' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6a0d179-4998-4754-bf82-2ea3dac54e12"}]: dispatch 2024-04-25T11:17:51.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:50 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/4192784169' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a6a0d179-4998-4754-bf82-2ea3dac54e12"}]': finished 2024-04-25T11:17:51.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:50 smithi069 ceph-mon[29188]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T11:17:51.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:50 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:17:51.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:50 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/1575674728' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:17:51.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:50 smithi152 ceph-mon[32701]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:51.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:50 smithi152 ceph-mon[32701]: mgrmap e14: x(active, since 78s), standbys: z 2024-04-25T11:17:51.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:50 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T11:17:51.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:50 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/4192784169' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6a0d179-4998-4754-bf82-2ea3dac54e12"}]: dispatch 2024-04-25T11:17:51.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:50 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/4192784169' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a6a0d179-4998-4754-bf82-2ea3dac54e12"}]': finished 2024-04-25T11:17:51.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:50 smithi152 ceph-mon[32701]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T11:17:51.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:50 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:17:51.238 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:50 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/1575674728' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:17:51.606 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Apr 25 11:17:51 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-y[34204]: 2024-04-25T11:17:51.219+0000 7f5554635200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T11:17:51.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:17:51.792+0000 7f794d7fa700 1 -- 172.21.15.69:0/1627418322 <== mon.0 v2:172.21.15.69:3300/0 8 ==== mgrmap(e 15) ==== 147464+0+0 (secure 0 0 0) 0x7f794000cfb0 con 0x7f7950105310 2024-04-25T11:17:52.106 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:51 smithi146 ceph-mon[32821]: Standby manager daemon y started 2024-04-25T11:17:52.106 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:51 smithi146 ceph-mon[32821]: from='mgr.? 172.21.15.146:0/2277908169' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T11:17:52.106 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:51 smithi146 ceph-mon[32821]: from='mgr.? 172.21.15.146:0/2277908169' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T11:17:52.106 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:51 smithi146 ceph-mon[32821]: from='mgr.? 172.21.15.146:0/2277908169' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T11:17:52.106 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:51 smithi146 ceph-mon[32821]: from='mgr.? 172.21.15.146:0/2277908169' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T11:17:52.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:51 smithi069 ceph-mon[29188]: Standby manager daemon y started 2024-04-25T11:17:52.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:51 smithi069 ceph-mon[29188]: from='mgr.? 172.21.15.146:0/2277908169' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T11:17:52.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:51 smithi069 ceph-mon[29188]: from='mgr.? 172.21.15.146:0/2277908169' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T11:17:52.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:51 smithi069 ceph-mon[29188]: from='mgr.? 172.21.15.146:0/2277908169' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T11:17:52.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:51 smithi069 ceph-mon[29188]: from='mgr.? 172.21.15.146:0/2277908169' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T11:17:52.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:51 smithi152 ceph-mon[32701]: Standby manager daemon y started 2024-04-25T11:17:52.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:51 smithi152 ceph-mon[32701]: from='mgr.? 172.21.15.146:0/2277908169' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T11:17:52.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:51 smithi152 ceph-mon[32701]: from='mgr.? 172.21.15.146:0/2277908169' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T11:17:52.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:51 smithi152 ceph-mon[32701]: from='mgr.? 172.21.15.146:0/2277908169' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T11:17:52.238 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:51 smithi152 ceph-mon[32701]: from='mgr.? 172.21.15.146:0/2277908169' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T11:17:53.106 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:52 smithi146 ceph-mon[32821]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:53.106 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:52 smithi146 ceph-mon[32821]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-25T11:17:53.106 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:52 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T11:17:53.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:52 smithi069 ceph-mon[29188]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:53.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:52 smithi069 ceph-mon[29188]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-25T11:17:53.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:52 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T11:17:53.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:52 smithi152 ceph-mon[32701]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:53.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:52 smithi152 ceph-mon[32701]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-25T11:17:53.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:52 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T11:17:55.048 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:54 smithi069 ceph-mon[29188]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:55.105 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:54 smithi146 ceph-mon[32821]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:55.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:54 smithi152 ceph-mon[32701]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:57.105 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:56 smithi146 ceph-mon[32821]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:57.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:56 smithi069 ceph-mon[29188]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:57.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:56 smithi152 ceph-mon[32701]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:57.843 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:57 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T11:17:57.843 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:57 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:17:58.105 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:57 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T11:17:58.105 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:57 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:17:58.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:57 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T11:17:58.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:57 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:17:58.853 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:58 smithi069 ceph-mon[29188]: Deploying daemon osd.0 on smithi069 2024-04-25T11:17:58.853 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:17:58 smithi069 ceph-mon[29188]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:59.105 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:58 smithi146 ceph-mon[32821]: Deploying daemon osd.0 on smithi069 2024-04-25T11:17:59.106 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:17:58 smithi146 ceph-mon[32821]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:17:59.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:58 smithi152 ceph-mon[32701]: Deploying daemon osd.0 on smithi069 2024-04-25T11:17:59.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:17:58 smithi152 ceph-mon[32701]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:18:01.105 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:00 smithi146 ceph-mon[32821]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:18:01.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:00 smithi069 ceph-mon[29188]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:18:01.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:00 smithi152 ceph-mon[32701]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:18:02.992 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:02 smithi069 ceph-mon[29188]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:18:02.993 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:02 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:02.993 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:02 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:02.993 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:02 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:18:03.105 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:02 smithi146 ceph-mon[32821]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:18:03.105 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:02 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:03.105 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:02 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:03.106 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:02 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:18:03.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:02 smithi152 ceph-mon[32701]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:18:03.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:02 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:03.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:02 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:03.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:02 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:18:05.066 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:04 smithi069 ceph-mon[29188]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:18:05.105 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:04 smithi146 ceph-mon[32821]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:18:05.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:04 smithi152 ceph-mon[32701]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:18:06.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:06.246+0000 7f794d7fa700 1 -- 172.21.15.69:0/1627418322 <== mgr.14152 v2:172.21.15.69:6800/4220065409 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7950063ad0 con 0x7f793803d850 2024-04-25T11:18:06.247 INFO:teuthology.orchestra.run.smithi069.stdout:Created osd(s) 0 on host 'smithi069' 2024-04-25T11:18:06.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:06.249+0000 7f79564e0700 1 -- 172.21.15.69:0/1627418322 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f793803d850 msgr2=0x7f793803fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:06.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:06.249+0000 7f79564e0700 1 --2- 172.21.15.69:0/1627418322 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f793803d850 0x7f793803fd60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7944006010 tx=0x7f794401a040 comp rx=0 tx=0).stop 2024-04-25T11:18:06.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:06.249+0000 7f79564e0700 1 -- 172.21.15.69:0/1627418322 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7950105310 msgr2=0x7f79500759e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:06.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:06.249+0000 7f79564e0700 1 --2- 172.21.15.69:0/1627418322 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7950105310 0x7f79500759e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f794000d980 tx=0x7f794000dc90 comp rx=0 tx=0).stop 2024-04-25T11:18:06.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:06.249+0000 7f79564e0700 1 -- 172.21.15.69:0/1627418322 shutdown_connections 2024-04-25T11:18:06.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:06.249+0000 7f79564e0700 1 --2- 172.21.15.69:0/1627418322 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f793803d850 0x7f793803fd60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:06.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:06.249+0000 7f79564e0700 1 --2- 172.21.15.69:0/1627418322 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7950105cb0 0x7f7950075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:06.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:06.249+0000 7f79564e0700 1 --2- 172.21.15.69:0/1627418322 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79501041e0 0x7f7950077330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:06.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:06.249+0000 7f79564e0700 1 --2- 172.21.15.69:0/1627418322 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7950105310 0x7f79500759e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:06.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:06.249+0000 7f79564e0700 1 -- 172.21.15.69:0/1627418322 >> 172.21.15.69:0/1627418322 conn(0x7f79500ff840 msgr2=0x7f79501085b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:06.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:06.249+0000 7f79564e0700 1 -- 172.21.15.69:0/1627418322 shutdown_connections 2024-04-25T11:18:06.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:06.249+0000 7f79564e0700 1 -- 172.21.15.69:0/1627418322 wait complete. 2024-04-25T11:18:06.449 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:06 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:06.449 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:06 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:06.449 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:06 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:18:06.449 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:06 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:18:06.449 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:06 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:06.450 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:06 smithi069 ceph-mon[29188]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:18:06.450 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:06 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:06.450 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:06 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:06.486 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:06 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:06.487 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:06 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:06.487 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:06 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:18:06.487 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:06 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:18:06.487 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:06 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:06.487 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:06 smithi152 ceph-mon[32701]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:18:06.487 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:06 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:06.487 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:06 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:06.605 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:06 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:06.606 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:06 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:06.606 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:06 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:18:06.606 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:06 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:18:06.606 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:06 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:06.606 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:06 smithi146 ceph-mon[32821]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:18:06.606 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:06 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:06.606 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:06 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:06.931 DEBUG:teuthology.orchestra.run.smithi069:osd.0> sudo journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@osd.0.service 2024-04-25T11:18:06.934 INFO:tasks.cephadm:Deploying osd.1 on smithi146 with /dev/vg_nvme/lv_4... 2024-04-25T11:18:06.934 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T11:18:07.176 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T11:18:07.449 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:07 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:18:07.450 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:07 smithi069 ceph-mon[29188]: from='osd.0 [v2:172.21.15.69:6802/3756025235,v1:172.21.15.69:6803/3756025235]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T11:18:07.486 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:07 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:18:07.487 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:07 smithi152 ceph-mon[32701]: from='osd.0 [v2:172.21.15.69:6802/3756025235,v1:172.21.15.69:6803/3756025235]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T11:18:07.519 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:07 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:18:07.519 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:07 smithi146 ceph-mon[32821]: from='osd.0 [v2:172.21.15.69:6802/3756025235,v1:172.21.15.69:6803/3756025235]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T11:18:08.370 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:08 smithi069 ceph-mon[29188]: from='osd.0 [v2:172.21.15.69:6802/3756025235,v1:172.21.15.69:6803/3756025235]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T11:18:08.371 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:08 smithi069 ceph-mon[29188]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T11:18:08.371 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:08 smithi069 ceph-mon[29188]: from='osd.0 [v2:172.21.15.69:6802/3756025235,v1:172.21.15.69:6803/3756025235]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-25T11:18:08.371 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:08 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:18:08.371 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:08 smithi069 ceph-mon[29188]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:18:08.419 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:08 smithi146 ceph-mon[32821]: from='osd.0 [v2:172.21.15.69:6802/3756025235,v1:172.21.15.69:6803/3756025235]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T11:18:08.419 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:08 smithi146 ceph-mon[32821]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T11:18:08.419 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:08 smithi146 ceph-mon[32821]: from='osd.0 [v2:172.21.15.69:6802/3756025235,v1:172.21.15.69:6803/3756025235]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-25T11:18:08.419 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:08 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:18:08.419 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:08 smithi146 ceph-mon[32821]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:18:08.737 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:08 smithi152 ceph-mon[32701]: from='osd.0 [v2:172.21.15.69:6802/3756025235,v1:172.21.15.69:6803/3756025235]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T11:18:08.737 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:08 smithi152 ceph-mon[32701]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T11:18:08.737 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:08 smithi152 ceph-mon[32701]: from='osd.0 [v2:172.21.15.69:6802/3756025235,v1:172.21.15.69:6803/3756025235]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-25T11:18:08.737 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:08 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:18:08.737 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:08 smithi152 ceph-mon[32701]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:18:08.988 INFO:teuthology.orchestra.run.smithi146.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T11:18:08.988 INFO:teuthology.orchestra.run.smithi146.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T11:18:08.989 INFO:teuthology.orchestra.run.smithi146.stderr: stderr: 10+0 records in 2024-04-25T11:18:08.989 INFO:teuthology.orchestra.run.smithi146.stderr:10+0 records out 2024-04-25T11:18:08.989 INFO:teuthology.orchestra.run.smithi146.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114878 s, 913 MB/s 2024-04-25T11:18:08.989 INFO:teuthology.orchestra.run.smithi146.stderr: stderr: 2024-04-25T11:18:08.989 INFO:teuthology.orchestra.run.smithi146.stderr:--> Zapping successful for: 2024-04-25T11:18:09.222 INFO:journalctl@ceph.osd.0.smithi069.stdout:Apr 25 11:18:08 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-0[44381]: 2024-04-25T11:18:08.958+0000 7f4348055700 -1 osd.0 0 waiting for initial osdmap 2024-04-25T11:18:09.222 INFO:journalctl@ceph.osd.0.smithi069.stdout:Apr 25 11:18:08 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-0[44381]: 2024-04-25T11:18:08.962+0000 7f433fe02700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:18:09.552 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:09 smithi069 ceph-mon[29188]: from='osd.0 [v2:172.21.15.69:6802/3756025235,v1:172.21.15.69:6803/3756025235]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-04-25T11:18:09.552 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:09 smithi069 ceph-mon[29188]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T11:18:09.553 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:09 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:18:09.553 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:09 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:18:09.553 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:09 smithi069 ceph-mon[29188]: from='osd.0 [v2:172.21.15.69:6802/3756025235,v1:172.21.15.69:6803/3756025235]' entity='osd.0' 2024-04-25T11:18:09.553 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:09 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:18:09.581 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:09 smithi146 ceph-mon[32821]: from='osd.0 [v2:172.21.15.69:6802/3756025235,v1:172.21.15.69:6803/3756025235]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-04-25T11:18:09.582 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:09 smithi146 ceph-mon[32821]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T11:18:09.582 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:09 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:18:09.582 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:09 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:18:09.582 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:09 smithi146 ceph-mon[32821]: from='osd.0 [v2:172.21.15.69:6802/3756025235,v1:172.21.15.69:6803/3756025235]' entity='osd.0' 2024-04-25T11:18:09.582 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:09 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:18:09.616 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi146:vg_nvme/lv_4 2024-04-25T11:18:09.736 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:09 smithi152 ceph-mon[32701]: from='osd.0 [v2:172.21.15.69:6802/3756025235,v1:172.21.15.69:6803/3756025235]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-04-25T11:18:09.737 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:09 smithi152 ceph-mon[32701]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T11:18:09.737 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:09 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:18:09.737 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:09 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:18:09.737 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:09 smithi152 ceph-mon[32701]: from='osd.0 [v2:172.21.15.69:6802/3756025235,v1:172.21.15.69:6803/3756025235]' entity='osd.0' 2024-04-25T11:18:09.737 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:09 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:18:09.860 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T11:18:10.553 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:10 smithi146 ceph-mon[32821]: purged_snaps scrub starts 2024-04-25T11:18:10.553 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:10 smithi146 ceph-mon[32821]: purged_snaps scrub ok 2024-04-25T11:18:10.553 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:10 smithi146 ceph-mon[32821]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:18:10.553 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:10 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:10.553 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:10 smithi146 ceph-mon[32821]: osd.0 [v2:172.21.15.69:6802/3756025235,v1:172.21.15.69:6803/3756025235] boot 2024-04-25T11:18:10.554 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:10 smithi146 ceph-mon[32821]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T11:18:10.554 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:10 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:18:10.554 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:10 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:10.554 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:10 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:18:10.554 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:10 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:10.554 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:10 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:18:10.554 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:10 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:18:10.554 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:10 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:10.699 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:10 smithi069 ceph-mon[29188]: purged_snaps scrub starts 2024-04-25T11:18:10.699 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:10 smithi069 ceph-mon[29188]: purged_snaps scrub ok 2024-04-25T11:18:10.700 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:10 smithi069 ceph-mon[29188]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:18:10.700 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:10 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:10.700 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:10 smithi069 ceph-mon[29188]: osd.0 [v2:172.21.15.69:6802/3756025235,v1:172.21.15.69:6803/3756025235] boot 2024-04-25T11:18:10.700 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:10 smithi069 ceph-mon[29188]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T11:18:10.700 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:10 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:18:10.700 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:10 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:10.700 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:10 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:18:10.700 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:10 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:10.700 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:10 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:18:10.700 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:10 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:18:10.700 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:10 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:10.737 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:10 smithi152 ceph-mon[32701]: purged_snaps scrub starts 2024-04-25T11:18:10.737 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:10 smithi152 ceph-mon[32701]: purged_snaps scrub ok 2024-04-25T11:18:10.737 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:10 smithi152 ceph-mon[32701]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:18:10.737 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:10 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:10.737 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:10 smithi152 ceph-mon[32701]: osd.0 [v2:172.21.15.69:6802/3756025235,v1:172.21.15.69:6803/3756025235] boot 2024-04-25T11:18:10.737 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:10 smithi152 ceph-mon[32701]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T11:18:10.737 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:10 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:18:10.738 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:10 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:10.738 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:10 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:18:10.738 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:10 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:10.738 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:10 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:18:10.738 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:10 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:18:10.738 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:10 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:11.348 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:11.346+0000 7f635ac5d700 1 -- 172.21.15.146:0/3224519337 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6354105370 msgr2=0x7f6354105780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:11.348 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:11.346+0000 7f635ac5d700 1 --2- 172.21.15.146:0/3224519337 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6354105370 0x7f6354105780 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6344009990 tx=0x7f6344009ca0 comp rx=0 tx=0).stop 2024-04-25T11:18:11.348 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:11.346+0000 7f635ac5d700 1 -- 172.21.15.146:0/3224519337 shutdown_connections 2024-04-25T11:18:11.348 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:11.346+0000 7f635ac5d700 1 --2- 172.21.15.146:0/3224519337 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6354105d10 0x7f635410c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:11.348 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:11.346+0000 7f635ac5d700 1 --2- 172.21.15.146:0/3224519337 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6354104240 0x7f6354104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:11.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:11.346+0000 7f635ac5d700 1 --2- 172.21.15.146:0/3224519337 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6354105370 0x7f6354105780 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:11.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:11.346+0000 7f635ac5d700 1 -- 172.21.15.146:0/3224519337 >> 172.21.15.146:0/3224519337 conn(0x7f63540ff8a0 msgr2=0x7f6354101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:11.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:11.346+0000 7f635ac5d700 1 -- 172.21.15.146:0/3224519337 shutdown_connections 2024-04-25T11:18:11.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:11.346+0000 7f635ac5d700 1 -- 172.21.15.146:0/3224519337 wait complete. 2024-04-25T11:18:11.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:11.347+0000 7f635ac5d700 1 Processor -- start 2024-04-25T11:18:11.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:11.347+0000 7f635ac5d700 1 -- start start 2024-04-25T11:18:11.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:11.347+0000 7f635ac5d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6354104240 0x7f635419a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:11.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:11.347+0000 7f635ac5d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6354105370 0x7f635419ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:11.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:11.347+0000 7f635ac5d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6354105d10 0x7f63541a1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:11.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:11.347+0000 7f635ac5d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f635410fb50 con 0x7f6354105d10 2024-04-25T11:18:11.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:11.347+0000 7f635ac5d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f635410f9d0 con 0x7f6354104240 2024-04-25T11:18:11.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:11.347+0000 7f635ac5d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f635410fcd0 con 0x7f6354105370 2024-04-25T11:18:11.349 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:11.347+0000 7f63589f9700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6354104240 0x7f635419a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:11.350 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:11.347+0000 7f63589f9700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6354104240 0x7f635419a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:38616/0 (socket says 172.21.15.146:38616) 2024-04-25T11:18:11.350 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:11.347+0000 7f63589f9700 1 -- 172.21.15.146:0/529014103 learned_addr learned my addr 172.21.15.146:0/529014103 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-04-25T11:18:11.350 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:11.348+0000 7f63589f9700 1 -- 172.21.15.146:0/529014103 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6354105370 msgr2=0x7f635419ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:11.350 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:11.348+0000 7f63589f9700 1 --2- 172.21.15.146:0/529014103 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6354105370 0x7f635419ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:11.350 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:11.348+0000 7f63589f9700 1 -- 172.21.15.146:0/529014103 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6354105d10 msgr2=0x7f63541a1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:11.350 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:11.348+0000 7f63589f9700 1 --2- 172.21.15.146:0/529014103 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6354105d10 0x7f63541a1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:11.350 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:11.348+0000 7f63589f9700 1 -- 172.21.15.146:0/529014103 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63541a2250 con 0x7f6354104240 2024-04-25T11:18:11.350 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:11.348+0000 7f63589f9700 1 --2- 172.21.15.146:0/529014103 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6354104240 0x7f635419a6f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f635000d950 tx=0x7f635000dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:11.350 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:11.348+0000 7f6349ffb700 1 -- 172.21.15.146:0/529014103 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6350017070 con 0x7f6354104240 2024-04-25T11:18:11.350 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:11.348+0000 7f6349ffb700 1 -- 172.21.15.146:0/529014103 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f635000de30 con 0x7f6354104240 2024-04-25T11:18:11.350 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:11.348+0000 7f635ac5d700 1 -- 172.21.15.146:0/529014103 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63541a2540 con 0x7f6354104240 2024-04-25T11:18:11.350 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:11.348+0000 7f6349ffb700 1 -- 172.21.15.146:0/529014103 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6350003aa0 con 0x7f6354104240 2024-04-25T11:18:11.350 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:11.348+0000 7f635ac5d700 1 -- 172.21.15.146:0/529014103 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63541a2a80 con 0x7f6354104240 2024-04-25T11:18:11.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:11.349+0000 7f635ac5d700 1 -- 172.21.15.146:0/529014103 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6354051190 con 0x7f6354104240 2024-04-25T11:18:11.351 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:11.350+0000 7f6349ffb700 1 -- 172.21.15.146:0/529014103 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 15) ==== 147464+0+0 (secure 0 0 0) 0x7f6350003c40 con 0x7f6354104240 2024-04-25T11:18:11.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:11.351+0000 7f6349ffb700 1 --2- 172.21.15.146:0/529014103 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f633c0afaf0 0x7f633c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:11.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:11.351+0000 7f6349ffb700 1 -- 172.21.15.146:0/529014103 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f63500dcd30 con 0x7f6354104240 2024-04-25T11:18:11.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:11.351+0000 7f634bfff700 1 --2- 172.21.15.146:0/529014103 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f633c0afaf0 0x7f633c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:11.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:11.352+0000 7f634bfff700 1 --2- 172.21.15.146:0/529014103 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f633c0afaf0 0x7f633c0b2000 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f63440098e0 tx=0x7f6344005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:11.355 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:11.356+0000 7f6349ffb700 1 -- 172.21.15.146:0/529014103 <== mon.2 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f63500e1070 con 0x7f6354104240 2024-04-25T11:18:11.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:11.562+0000 7f635ac5d700 1 -- 172.21.15.146:0/529014103 --> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f6354063ad0 con 0x7f633c0afaf0 2024-04-25T11:18:11.562 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:11 smithi146 ceph-mon[32821]: Detected new or changed devices on smithi069 2024-04-25T11:18:11.562 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:11 smithi146 ceph-mon[32821]: Adjusting osd_memory_target on smithi069 to 17088M 2024-04-25T11:18:11.638 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:11 smithi152 ceph-mon[32701]: Detected new or changed devices on smithi069 2024-04-25T11:18:11.638 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:11 smithi152 ceph-mon[32701]: Adjusting osd_memory_target on smithi069 to 17088M 2024-04-25T11:18:11.699 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:11 smithi069 ceph-mon[29188]: Detected new or changed devices on smithi069 2024-04-25T11:18:11.699 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:11 smithi069 ceph-mon[29188]: Adjusting osd_memory_target on smithi069 to 17088M 2024-04-25T11:18:12.409 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:12 smithi146 ceph-mon[32821]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T11:18:12.409 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:12 smithi146 ceph-mon[32821]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:18:12.409 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:12 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:18:12.409 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:12 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:18:12.409 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:12 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:18:12.409 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:12 smithi146 ceph-mon[32821]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:18:12.699 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:12 smithi069 ceph-mon[29188]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T11:18:12.699 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:12 smithi069 ceph-mon[29188]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:18:12.699 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:12 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:18:12.699 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:12 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:18:12.699 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:12 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:18:12.699 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:12 smithi069 ceph-mon[29188]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:18:12.736 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:12 smithi152 ceph-mon[32701]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T11:18:12.737 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:12 smithi152 ceph-mon[32701]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:18:12.737 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:12 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:18:12.737 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:12 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:18:12.737 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:12 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:18:12.737 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:12 smithi152 ceph-mon[32701]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:18:15.106 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:14 smithi146 ceph-mon[32821]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:18:15.106 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:14 smithi146 ceph-mon[32821]: from='client.? 172.21.15.146:0/833333552' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0c66930-4233-493f-b2bb-99c473994b59"}]: dispatch 2024-04-25T11:18:15.106 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:14 smithi146 ceph-mon[32821]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0c66930-4233-493f-b2bb-99c473994b59"}]: dispatch 2024-04-25T11:18:15.106 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:14 smithi146 ceph-mon[32821]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0c66930-4233-493f-b2bb-99c473994b59"}]': finished 2024-04-25T11:18:15.106 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:14 smithi146 ceph-mon[32821]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T11:18:15.106 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:14 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:18:15.106 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:14 smithi146 ceph-mon[32821]: from='client.? 172.21.15.146:0/1362890919' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:18:15.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:14 smithi069 ceph-mon[29188]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:18:15.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:14 smithi069 ceph-mon[29188]: from='client.? 172.21.15.146:0/833333552' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0c66930-4233-493f-b2bb-99c473994b59"}]: dispatch 2024-04-25T11:18:15.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:14 smithi069 ceph-mon[29188]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0c66930-4233-493f-b2bb-99c473994b59"}]: dispatch 2024-04-25T11:18:15.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:14 smithi069 ceph-mon[29188]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0c66930-4233-493f-b2bb-99c473994b59"}]': finished 2024-04-25T11:18:15.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:14 smithi069 ceph-mon[29188]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T11:18:15.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:14 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:18:15.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:14 smithi069 ceph-mon[29188]: from='client.? 172.21.15.146:0/1362890919' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:18:15.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:14 smithi152 ceph-mon[32701]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:18:15.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:14 smithi152 ceph-mon[32701]: from='client.? 172.21.15.146:0/833333552' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0c66930-4233-493f-b2bb-99c473994b59"}]: dispatch 2024-04-25T11:18:15.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:14 smithi152 ceph-mon[32701]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0c66930-4233-493f-b2bb-99c473994b59"}]: dispatch 2024-04-25T11:18:15.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:14 smithi152 ceph-mon[32701]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0c66930-4233-493f-b2bb-99c473994b59"}]': finished 2024-04-25T11:18:15.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:14 smithi152 ceph-mon[32701]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T11:18:15.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:14 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:18:15.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:14 smithi152 ceph-mon[32701]: from='client.? 172.21.15.146:0/1362890919' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:18:17.105 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:16 smithi146 ceph-mon[32821]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:18:17.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:16 smithi069 ceph-mon[29188]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:18:17.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:16 smithi152 ceph-mon[32701]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:18:19.019 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:18 smithi146 ceph-mon[32821]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:18:19.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:18 smithi069 ceph-mon[29188]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:18:19.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:18 smithi152 ceph-mon[32701]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:18:21.031 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:20 smithi146 ceph-mon[32821]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:18:21.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:20 smithi069 ceph-mon[29188]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:18:21.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:20 smithi152 ceph-mon[32701]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:18:21.941 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:21 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T11:18:21.942 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:21 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:18:22.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:21 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T11:18:22.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:21 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:18:22.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:21 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T11:18:22.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:21 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:18:23.085 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:22 smithi146 ceph-mon[32821]: Deploying daemon osd.1 on smithi146 2024-04-25T11:18:23.085 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:22 smithi146 ceph-mon[32821]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:18:23.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:22 smithi069 ceph-mon[29188]: Deploying daemon osd.1 on smithi146 2024-04-25T11:18:23.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:22 smithi069 ceph-mon[29188]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:18:23.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:22 smithi152 ceph-mon[32701]: Deploying daemon osd.1 on smithi146 2024-04-25T11:18:23.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:22 smithi152 ceph-mon[32701]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:18:25.105 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:24 smithi146 ceph-mon[32821]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:18:25.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:24 smithi069 ceph-mon[29188]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:18:25.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:24 smithi152 ceph-mon[32701]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:18:27.009 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:26 smithi146 ceph-mon[32821]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:18:27.010 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:26 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:27.010 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:26 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:27.010 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:26 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:18:27.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:26 smithi069 ceph-mon[29188]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:18:27.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:26 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:27.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:26 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:27.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:26 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:18:27.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:26 smithi152 ceph-mon[32701]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:18:27.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:26 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:27.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:26 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:27.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:26 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:18:28.917 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:28 smithi146 ceph-mon[32821]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:18:29.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:28 smithi069 ceph-mon[29188]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:18:29.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:28 smithi152 ceph-mon[32701]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:18:29.575 INFO:teuthology.orchestra.run.smithi146.stdout:Created osd(s) 1 on host 'smithi146' 2024-04-25T11:18:29.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:29.575+0000 7f6349ffb700 1 -- 172.21.15.146:0/529014103 <== mgr.14152 v2:172.21.15.69:6800/4220065409 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6354063ad0 con 0x7f633c0afaf0 2024-04-25T11:18:29.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:29.577+0000 7f635ac5d700 1 -- 172.21.15.146:0/529014103 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f633c0afaf0 msgr2=0x7f633c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:29.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:29.578+0000 7f635ac5d700 1 --2- 172.21.15.146:0/529014103 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f633c0afaf0 0x7f633c0b2000 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f63440098e0 tx=0x7f6344005fb0 comp rx=0 tx=0).stop 2024-04-25T11:18:29.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:29.578+0000 7f635ac5d700 1 -- 172.21.15.146:0/529014103 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6354104240 msgr2=0x7f635419a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:29.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:29.578+0000 7f635ac5d700 1 --2- 172.21.15.146:0/529014103 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6354104240 0x7f635419a6f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f635000d950 tx=0x7f635000dd10 comp rx=0 tx=0).stop 2024-04-25T11:18:29.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:29.578+0000 7f635ac5d700 1 -- 172.21.15.146:0/529014103 shutdown_connections 2024-04-25T11:18:29.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:29.578+0000 7f635ac5d700 1 --2- 172.21.15.146:0/529014103 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f633c0afaf0 0x7f633c0b2000 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:29.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:29.578+0000 7f635ac5d700 1 --2- 172.21.15.146:0/529014103 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6354105d10 0x7f63541a1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:29.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:29.578+0000 7f635ac5d700 1 --2- 172.21.15.146:0/529014103 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6354104240 0x7f635419a6f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:29.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:29.578+0000 7f635ac5d700 1 --2- 172.21.15.146:0/529014103 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6354105370 0x7f635419ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:29.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:29.578+0000 7f635ac5d700 1 -- 172.21.15.146:0/529014103 >> 172.21.15.146:0/529014103 conn(0x7f63540ff8a0 msgr2=0x7f63541085a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:29.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:29.578+0000 7f635ac5d700 1 -- 172.21.15.146:0/529014103 shutdown_connections 2024-04-25T11:18:29.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:18:29.578+0000 7f635ac5d700 1 -- 172.21.15.146:0/529014103 wait complete. 2024-04-25T11:18:30.184 DEBUG:teuthology.orchestra.run.smithi146:osd.1> sudo journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@osd.1.service 2024-04-25T11:18:30.186 INFO:tasks.cephadm:Deploying osd.2 on smithi152 with /dev/vg_nvme/lv_4... 2024-04-25T11:18:30.186 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T11:18:30.479 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T11:18:30.500 INFO:journalctl@ceph.osd.1.smithi146.stdout:Apr 25 11:18:30 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-1[37409]: 2024-04-25T11:18:30.259+0000 7f26982be7c0 -1 osd.1 0 log_to_monitors true 2024-04-25T11:18:30.799 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:30 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:30.799 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:30 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:30.799 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:30 smithi146 ceph-mon[32821]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:18:30.799 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:30 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:30.800 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:30 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:30.800 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:30 smithi146 ceph-mon[32821]: from='osd.1 [v2:172.21.15.146:6800/3166305213,v1:172.21.15.146:6801/3166305213]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T11:18:30.800 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:30 smithi146 ceph-mon[32821]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T11:18:30.894 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:30 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:30.894 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:30 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:30.895 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:30 smithi152 ceph-mon[32701]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:18:30.895 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:30 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:30.895 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:30 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:30.895 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:30 smithi152 ceph-mon[32701]: from='osd.1 [v2:172.21.15.146:6800/3166305213,v1:172.21.15.146:6801/3166305213]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T11:18:30.895 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:30 smithi152 ceph-mon[32701]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T11:18:30.949 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:30 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:30.949 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:30 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:30.949 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:30 smithi069 ceph-mon[29188]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:18:30.949 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:30 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:30.949 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:30 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:30.949 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:30 smithi069 ceph-mon[29188]: from='osd.1 [v2:172.21.15.146:6800/3166305213,v1:172.21.15.146:6801/3166305213]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T11:18:30.949 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:30 smithi069 ceph-mon[29188]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T11:18:31.986 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:31 smithi152 ceph-mon[32701]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T11:18:31.986 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:31 smithi152 ceph-mon[32701]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T11:18:31.986 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:31 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:18:31.986 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:31 smithi152 ceph-mon[32701]: from='osd.1 [v2:172.21.15.146:6800/3166305213,v1:172.21.15.146:6801/3166305213]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-04-25T11:18:31.987 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:31 smithi152 ceph-mon[32701]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-04-25T11:18:31.987 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:31 smithi152 ceph-mon[32701]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:18:32.105 INFO:journalctl@ceph.osd.1.smithi146.stdout:Apr 25 11:18:31 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-1[37409]: 2024-04-25T11:18:31.891+0000 7f268d923700 -1 osd.1 0 waiting for initial osdmap 2024-04-25T11:18:32.105 INFO:journalctl@ceph.osd.1.smithi146.stdout:Apr 25 11:18:31 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-1[37409]: 2024-04-25T11:18:31.894+0000 7f26896d8700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:18:32.105 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:31 smithi146 ceph-mon[32821]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T11:18:32.105 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:31 smithi146 ceph-mon[32821]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T11:18:32.105 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:31 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:18:32.105 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:31 smithi146 ceph-mon[32821]: from='osd.1 [v2:172.21.15.146:6800/3166305213,v1:172.21.15.146:6801/3166305213]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-04-25T11:18:32.106 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:31 smithi146 ceph-mon[32821]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-04-25T11:18:32.106 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:31 smithi146 ceph-mon[32821]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:18:32.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:31 smithi069 ceph-mon[29188]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T11:18:32.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:31 smithi069 ceph-mon[29188]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T11:18:32.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:31 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:18:32.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:31 smithi069 ceph-mon[29188]: from='osd.1 [v2:172.21.15.146:6800/3166305213,v1:172.21.15.146:6801/3166305213]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-04-25T11:18:32.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:31 smithi069 ceph-mon[29188]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-04-25T11:18:32.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:31 smithi069 ceph-mon[29188]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:18:32.257 INFO:teuthology.orchestra.run.smithi152.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T11:18:32.257 INFO:teuthology.orchestra.run.smithi152.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T11:18:32.257 INFO:teuthology.orchestra.run.smithi152.stderr: stderr: 10+0 records in 2024-04-25T11:18:32.257 INFO:teuthology.orchestra.run.smithi152.stderr:10+0 records out 2024-04-25T11:18:32.257 INFO:teuthology.orchestra.run.smithi152.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114233 s, 918 MB/s 2024-04-25T11:18:32.257 INFO:teuthology.orchestra.run.smithi152.stderr:--> Zapping successful for: 2024-04-25T11:18:32.829 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi152:vg_nvme/lv_4 2024-04-25T11:18:33.059 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:32 smithi146 ceph-mon[32821]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-04-25T11:18:33.059 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:32 smithi146 ceph-mon[32821]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T11:18:33.059 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:32 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:18:33.059 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:32 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:18:33.064 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T11:18:33.186 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:32 smithi152 ceph-mon[32701]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-04-25T11:18:33.186 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:32 smithi152 ceph-mon[32701]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T11:18:33.186 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:32 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:18:33.186 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:32 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:18:33.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:32 smithi069 ceph-mon[29188]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-04-25T11:18:33.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:32 smithi069 ceph-mon[29188]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T11:18:33.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:32 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:18:33.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:32 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:18:34.167 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:33 smithi152 ceph-mon[32701]: purged_snaps scrub starts 2024-04-25T11:18:34.167 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:33 smithi152 ceph-mon[32701]: purged_snaps scrub ok 2024-04-25T11:18:34.167 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:33 smithi152 ceph-mon[32701]: osd.1 [v2:172.21.15.146:6800/3166305213,v1:172.21.15.146:6801/3166305213] boot 2024-04-25T11:18:34.167 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:33 smithi152 ceph-mon[32701]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T11:18:34.167 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:33 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:18:34.167 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:33 smithi152 ceph-mon[32701]: Detected new or changed devices on smithi146 2024-04-25T11:18:34.168 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:33 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:34.168 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:33 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:34.168 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:33 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:18:34.168 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:33 smithi152 ceph-mon[32701]: Adjusting osd_memory_target on smithi146 to 17088M 2024-04-25T11:18:34.168 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:33 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:34.168 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:33 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:18:34.168 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:33 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:18:34.168 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:33 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:34.168 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:33 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:18:34.168 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:33 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:18:34.168 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:33 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:18:34.168 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:33 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:34.168 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:33 smithi152 ceph-mon[32701]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:18:34.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:33 smithi069 ceph-mon[29188]: purged_snaps scrub starts 2024-04-25T11:18:34.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:33 smithi069 ceph-mon[29188]: purged_snaps scrub ok 2024-04-25T11:18:34.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:33 smithi069 ceph-mon[29188]: osd.1 [v2:172.21.15.146:6800/3166305213,v1:172.21.15.146:6801/3166305213] boot 2024-04-25T11:18:34.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:33 smithi069 ceph-mon[29188]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T11:18:34.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:33 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:18:34.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:33 smithi069 ceph-mon[29188]: Detected new or changed devices on smithi146 2024-04-25T11:18:34.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:33 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:34.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:33 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:34.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:33 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:18:34.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:33 smithi069 ceph-mon[29188]: Adjusting osd_memory_target on smithi146 to 17088M 2024-04-25T11:18:34.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:33 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:34.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:33 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:18:34.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:33 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:18:34.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:33 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:34.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:33 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:18:34.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:33 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:18:34.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:33 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:18:34.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:33 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:34.200 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:33 smithi069 ceph-mon[29188]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:18:34.355 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:33 smithi146 ceph-mon[32821]: purged_snaps scrub starts 2024-04-25T11:18:34.355 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:33 smithi146 ceph-mon[32821]: purged_snaps scrub ok 2024-04-25T11:18:34.355 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:33 smithi146 ceph-mon[32821]: osd.1 [v2:172.21.15.146:6800/3166305213,v1:172.21.15.146:6801/3166305213] boot 2024-04-25T11:18:34.355 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:33 smithi146 ceph-mon[32821]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T11:18:34.355 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:33 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:18:34.355 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:33 smithi146 ceph-mon[32821]: Detected new or changed devices on smithi146 2024-04-25T11:18:34.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:33 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:34.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:33 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:34.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:33 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:18:34.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:33 smithi146 ceph-mon[32821]: Adjusting osd_memory_target on smithi146 to 17088M 2024-04-25T11:18:34.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:33 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:34.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:33 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:18:34.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:33 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:18:34.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:33 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:34.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:33 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:18:34.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:33 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:18:34.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:33 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:18:34.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:33 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:34.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:33 smithi146 ceph-mon[32821]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:18:34.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:34.675+0000 7f987b222700 1 -- 172.21.15.152:0/1959374173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9874109ed0 msgr2=0x7f987410c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:34.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:34.675+0000 7f987b222700 1 --2- 172.21.15.152:0/1959374173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9874109ed0 0x7f987410c3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f986c00b0a0 tx=0x7f986c00b3b0 comp rx=0 tx=0).stop 2024-04-25T11:18:34.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:34.676+0000 7f987b222700 1 -- 172.21.15.152:0/1959374173 shutdown_connections 2024-04-25T11:18:34.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:34.676+0000 7f987b222700 1 --2- 172.21.15.152:0/1959374173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9874109ed0 0x7f987410c3e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:34.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:34.676+0000 7f987b222700 1 --2- 172.21.15.152:0/1959374173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9874076540 0x7f9874106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:34.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:34.676+0000 7f987b222700 1 --2- 172.21.15.152:0/1959374173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98741073a0 0x7f9874109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:34.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:34.676+0000 7f987b222700 1 -- 172.21.15.152:0/1959374173 >> 172.21.15.152:0/1959374173 conn(0x7f98740fe760 msgr2=0x7f9874100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:34.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:34.676+0000 7f987b222700 1 -- 172.21.15.152:0/1959374173 shutdown_connections 2024-04-25T11:18:34.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:34.676+0000 7f987b222700 1 -- 172.21.15.152:0/1959374173 wait complete. 2024-04-25T11:18:34.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:34.676+0000 7f987b222700 1 Processor -- start 2024-04-25T11:18:34.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:34.676+0000 7f987b222700 1 -- start start 2024-04-25T11:18:34.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:34.676+0000 7f987b222700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9874076540 0x7f987419a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:34.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:34.676+0000 7f987b222700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98741073a0 0x7f987419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:34.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:34.676+0000 7f987b222700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9874109ed0 0x7f98741a1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:34.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:34.676+0000 7f987b222700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f987410fb40 con 0x7f9874076540 2024-04-25T11:18:34.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:34.676+0000 7f987b222700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f987410f9c0 con 0x7f9874109ed0 2024-04-25T11:18:34.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:34.676+0000 7f987b222700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f987410fcc0 con 0x7f98741073a0 2024-04-25T11:18:34.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:34.677+0000 7f9873fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98741073a0 0x7f987419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:34.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:34.677+0000 7f9873fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98741073a0 0x7f987419aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43428/0 (socket says 172.21.15.152:43428) 2024-04-25T11:18:34.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:34.677+0000 7f9873fff700 1 -- 172.21.15.152:0/1673063989 learned_addr learned my addr 172.21.15.152:0/1673063989 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-25T11:18:34.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:34.677+0000 7f9878fbe700 1 --2- 172.21.15.152:0/1673063989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9874076540 0x7f987419a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:34.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:34.677+0000 7f98797bf700 1 --2- 172.21.15.152:0/1673063989 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9874109ed0 0x7f98741a1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:34.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:34.677+0000 7f9873fff700 1 -- 172.21.15.152:0/1673063989 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9874109ed0 msgr2=0x7f98741a1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:34.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:34.677+0000 7f9873fff700 1 --2- 172.21.15.152:0/1673063989 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9874109ed0 0x7f98741a1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:34.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:34.677+0000 7f9873fff700 1 -- 172.21.15.152:0/1673063989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9874076540 msgr2=0x7f987419a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:34.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:34.677+0000 7f9873fff700 1 --2- 172.21.15.152:0/1673063989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9874076540 0x7f987419a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:34.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:34.677+0000 7f9873fff700 1 -- 172.21.15.152:0/1673063989 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98741a21c0 con 0x7f98741073a0 2024-04-25T11:18:34.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:34.677+0000 7f9878fbe700 1 --2- 172.21.15.152:0/1673063989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9874076540 0x7f987419a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:18:34.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:34.677+0000 7f9873fff700 1 --2- 172.21.15.152:0/1673063989 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98741073a0 0x7f987419aba0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f986000d980 tx=0x7f986000dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:34.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:34.677+0000 7f9871ffb700 1 -- 172.21.15.152:0/1673063989 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9860017070 con 0x7f98741073a0 2024-04-25T11:18:34.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:34.677+0000 7f9871ffb700 1 -- 172.21.15.152:0/1673063989 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9860004830 con 0x7f98741073a0 2024-04-25T11:18:34.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:34.677+0000 7f987b222700 1 -- 172.21.15.152:0/1673063989 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98741a24b0 con 0x7f98741073a0 2024-04-25T11:18:34.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:34.677+0000 7f9871ffb700 1 -- 172.21.15.152:0/1673063989 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98600053a0 con 0x7f98741073a0 2024-04-25T11:18:34.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:34.678+0000 7f987b222700 1 -- 172.21.15.152:0/1673063989 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98741a29f0 con 0x7f98741073a0 2024-04-25T11:18:34.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:34.678+0000 7f987b222700 1 -- 172.21.15.152:0/1673063989 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9874051190 con 0x7f98741073a0 2024-04-25T11:18:34.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:34.680+0000 7f9871ffb700 1 -- 172.21.15.152:0/1673063989 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 15) ==== 147464+0+0 (secure 0 0 0) 0x7f98600049e0 con 0x7f98741073a0 2024-04-25T11:18:34.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:34.680+0000 7f9871ffb700 1 --2- 172.21.15.152:0/1673063989 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f98640afb70 0x7f98640b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:34.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:34.681+0000 7f9871ffb700 1 -- 172.21.15.152:0/1673063989 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f98600dd6d0 con 0x7f98741073a0 2024-04-25T11:18:34.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:34.681+0000 7f9878fbe700 1 --2- 172.21.15.152:0/1673063989 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f98640afb70 0x7f98640b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:34.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:34.681+0000 7f9878fbe700 1 --2- 172.21.15.152:0/1673063989 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f98640afb70 0x7f98640b2080 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9868005950 tx=0x7f9868005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:34.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:34.684+0000 7f9871ffb700 1 -- 172.21.15.152:0/1673063989 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9860020020 con 0x7f98741073a0 2024-04-25T11:18:34.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:34.891+0000 7f987b222700 1 -- 172.21.15.152:0/1673063989 --> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f9874063ad0 con 0x7f98640afb70 2024-04-25T11:18:35.486 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:35 smithi152 ceph-mon[32701]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T11:18:35.486 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:35 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:18:35.487 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:35 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:18:35.487 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:35 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:18:35.605 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:35 smithi146 ceph-mon[32821]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T11:18:35.605 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:35 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:18:35.605 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:35 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:18:35.605 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:35 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:18:35.699 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:35 smithi069 ceph-mon[29188]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T11:18:35.699 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:35 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:18:35.699 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:35 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:18:35.699 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:35 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:18:36.433 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:36 smithi152 ceph-mon[32701]: from='client.24148 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:18:36.434 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:36 smithi152 ceph-mon[32701]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:18:36.605 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:36 smithi146 ceph-mon[32821]: from='client.24148 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:18:36.605 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:36 smithi146 ceph-mon[32821]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:18:36.698 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:36 smithi069 ceph-mon[29188]: from='client.24148 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:18:36.699 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:36 smithi069 ceph-mon[29188]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:18:37.900 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:37 smithi152 ceph-mon[32701]: from='client.? 172.21.15.152:0/1097921206' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4ef735b-9786-4cb9-83ef-7c3f0691a729"}]: dispatch 2024-04-25T11:18:37.900 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:37 smithi152 ceph-mon[32701]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4ef735b-9786-4cb9-83ef-7c3f0691a729"}]: dispatch 2024-04-25T11:18:37.901 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:37 smithi152 ceph-mon[32701]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d4ef735b-9786-4cb9-83ef-7c3f0691a729"}]': finished 2024-04-25T11:18:37.901 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:37 smithi152 ceph-mon[32701]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T11:18:37.901 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:37 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T11:18:37.948 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:37 smithi069 ceph-mon[29188]: from='client.? 172.21.15.152:0/1097921206' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4ef735b-9786-4cb9-83ef-7c3f0691a729"}]: dispatch 2024-04-25T11:18:37.949 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:37 smithi069 ceph-mon[29188]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4ef735b-9786-4cb9-83ef-7c3f0691a729"}]: dispatch 2024-04-25T11:18:37.949 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:37 smithi069 ceph-mon[29188]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d4ef735b-9786-4cb9-83ef-7c3f0691a729"}]': finished 2024-04-25T11:18:37.949 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:37 smithi069 ceph-mon[29188]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T11:18:37.949 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:37 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T11:18:38.105 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:37 smithi146 ceph-mon[32821]: from='client.? 172.21.15.152:0/1097921206' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4ef735b-9786-4cb9-83ef-7c3f0691a729"}]: dispatch 2024-04-25T11:18:38.105 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:37 smithi146 ceph-mon[32821]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4ef735b-9786-4cb9-83ef-7c3f0691a729"}]: dispatch 2024-04-25T11:18:38.105 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:37 smithi146 ceph-mon[32821]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d4ef735b-9786-4cb9-83ef-7c3f0691a729"}]': finished 2024-04-25T11:18:38.105 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:37 smithi146 ceph-mon[32821]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T11:18:38.105 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:37 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T11:18:38.948 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:38 smithi069 ceph-mon[29188]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:18:38.948 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:38 smithi069 ceph-mon[29188]: from='client.? 172.21.15.152:0/1027096383' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:18:38.986 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:38 smithi152 ceph-mon[32701]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:18:38.986 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:38 smithi152 ceph-mon[32701]: from='client.? 172.21.15.152:0/1027096383' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:18:39.104 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:38 smithi146 ceph-mon[32821]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:18:39.105 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:38 smithi146 ceph-mon[32821]: from='client.? 172.21.15.152:0/1027096383' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:18:41.104 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:40 smithi146 ceph-mon[32821]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:18:41.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:40 smithi069 ceph-mon[29188]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:18:41.220 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:40 smithi152 ceph-mon[32701]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:18:42.986 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:42 smithi152 ceph-mon[32701]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:18:43.104 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:42 smithi146 ceph-mon[32821]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:18:43.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:42 smithi069 ceph-mon[29188]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:18:44.853 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:44 smithi152 ceph-mon[32701]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:18:44.853 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:44 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T11:18:44.853 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:44 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:18:45.105 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:44 smithi146 ceph-mon[32821]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:18:45.105 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:44 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T11:18:45.105 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:44 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:18:45.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:44 smithi069 ceph-mon[29188]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:18:45.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:44 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T11:18:45.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:44 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:18:45.941 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:45 smithi152 ceph-mon[32701]: Deploying daemon osd.2 on smithi152 2024-04-25T11:18:46.104 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:45 smithi146 ceph-mon[32821]: Deploying daemon osd.2 on smithi152 2024-04-25T11:18:46.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:45 smithi069 ceph-mon[29188]: Deploying daemon osd.2 on smithi152 2024-04-25T11:18:46.876 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:46 smithi152 ceph-mon[32701]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:18:47.104 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:46 smithi146 ceph-mon[32821]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:18:47.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:46 smithi069 ceph-mon[29188]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:18:49.104 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:48 smithi146 ceph-mon[32821]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:18:49.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:48 smithi069 ceph-mon[29188]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:18:49.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:48 smithi152 ceph-mon[32701]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:18:51.043 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:50 smithi152 ceph-mon[32701]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:18:51.043 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:50 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:18:51.043 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:50 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:51.043 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:50 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:51.104 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:50 smithi146 ceph-mon[32821]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:18:51.104 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:50 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:18:51.104 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:50 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:51.104 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:50 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:51.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:50 smithi069 ceph-mon[29188]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:18:51.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:50 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:18:51.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:50 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:51.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:50 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:53.104 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:52 smithi146 ceph-mon[32821]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:18:53.167 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:52 smithi152 ceph-mon[32701]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:18:53.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:52 smithi069 ceph-mon[29188]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:18:53.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:53.806+0000 7f9871ffb700 1 -- 172.21.15.152:0/1673063989 <== mgr.14152 v2:172.21.15.69:6800/4220065409 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9874063ad0 con 0x7f98640afb70 2024-04-25T11:18:53.805 INFO:teuthology.orchestra.run.smithi152.stdout:Created osd(s) 2 on host 'smithi152' 2024-04-25T11:18:53.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:53.808+0000 7f987b222700 1 -- 172.21.15.152:0/1673063989 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f98640afb70 msgr2=0x7f98640b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:53.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:53.808+0000 7f987b222700 1 --2- 172.21.15.152:0/1673063989 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f98640afb70 0x7f98640b2080 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9868005950 tx=0x7f9868005f50 comp rx=0 tx=0).stop 2024-04-25T11:18:53.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:53.808+0000 7f987b222700 1 -- 172.21.15.152:0/1673063989 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98741073a0 msgr2=0x7f987419aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:53.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:53.808+0000 7f987b222700 1 --2- 172.21.15.152:0/1673063989 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98741073a0 0x7f987419aba0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f986000d980 tx=0x7f986000dd40 comp rx=0 tx=0).stop 2024-04-25T11:18:53.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:53.808+0000 7f987b222700 1 -- 172.21.15.152:0/1673063989 shutdown_connections 2024-04-25T11:18:53.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:53.808+0000 7f987b222700 1 --2- 172.21.15.152:0/1673063989 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f98640afb70 0x7f98640b2080 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:53.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:53.808+0000 7f987b222700 1 --2- 172.21.15.152:0/1673063989 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9874109ed0 0x7f98741a1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:53.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:53.808+0000 7f987b222700 1 --2- 172.21.15.152:0/1673063989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9874076540 0x7f987419a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:53.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:53.808+0000 7f987b222700 1 --2- 172.21.15.152:0/1673063989 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98741073a0 0x7f987419aba0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:53.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:53.808+0000 7f987b222700 1 -- 172.21.15.152:0/1673063989 >> 172.21.15.152:0/1673063989 conn(0x7f98740fe760 msgr2=0x7f9874105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:53.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:53.808+0000 7f987b222700 1 -- 172.21.15.152:0/1673063989 shutdown_connections 2024-04-25T11:18:53.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:18:53.808+0000 7f987b222700 1 -- 172.21.15.152:0/1673063989 wait complete. 2024-04-25T11:18:54.104 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:53 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:54.105 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:53 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:54.105 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:53 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:18:54.105 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:53 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:18:54.105 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:53 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:54.105 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:53 smithi146 ceph-mon[32821]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:18:54.105 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:53 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:54.105 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:53 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:54.105 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:53 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:18:54.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:53 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:54.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:53 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:54.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:53 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:18:54.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:53 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:18:54.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:53 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:54.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:53 smithi069 ceph-mon[29188]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:18:54.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:53 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:54.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:53 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:54.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:53 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:18:54.200 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:53 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:54.200 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:53 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:54.200 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:53 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:18:54.200 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:53 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:18:54.200 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:53 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:54.200 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:53 smithi152 ceph-mon[32701]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:18:54.200 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:53 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:54.200 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:53 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:54.200 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:53 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:18:54.552 DEBUG:teuthology.orchestra.run.smithi152:osd.2> sudo journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@osd.2.service 2024-04-25T11:18:54.554 INFO:tasks.cephadm:Deploying osd.3 on smithi069 with /dev/vg_nvme/lv_3... 2024-04-25T11:18:54.555 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T11:18:54.801 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:18:55.167 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:54 smithi069 ceph-mon[29188]: from='osd.2 [v2:172.21.15.152:6800/1474319175,v1:172.21.15.152:6801/1474319175]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T11:18:55.167 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:54 smithi069 ceph-mon[29188]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T11:18:55.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:54 smithi152 ceph-mon[32701]: from='osd.2 [v2:172.21.15.152:6800/1474319175,v1:172.21.15.152:6801/1474319175]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T11:18:55.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:54 smithi152 ceph-mon[32701]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T11:18:55.354 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:54 smithi146 ceph-mon[32821]: from='osd.2 [v2:172.21.15.152:6800/1474319175,v1:172.21.15.152:6801/1474319175]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T11:18:55.354 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:54 smithi146 ceph-mon[32821]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T11:18:55.986 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:55 smithi152 ceph-mon[32701]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T11:18:55.986 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:55 smithi152 ceph-mon[32701]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T11:18:55.986 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:55 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T11:18:55.986 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:55 smithi152 ceph-mon[32701]: from='osd.2 [v2:172.21.15.152:6800/1474319175,v1:172.21.15.152:6801/1474319175]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T11:18:55.986 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:55 smithi152 ceph-mon[32701]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T11:18:55.986 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:55 smithi152 ceph-mon[32701]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:18:55.986 INFO:journalctl@ceph.osd.2.smithi152.stdout:Apr 25 11:18:55 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-2[37847]: 2024-04-25T11:18:55.888+0000 7f794fb01700 -1 osd.2 0 waiting for initial osdmap 2024-04-25T11:18:55.987 INFO:journalctl@ceph.osd.2.smithi152.stdout:Apr 25 11:18:55 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-2[37847]: 2024-04-25T11:18:55.891+0000 7f794b8b6700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:18:56.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:55 smithi069 ceph-mon[29188]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T11:18:56.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:55 smithi069 ceph-mon[29188]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T11:18:56.196 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:55 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T11:18:56.196 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:55 smithi069 ceph-mon[29188]: from='osd.2 [v2:172.21.15.152:6800/1474319175,v1:172.21.15.152:6801/1474319175]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T11:18:56.196 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:55 smithi069 ceph-mon[29188]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T11:18:56.196 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:55 smithi069 ceph-mon[29188]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:18:56.354 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:55 smithi146 ceph-mon[32821]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T11:18:56.354 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:55 smithi146 ceph-mon[32821]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T11:18:56.354 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:55 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T11:18:56.355 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:55 smithi146 ceph-mon[32821]: from='osd.2 [v2:172.21.15.152:6800/1474319175,v1:172.21.15.152:6801/1474319175]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T11:18:56.355 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:55 smithi146 ceph-mon[32821]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T11:18:56.355 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:55 smithi146 ceph-mon[32821]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:18:56.949 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T11:18:56.949 INFO:teuthology.orchestra.run.smithi069.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T11:18:56.949 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10+0 records in 2024-04-25T11:18:56.949 INFO:teuthology.orchestra.run.smithi069.stderr:10+0 records out 2024-04-25T11:18:56.949 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114596 s, 915 MB/s 2024-04-25T11:18:56.949 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping successful for: 2024-04-25T11:18:57.103 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:56 smithi152 ceph-mon[32701]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-04-25T11:18:57.104 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:56 smithi152 ceph-mon[32701]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T11:18:57.104 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:56 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T11:18:57.104 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:56 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T11:18:57.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:56 smithi069 ceph-mon[29188]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-04-25T11:18:57.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:56 smithi069 ceph-mon[29188]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T11:18:57.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:56 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T11:18:57.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:56 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T11:18:57.354 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:56 smithi146 ceph-mon[32821]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-04-25T11:18:57.354 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:56 smithi146 ceph-mon[32821]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T11:18:57.354 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:56 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T11:18:57.354 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:56 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T11:18:57.676 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi069:vg_nvme/lv_3 2024-04-25T11:18:57.920 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:18:57.948 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:57 smithi069 ceph-mon[29188]: purged_snaps scrub starts 2024-04-25T11:18:57.948 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:57 smithi069 ceph-mon[29188]: purged_snaps scrub ok 2024-04-25T11:18:57.948 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:57 smithi069 ceph-mon[29188]: osd.2 [v2:172.21.15.152:6800/1474319175,v1:172.21.15.152:6801/1474319175] boot 2024-04-25T11:18:57.949 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:57 smithi069 ceph-mon[29188]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T11:18:57.949 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:57 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T11:18:57.949 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:57 smithi069 ceph-mon[29188]: Detected new or changed devices on smithi152 2024-04-25T11:18:57.949 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:57 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:57.949 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:57 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:57.949 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:57 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:18:57.949 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:57 smithi069 ceph-mon[29188]: Adjusting osd_memory_target on smithi152 to 17088M 2024-04-25T11:18:57.949 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:57 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:57.949 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:57 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:18:57.949 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:57 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:18:57.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:57 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:57.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:57 smithi069 ceph-mon[29188]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:18:57.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:57 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T11:18:58.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:57 smithi152 ceph-mon[32701]: purged_snaps scrub starts 2024-04-25T11:18:58.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:57 smithi152 ceph-mon[32701]: purged_snaps scrub ok 2024-04-25T11:18:58.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:57 smithi152 ceph-mon[32701]: osd.2 [v2:172.21.15.152:6800/1474319175,v1:172.21.15.152:6801/1474319175] boot 2024-04-25T11:18:58.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:57 smithi152 ceph-mon[32701]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T11:18:58.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:57 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T11:18:58.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:57 smithi152 ceph-mon[32701]: Detected new or changed devices on smithi152 2024-04-25T11:18:58.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:57 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:58.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:57 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:58.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:57 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:18:58.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:57 smithi152 ceph-mon[32701]: Adjusting osd_memory_target on smithi152 to 17088M 2024-04-25T11:18:58.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:57 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:58.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:57 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:18:58.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:57 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:18:58.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:57 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:58.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:57 smithi152 ceph-mon[32701]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:18:58.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:57 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T11:18:58.354 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:57 smithi146 ceph-mon[32821]: purged_snaps scrub starts 2024-04-25T11:18:58.354 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:57 smithi146 ceph-mon[32821]: purged_snaps scrub ok 2024-04-25T11:18:58.355 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:57 smithi146 ceph-mon[32821]: osd.2 [v2:172.21.15.152:6800/1474319175,v1:172.21.15.152:6801/1474319175] boot 2024-04-25T11:18:58.355 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:57 smithi146 ceph-mon[32821]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T11:18:58.355 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:57 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T11:18:58.355 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:57 smithi146 ceph-mon[32821]: Detected new or changed devices on smithi152 2024-04-25T11:18:58.355 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:57 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:58.355 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:57 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:58.355 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:57 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:18:58.355 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:57 smithi146 ceph-mon[32821]: Adjusting osd_memory_target on smithi152 to 17088M 2024-04-25T11:18:58.355 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:57 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:58.355 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:57 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:18:58.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:57 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:18:58.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:57 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:18:58.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:57 smithi146 ceph-mon[32821]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:18:58.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:57 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T11:18:58.948 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:58 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T11:18:58.948 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:58 smithi069 ceph-mon[29188]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T11:18:58.948 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:58 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T11:18:59.201 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:58 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T11:18:59.617 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:58 smithi152 ceph-mon[32701]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T11:18:59.617 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:58 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T11:18:59.618 INFO:journalctl@ceph.osd.2.smithi152.stdout:Apr 25 11:18:59 smithi152 sudo[41453]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T11:18:59.618 INFO:journalctl@ceph.osd.2.smithi152.stdout:Apr 25 11:18:59 smithi152 sudo[41453]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T11:18:59.618 INFO:journalctl@ceph.osd.2.smithi152.stdout:Apr 25 11:18:59 smithi152 sudo[41453]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T11:18:59.618 INFO:journalctl@ceph.osd.2.smithi152.stdout:Apr 25 11:18:59 smithi152 sudo[41453]: pam_unix(sudo:session): session closed for user root 2024-04-25T11:18:59.618 INFO:journalctl@ceph.osd.2.smithi152.stdout:Apr 25 11:18:59 smithi152 sudo[41456]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T11:18:59.618 INFO:journalctl@ceph.osd.2.smithi152.stdout:Apr 25 11:18:59 smithi152 sudo[41456]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T11:18:59.618 INFO:journalctl@ceph.osd.2.smithi152.stdout:Apr 25 11:18:59 smithi152 sudo[41456]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T11:18:59.618 INFO:journalctl@ceph.osd.2.smithi152.stdout:Apr 25 11:18:59 smithi152 sudo[41456]: pam_unix(sudo:session): session closed for user root 2024-04-25T11:18:59.619 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:59 smithi152 sudo[41459]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T11:18:59.619 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:59 smithi152 sudo[41459]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T11:18:59.619 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:59 smithi152 sudo[41459]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T11:18:59.619 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:59 smithi152 sudo[41459]: pam_unix(sudo:session): session closed for user root 2024-04-25T11:18:59.619 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:59 smithi152 sudo[41462]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T11:18:59.619 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:59 smithi152 sudo[41462]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T11:18:59.619 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:59 smithi152 sudo[41462]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T11:18:59.620 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:59 smithi152 sudo[41462]: pam_unix(sudo:session): session closed for user root 2024-04-25T11:18:59.620 INFO:journalctl@ceph.osd.0.smithi069.stdout:Apr 25 11:18:58 smithi069 sudo[48588]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T11:18:59.621 INFO:journalctl@ceph.osd.0.smithi069.stdout:Apr 25 11:18:58 smithi069 sudo[48588]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T11:18:59.621 INFO:journalctl@ceph.osd.0.smithi069.stdout:Apr 25 11:18:58 smithi069 sudo[48588]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T11:18:59.621 INFO:journalctl@ceph.osd.0.smithi069.stdout:Apr 25 11:18:58 smithi069 sudo[48588]: pam_unix(sudo:session): session closed for user root 2024-04-25T11:18:59.621 INFO:journalctl@ceph.osd.0.smithi069.stdout:Apr 25 11:18:58 smithi069 sudo[48591]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T11:18:59.621 INFO:journalctl@ceph.osd.0.smithi069.stdout:Apr 25 11:18:58 smithi069 sudo[48591]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T11:18:59.621 INFO:journalctl@ceph.osd.0.smithi069.stdout:Apr 25 11:18:58 smithi069 sudo[48591]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T11:18:59.621 INFO:journalctl@ceph.osd.0.smithi069.stdout:Apr 25 11:18:58 smithi069 sudo[48591]: pam_unix(sudo:session): session closed for user root 2024-04-25T11:18:59.621 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:58 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T11:18:59.622 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:58 smithi146 ceph-mon[32821]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T11:18:59.622 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:58 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T11:18:59.622 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:59 smithi069 sudo[48614]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T11:18:59.622 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:59 smithi069 sudo[48614]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T11:18:59.622 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:59 smithi069 sudo[48614]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T11:18:59.622 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:59 smithi069 sudo[48614]: pam_unix(sudo:session): session closed for user root 2024-04-25T11:18:59.623 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:59 smithi069 sudo[48617]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T11:18:59.623 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:59 smithi069 sudo[48617]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T11:18:59.623 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:59 smithi069 sudo[48617]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T11:18:59.623 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:59 smithi069 sudo[48617]: pam_unix(sudo:session): session closed for user root 2024-04-25T11:18:59.623 INFO:journalctl@ceph.osd.1.smithi146.stdout:Apr 25 11:18:59 smithi146 sudo[41094]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T11:18:59.623 INFO:journalctl@ceph.osd.1.smithi146.stdout:Apr 25 11:18:59 smithi146 sudo[41094]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T11:18:59.624 INFO:journalctl@ceph.osd.1.smithi146.stdout:Apr 25 11:18:59 smithi146 sudo[41094]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T11:18:59.624 INFO:journalctl@ceph.osd.1.smithi146.stdout:Apr 25 11:18:59 smithi146 sudo[41094]: pam_unix(sudo:session): session closed for user root 2024-04-25T11:18:59.624 INFO:journalctl@ceph.osd.1.smithi146.stdout:Apr 25 11:18:59 smithi146 sudo[41097]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T11:18:59.624 INFO:journalctl@ceph.osd.1.smithi146.stdout:Apr 25 11:18:59 smithi146 sudo[41097]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T11:18:59.624 INFO:journalctl@ceph.osd.1.smithi146.stdout:Apr 25 11:18:59 smithi146 sudo[41097]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T11:18:59.624 INFO:journalctl@ceph.osd.1.smithi146.stdout:Apr 25 11:18:59 smithi146 sudo[41097]: pam_unix(sudo:session): session closed for user root 2024-04-25T11:18:59.625 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:59 smithi146 sudo[41100]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T11:18:59.625 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:59 smithi146 sudo[41100]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T11:18:59.625 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:59 smithi146 sudo[41100]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T11:18:59.625 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:59 smithi146 sudo[41100]: pam_unix(sudo:session): session closed for user root 2024-04-25T11:18:59.625 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:59 smithi146 sudo[41103]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T11:18:59.625 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:59 smithi146 sudo[41103]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T11:18:59.625 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:59 smithi146 sudo[41103]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T11:18:59.625 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:59 smithi146 sudo[41103]: pam_unix(sudo:session): session closed for user root 2024-04-25T11:18:59.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:59.547+0000 7f3317f63700 1 -- 172.21.15.69:0/1383509686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3310104200 msgr2=0x7f3310104610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:59.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:59.547+0000 7f3317f63700 1 --2- 172.21.15.69:0/1383509686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3310104200 0x7f3310104610 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3300009990 tx=0x7f3300009ca0 comp rx=0 tx=0).stop 2024-04-25T11:18:59.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:59.547+0000 7f3317f63700 1 -- 172.21.15.69:0/1383509686 shutdown_connections 2024-04-25T11:18:59.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:59.547+0000 7f3317f63700 1 --2- 172.21.15.69:0/1383509686 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3310104ba0 0x7f331010b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:59.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:59.547+0000 7f3317f63700 1 --2- 172.21.15.69:0/1383509686 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33101030d0 0x7f33101034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:59.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:59.547+0000 7f3317f63700 1 --2- 172.21.15.69:0/1383509686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3310104200 0x7f3310104610 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:59.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:59.547+0000 7f3317f63700 1 -- 172.21.15.69:0/1383509686 >> 172.21.15.69:0/1383509686 conn(0x7f33100fe710 msgr2=0x7f3310100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:59.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:59.548+0000 7f3317f63700 1 -- 172.21.15.69:0/1383509686 shutdown_connections 2024-04-25T11:18:59.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:59.548+0000 7f3317f63700 1 -- 172.21.15.69:0/1383509686 wait complete. 2024-04-25T11:18:59.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:59.548+0000 7f3317f63700 1 Processor -- start 2024-04-25T11:18:59.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:59.548+0000 7f3317f63700 1 -- start start 2024-04-25T11:18:59.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:59.548+0000 7f3317f63700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f33101030d0 0x7f331019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:59.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:59.548+0000 7f3317f63700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3310104200 0x7f331019abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:59.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:59.548+0000 7f3317f63700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3310104ba0 0x7f33101a1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:59.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:59.548+0000 7f3317f63700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f331010ea00 con 0x7f33101030d0 2024-04-25T11:18:59.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:59.548+0000 7f3317f63700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f331010e880 con 0x7f3310104ba0 2024-04-25T11:18:59.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:59.548+0000 7f3317f63700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f331010eb80 con 0x7f3310104200 2024-04-25T11:18:59.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:59.549+0000 7f3315cff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f33101030d0 0x7f331019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:59.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:59.549+0000 7f3315cff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f33101030d0 0x7f331019a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59972/0 (socket says 172.21.15.69:59972) 2024-04-25T11:18:59.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:59.549+0000 7f3315cff700 1 -- 172.21.15.69:0/2731497502 learned_addr learned my addr 172.21.15.69:0/2731497502 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:18:59.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:59.549+0000 7f33154fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3310104200 0x7f331019abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:59.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:59.549+0000 7f3316500700 1 --2- 172.21.15.69:0/2731497502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3310104ba0 0x7f33101a1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:59.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:59.549+0000 7f3315cff700 1 -- 172.21.15.69:0/2731497502 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3310104200 msgr2=0x7f331019abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:59.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:59.549+0000 7f3315cff700 1 --2- 172.21.15.69:0/2731497502 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3310104200 0x7f331019abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:59.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:59.549+0000 7f3315cff700 1 -- 172.21.15.69:0/2731497502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3310104ba0 msgr2=0x7f33101a1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:59.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:59.549+0000 7f3315cff700 1 --2- 172.21.15.69:0/2731497502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3310104ba0 0x7f33101a1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:59.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:59.549+0000 7f3315cff700 1 -- 172.21.15.69:0/2731497502 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33101a2180 con 0x7f33101030d0 2024-04-25T11:18:59.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:59.549+0000 7f3315cff700 1 --2- 172.21.15.69:0/2731497502 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f33101030d0 0x7f331019a6b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f330c00c970 tx=0x7f330c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:59.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:59.549+0000 7f3306ffd700 1 -- 172.21.15.69:0/2731497502 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330c007910 con 0x7f33101030d0 2024-04-25T11:18:59.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:59.549+0000 7f3306ffd700 1 -- 172.21.15.69:0/2731497502 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f330c00f450 con 0x7f33101030d0 2024-04-25T11:18:59.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:59.549+0000 7f3317f63700 1 -- 172.21.15.69:0/2731497502 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33101a2470 con 0x7f33101030d0 2024-04-25T11:18:59.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:59.549+0000 7f3306ffd700 1 -- 172.21.15.69:0/2731497502 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330c018660 con 0x7f33101030d0 2024-04-25T11:18:59.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:59.549+0000 7f3317f63700 1 -- 172.21.15.69:0/2731497502 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33101a29b0 con 0x7f33101030d0 2024-04-25T11:18:59.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:59.551+0000 7f3317f63700 1 -- 172.21.15.69:0/2731497502 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3310051190 con 0x7f33101030d0 2024-04-25T11:18:59.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:59.552+0000 7f3306ffd700 1 -- 172.21.15.69:0/2731497502 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 15) ==== 147464+0+0 (secure 0 0 0) 0x7f330c00f600 con 0x7f33101030d0 2024-04-25T11:18:59.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:59.553+0000 7f3306ffd700 1 --2- 172.21.15.69:0/2731497502 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f32fc0afaf0 0x7f32fc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:59.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:59.553+0000 7f33154fe700 1 --2- 172.21.15.69:0/2731497502 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f32fc0afaf0 0x7f32fc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:59.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:59.553+0000 7f3306ffd700 1 -- 172.21.15.69:0/2731497502 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3237+0+0 (secure 0 0 0) 0x7f330c0dd710 con 0x7f33101030d0 2024-04-25T11:18:59.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:59.554+0000 7f33154fe700 1 --2- 172.21.15.69:0/2731497502 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f32fc0afaf0 0x7f32fc0b2000 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f330000b5c0 tx=0x7f330001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:59.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:59.557+0000 7f3306ffd700 1 -- 172.21.15.69:0/2731497502 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f330c052ff0 con 0x7f33101030d0 2024-04-25T11:18:59.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:18:59.763+0000 7f3317f63700 1 -- 172.21.15.69:0/2731497502 --> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f33101a2c70 con 0x7f32fc0afaf0 2024-04-25T11:19:00.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:19:00.163+0000 7f3306ffd700 1 -- 172.21.15.69:0/2731497502 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f330c0a33c0 con 0x7f33101030d0 2024-04-25T11:19:00.162 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:59 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T11:19:00.163 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:59 smithi069 ceph-mon[29188]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T11:19:00.163 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:59 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:00.163 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:59 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:00.163 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:59 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:00.163 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:59 smithi069 ceph-mon[29188]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T11:19:00.163 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:59 smithi069 ceph-mon[29188]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T11:19:00.163 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:59 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T11:19:00.163 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:59 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:19:00.163 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:59 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:19:00.163 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:59 smithi069 ceph-mon[29188]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T11:19:00.163 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:59 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T11:19:00.163 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:59 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:19:00.163 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:59 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:19:00.163 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:59 smithi069 ceph-mon[29188]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T11:19:00.164 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:59 smithi069 ceph-mon[29188]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T11:19:00.164 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:59 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T11:19:00.164 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:59 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:19:00.164 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:59 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:19:00.164 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:59 smithi069 ceph-mon[29188]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T11:19:00.164 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:59 smithi069 ceph-mon[29188]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:19:00.164 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:59 smithi069 ceph-mon[29188]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:19:00.164 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:59 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:19:00.164 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:59 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:19:00.164 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:18:59 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:19:00.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:59 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T11:19:00.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:59 smithi152 ceph-mon[32701]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T11:19:00.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:59 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:00.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:59 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:00.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:59 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:00.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:59 smithi152 ceph-mon[32701]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T11:19:00.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:59 smithi152 ceph-mon[32701]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T11:19:00.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:59 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T11:19:00.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:59 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:19:00.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:59 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:19:00.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:59 smithi152 ceph-mon[32701]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T11:19:00.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:59 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T11:19:00.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:59 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:19:00.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:59 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:19:00.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:59 smithi152 ceph-mon[32701]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T11:19:00.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:59 smithi152 ceph-mon[32701]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T11:19:00.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:59 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T11:19:00.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:59 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:19:00.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:59 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:19:00.237 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:59 smithi152 ceph-mon[32701]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T11:19:00.238 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:59 smithi152 ceph-mon[32701]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:19:00.238 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:59 smithi152 ceph-mon[32701]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:19:00.238 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:59 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:19:00.238 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:59 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:19:00.238 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:18:59 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:19:00.354 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:59 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T11:19:00.355 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:59 smithi146 ceph-mon[32821]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T11:19:00.355 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:59 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:00.355 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:59 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:00.355 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:59 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:00.355 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:59 smithi146 ceph-mon[32821]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T11:19:00.355 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:59 smithi146 ceph-mon[32821]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T11:19:00.355 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:59 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T11:19:00.355 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:59 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:19:00.355 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:59 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:19:00.355 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:59 smithi146 ceph-mon[32821]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T11:19:00.355 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:59 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T11:19:00.355 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:59 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:19:00.355 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:59 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:19:00.355 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:59 smithi146 ceph-mon[32821]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T11:19:00.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:59 smithi146 ceph-mon[32821]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T11:19:00.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:59 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T11:19:00.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:59 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:19:00.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:59 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:19:00.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:59 smithi146 ceph-mon[32821]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T11:19:00.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:59 smithi146 ceph-mon[32821]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:19:00.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:59 smithi146 ceph-mon[32821]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:19:00.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:59 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:19:00.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:59 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:19:00.356 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:18:59 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:19:01.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:00 smithi069 ceph-mon[29188]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T11:19:01.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:00 smithi069 ceph-mon[29188]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T11:19:01.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:00 smithi152 ceph-mon[32701]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T11:19:01.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:00 smithi152 ceph-mon[32701]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T11:19:01.354 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:00 smithi146 ceph-mon[32821]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T11:19:01.354 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:00 smithi146 ceph-mon[32821]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T11:19:01.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:01 smithi069 ceph-mon[29188]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:19:02.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:01 smithi152 ceph-mon[32701]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:19:02.354 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:01 smithi146 ceph-mon[32821]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:19:03.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:02 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/1028785510' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0ab85fb8-5bbb-4ffc-bddf-cae7644a64ab"}]: dispatch 2024-04-25T11:19:03.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:02 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/1028785510' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0ab85fb8-5bbb-4ffc-bddf-cae7644a64ab"}]': finished 2024-04-25T11:19:03.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:02 smithi069 ceph-mon[29188]: osdmap e22: 4 total, 3 up, 4 in 2024-04-25T11:19:03.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:02 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T11:19:03.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:02 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/1028785510' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0ab85fb8-5bbb-4ffc-bddf-cae7644a64ab"}]: dispatch 2024-04-25T11:19:03.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:02 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/1028785510' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0ab85fb8-5bbb-4ffc-bddf-cae7644a64ab"}]': finished 2024-04-25T11:19:03.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:02 smithi152 ceph-mon[32701]: osdmap e22: 4 total, 3 up, 4 in 2024-04-25T11:19:03.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:02 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T11:19:03.354 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:02 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/1028785510' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0ab85fb8-5bbb-4ffc-bddf-cae7644a64ab"}]: dispatch 2024-04-25T11:19:03.354 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:02 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/1028785510' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0ab85fb8-5bbb-4ffc-bddf-cae7644a64ab"}]': finished 2024-04-25T11:19:03.354 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:02 smithi146 ceph-mon[32821]: osdmap e22: 4 total, 3 up, 4 in 2024-04-25T11:19:03.354 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:02 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T11:19:04.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:03 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/1003975687' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:19:04.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:03 smithi069 ceph-mon[29188]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:19:04.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:03 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/1003975687' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:19:04.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:03 smithi152 ceph-mon[32701]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:19:04.354 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:03 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/1003975687' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:19:04.354 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:03 smithi146 ceph-mon[32821]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:19:06.948 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:06 smithi069 ceph-mon[29188]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:19:07.104 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:06 smithi146 ceph-mon[32821]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:19:07.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:06 smithi152 ceph-mon[32701]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:19:08.941 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:08 smithi069 ceph-mon[29188]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:19:09.104 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:08 smithi146 ceph-mon[32821]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:19:09.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:08 smithi152 ceph-mon[32701]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:19:10.920 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:10 smithi069 ceph-mon[29188]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:19:10.920 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:10 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T11:19:10.920 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:10 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:19:11.038 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:10 smithi152 ceph-mon[32701]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:19:11.038 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:10 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T11:19:11.038 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:10 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:19:11.104 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:10 smithi146 ceph-mon[32821]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:19:11.104 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:10 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T11:19:11.104 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:10 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:19:11.880 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:11 smithi069 ceph-mon[29188]: Deploying daemon osd.3 on smithi069 2024-04-25T11:19:12.104 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:11 smithi146 ceph-mon[32821]: Deploying daemon osd.3 on smithi069 2024-04-25T11:19:12.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:11 smithi152 ceph-mon[32701]: Deploying daemon osd.3 on smithi069 2024-04-25T11:19:13.088 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:12 smithi069 ceph-mon[29188]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:19:13.104 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:12 smithi146 ceph-mon[32821]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:19:13.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:12 smithi152 ceph-mon[32701]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:19:15.104 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:14 smithi146 ceph-mon[32821]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:19:15.181 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:14 smithi069 ceph-mon[29188]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:19:15.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:14 smithi152 ceph-mon[32701]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:19:16.948 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:16 smithi069 ceph-mon[29188]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:19:16.948 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:16 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:16.948 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:16 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:16.948 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:16 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:19:17.106 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:16 smithi146 ceph-mon[32821]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:19:17.106 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:16 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:17.107 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:16 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:17.107 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:16 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:19:17.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:16 smithi152 ceph-mon[32701]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:19:17.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:16 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:17.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:16 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:17.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:16 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:19:18.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:18 smithi069 ceph-mon[29188]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:19:19.104 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:18 smithi146 ceph-mon[32821]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:19:19.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:18 smithi152 ceph-mon[32701]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:19:19.722 INFO:teuthology.orchestra.run.smithi069.stdout:Created osd(s) 3 on host 'smithi069' 2024-04-25T11:19:19.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:19:19.723+0000 7f3306ffd700 1 -- 172.21.15.69:0/2731497502 <== mgr.14152 v2:172.21.15.69:6800/4220065409 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f33101a2c70 con 0x7f32fc0afaf0 2024-04-25T11:19:19.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:19:19.725+0000 7f3317f63700 1 -- 172.21.15.69:0/2731497502 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f32fc0afaf0 msgr2=0x7f32fc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:19.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:19:19.725+0000 7f3317f63700 1 --2- 172.21.15.69:0/2731497502 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f32fc0afaf0 0x7f32fc0b2000 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f330000b5c0 tx=0x7f330001a040 comp rx=0 tx=0).stop 2024-04-25T11:19:19.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:19:19.726+0000 7f3317f63700 1 -- 172.21.15.69:0/2731497502 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f33101030d0 msgr2=0x7f331019a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:19.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:19:19.726+0000 7f3317f63700 1 --2- 172.21.15.69:0/2731497502 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f33101030d0 0x7f331019a6b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f330c00c970 tx=0x7f330c00cc80 comp rx=0 tx=0).stop 2024-04-25T11:19:19.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:19:19.726+0000 7f3317f63700 1 -- 172.21.15.69:0/2731497502 shutdown_connections 2024-04-25T11:19:19.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:19:19.726+0000 7f3317f63700 1 --2- 172.21.15.69:0/2731497502 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f32fc0afaf0 0x7f32fc0b2000 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:19.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:19:19.726+0000 7f3317f63700 1 --2- 172.21.15.69:0/2731497502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3310104ba0 0x7f33101a1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:19.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:19:19.726+0000 7f3317f63700 1 --2- 172.21.15.69:0/2731497502 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f33101030d0 0x7f331019a6b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:19.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:19:19.726+0000 7f3317f63700 1 --2- 172.21.15.69:0/2731497502 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3310104200 0x7f331019abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:19.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:19:19.726+0000 7f3317f63700 1 -- 172.21.15.69:0/2731497502 >> 172.21.15.69:0/2731497502 conn(0x7f33100fe710 msgr2=0x7f331006bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:19.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:19:19.726+0000 7f3317f63700 1 -- 172.21.15.69:0/2731497502 shutdown_connections 2024-04-25T11:19:19.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:19:19.726+0000 7f3317f63700 1 -- 172.21.15.69:0/2731497502 wait complete. 2024-04-25T11:19:19.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:19 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:19.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:19 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:19.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:19 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:19:19.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:19 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:19:19.950 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:19 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:19.951 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:19 smithi069 ceph-mon[29188]: from='osd.3 [v2:172.21.15.69:6810/1809071296,v1:172.21.15.69:6811/1809071296]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T11:19:19.951 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:19 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:19.951 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:19 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:19.951 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:19 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:19:20.104 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:19 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:20.104 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:19 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:20.104 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:19 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:19:20.104 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:19 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:19:20.104 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:19 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:20.104 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:19 smithi146 ceph-mon[32821]: from='osd.3 [v2:172.21.15.69:6810/1809071296,v1:172.21.15.69:6811/1809071296]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T11:19:20.104 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:19 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:20.105 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:19 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:20.105 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:19 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:19:20.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:19 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:20.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:19 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:20.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:19 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:19:20.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:19 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:19:20.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:19 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:20.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:19 smithi152 ceph-mon[32701]: from='osd.3 [v2:172.21.15.69:6810/1809071296,v1:172.21.15.69:6811/1809071296]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T11:19:20.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:19 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:20.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:19 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:20.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:19 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:19:20.389 DEBUG:teuthology.orchestra.run.smithi069:osd.3> sudo journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@osd.3.service 2024-04-25T11:19:20.392 INFO:tasks.cephadm:Deploying osd.4 on smithi146 with /dev/vg_nvme/lv_3... 2024-04-25T11:19:20.392 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T11:19:20.638 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T11:19:20.948 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:20 smithi069 ceph-mon[29188]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:19:20.948 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:20 smithi069 ceph-mon[29188]: from='osd.3 [v2:172.21.15.69:6810/1809071296,v1:172.21.15.69:6811/1809071296]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T11:19:20.948 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:20 smithi069 ceph-mon[29188]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T11:19:20.948 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:20 smithi069 ceph-mon[29188]: from='osd.3 [v2:172.21.15.69:6810/1809071296,v1:172.21.15.69:6811/1809071296]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-25T11:19:20.948 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:20 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T11:19:20.949 INFO:journalctl@ceph.osd.3.smithi069.stdout:Apr 25 11:19:20 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-3[50905]: 2024-04-25T11:19:20.829+0000 7fe665b77700 -1 osd.3 0 waiting for initial osdmap 2024-04-25T11:19:20.949 INFO:journalctl@ceph.osd.3.smithi069.stdout:Apr 25 11:19:20 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-3[50905]: 2024-04-25T11:19:20.836+0000 7fe66192c700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:19:21.074 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:20 smithi146 ceph-mon[32821]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:19:21.074 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:20 smithi146 ceph-mon[32821]: from='osd.3 [v2:172.21.15.69:6810/1809071296,v1:172.21.15.69:6811/1809071296]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T11:19:21.074 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:20 smithi146 ceph-mon[32821]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T11:19:21.074 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:20 smithi146 ceph-mon[32821]: from='osd.3 [v2:172.21.15.69:6810/1809071296,v1:172.21.15.69:6811/1809071296]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-25T11:19:21.074 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:20 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T11:19:21.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:20 smithi152 ceph-mon[32701]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:19:21.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:20 smithi152 ceph-mon[32701]: from='osd.3 [v2:172.21.15.69:6810/1809071296,v1:172.21.15.69:6811/1809071296]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T11:19:21.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:20 smithi152 ceph-mon[32701]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T11:19:21.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:20 smithi152 ceph-mon[32701]: from='osd.3 [v2:172.21.15.69:6810/1809071296,v1:172.21.15.69:6811/1809071296]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-25T11:19:21.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:20 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T11:19:21.948 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:21 smithi069 ceph-mon[29188]: from='osd.3 [v2:172.21.15.69:6810/1809071296,v1:172.21.15.69:6811/1809071296]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-04-25T11:19:21.948 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:21 smithi069 ceph-mon[29188]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T11:19:21.948 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:21 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T11:19:21.948 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:21 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T11:19:22.023 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:21 smithi146 ceph-mon[32821]: from='osd.3 [v2:172.21.15.69:6810/1809071296,v1:172.21.15.69:6811/1809071296]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-04-25T11:19:22.024 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:21 smithi146 ceph-mon[32821]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T11:19:22.024 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:21 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T11:19:22.024 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:21 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T11:19:22.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:21 smithi152 ceph-mon[32701]: from='osd.3 [v2:172.21.15.69:6810/1809071296,v1:172.21.15.69:6811/1809071296]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-04-25T11:19:22.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:21 smithi152 ceph-mon[32701]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T11:19:22.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:21 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T11:19:22.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:21 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T11:19:22.590 INFO:teuthology.orchestra.run.smithi146.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T11:19:22.591 INFO:teuthology.orchestra.run.smithi146.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T11:19:22.591 INFO:teuthology.orchestra.run.smithi146.stderr: stderr: 10+0 records in 2024-04-25T11:19:22.591 INFO:teuthology.orchestra.run.smithi146.stderr:10+0 records out 2024-04-25T11:19:22.591 INFO:teuthology.orchestra.run.smithi146.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0115495 s, 908 MB/s 2024-04-25T11:19:22.591 INFO:teuthology.orchestra.run.smithi146.stderr:--> Zapping successful for: 2024-04-25T11:19:23.044 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:22 smithi069 ceph-mon[29188]: purged_snaps scrub starts 2024-04-25T11:19:23.044 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:22 smithi069 ceph-mon[29188]: purged_snaps scrub ok 2024-04-25T11:19:23.044 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:22 smithi069 ceph-mon[29188]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:19:23.044 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:22 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T11:19:23.044 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:22 smithi069 ceph-mon[29188]: osd.3 [v2:172.21.15.69:6810/1809071296,v1:172.21.15.69:6811/1809071296] boot 2024-04-25T11:19:23.044 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:22 smithi069 ceph-mon[29188]: osdmap e25: 4 total, 4 up, 4 in 2024-04-25T11:19:23.044 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:22 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T11:19:23.084 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:22 smithi146 ceph-mon[32821]: purged_snaps scrub starts 2024-04-25T11:19:23.084 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:22 smithi146 ceph-mon[32821]: purged_snaps scrub ok 2024-04-25T11:19:23.084 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:22 smithi146 ceph-mon[32821]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:19:23.084 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:22 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T11:19:23.084 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:22 smithi146 ceph-mon[32821]: osd.3 [v2:172.21.15.69:6810/1809071296,v1:172.21.15.69:6811/1809071296] boot 2024-04-25T11:19:23.084 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:22 smithi146 ceph-mon[32821]: osdmap e25: 4 total, 4 up, 4 in 2024-04-25T11:19:23.084 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:22 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T11:19:23.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:22 smithi152 ceph-mon[32701]: purged_snaps scrub starts 2024-04-25T11:19:23.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:22 smithi152 ceph-mon[32701]: purged_snaps scrub ok 2024-04-25T11:19:23.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:22 smithi152 ceph-mon[32701]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:19:23.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:22 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T11:19:23.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:22 smithi152 ceph-mon[32701]: osd.3 [v2:172.21.15.69:6810/1809071296,v1:172.21.15.69:6811/1809071296] boot 2024-04-25T11:19:23.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:22 smithi152 ceph-mon[32701]: osdmap e25: 4 total, 4 up, 4 in 2024-04-25T11:19:23.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:22 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T11:19:23.420 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi146:vg_nvme/lv_3 2024-04-25T11:19:23.663 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T11:19:24.104 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:23 smithi146 ceph-mon[32821]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T11:19:24.104 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:23 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:24.104 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:23 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:24.104 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:23 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:19:24.104 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:23 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:19:24.104 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:23 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:24.104 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:23 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:19:24.104 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:23 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:19:24.104 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:23 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:24.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:23 smithi069 ceph-mon[29188]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T11:19:24.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:23 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:24.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:23 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:24.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:23 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:19:24.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:23 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:19:24.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:23 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:24.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:23 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:19:24.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:23 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:19:24.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:23 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:24.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:23 smithi152 ceph-mon[32701]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T11:19:24.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:23 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:24.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:23 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:24.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:23 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:19:24.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:23 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:19:24.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:23 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:24.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:23 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:19:24.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:23 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:19:24.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:23 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:25.055 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:24 smithi146 ceph-mon[32821]: Detected new or changed devices on smithi069 2024-04-25T11:19:25.055 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:24 smithi146 ceph-mon[32821]: Adjusting osd_memory_target on smithi069 to 8544M 2024-04-25T11:19:25.056 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:24 smithi146 ceph-mon[32821]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-25T11:19:25.056 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:24 smithi146 ceph-mon[32821]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T11:19:25.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:24 smithi069 ceph-mon[29188]: Detected new or changed devices on smithi069 2024-04-25T11:19:25.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:24 smithi069 ceph-mon[29188]: Adjusting osd_memory_target on smithi069 to 8544M 2024-04-25T11:19:25.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:24 smithi069 ceph-mon[29188]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-25T11:19:25.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:24 smithi069 ceph-mon[29188]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T11:19:25.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:24 smithi152 ceph-mon[32701]: Detected new or changed devices on smithi069 2024-04-25T11:19:25.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:24 smithi152 ceph-mon[32701]: Adjusting osd_memory_target on smithi069 to 8544M 2024-04-25T11:19:25.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:24 smithi152 ceph-mon[32701]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-25T11:19:25.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:24 smithi152 ceph-mon[32701]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T11:19:25.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:25.293+0000 7fe176641700 1 -- 172.21.15.146:0/1318737116 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe170109fc0 msgr2=0x7fe17010c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:25.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:25.293+0000 7fe176641700 1 --2- 172.21.15.146:0/1318737116 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe170109fc0 0x7fe17010c3e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe16800b0a0 tx=0x7fe16800b3b0 comp rx=0 tx=0).stop 2024-04-25T11:19:25.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:25.293+0000 7fe176641700 1 -- 172.21.15.146:0/1318737116 shutdown_connections 2024-04-25T11:19:25.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:25.293+0000 7fe176641700 1 --2- 172.21.15.146:0/1318737116 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe170109fc0 0x7fe17010c3e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:25.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:25.293+0000 7fe176641700 1 --2- 172.21.15.146:0/1318737116 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe170102570 0x7fe1701029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:25.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:25.293+0000 7fe176641700 1 --2- 172.21.15.146:0/1318737116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe170107660 0x7fe170109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:25.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:25.293+0000 7fe176641700 1 -- 172.21.15.146:0/1318737116 >> 172.21.15.146:0/1318737116 conn(0x7fe1700fded0 msgr2=0x7fe170100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:25.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:25.293+0000 7fe176641700 1 -- 172.21.15.146:0/1318737116 shutdown_connections 2024-04-25T11:19:25.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:25.293+0000 7fe176641700 1 -- 172.21.15.146:0/1318737116 wait complete. 2024-04-25T11:19:25.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:25.294+0000 7fe176641700 1 Processor -- start 2024-04-25T11:19:25.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:25.294+0000 7fe176641700 1 -- start start 2024-04-25T11:19:25.294 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:25.294+0000 7fe176641700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe170102570 0x7fe17019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:25.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:25.294+0000 7fe176641700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe170107660 0x7fe17019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:25.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:25.294+0000 7fe176641700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe170109fc0 0x7fe1701a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:25.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:25.294+0000 7fe176641700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe17010fb00 con 0x7fe170102570 2024-04-25T11:19:25.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:25.294+0000 7fe176641700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe17010f980 con 0x7fe170109fc0 2024-04-25T11:19:25.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:25.294+0000 7fe176641700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe17010fc80 con 0x7fe170107660 2024-04-25T11:19:25.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:25.294+0000 7fe174bde700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe170109fc0 0x7fe1701a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:25.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:25.294+0000 7fe174bde700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe170109fc0 0x7fe1701a1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:42116/0 (socket says 172.21.15.146:42116) 2024-04-25T11:19:25.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:25.294+0000 7fe174bde700 1 -- 172.21.15.146:0/4015097971 learned_addr learned my addr 172.21.15.146:0/4015097971 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-04-25T11:19:25.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:25.294+0000 7fe16f7fe700 1 --2- 172.21.15.146:0/4015097971 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe170107660 0x7fe17019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:25.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:25.295+0000 7fe16ffff700 1 --2- 172.21.15.146:0/4015097971 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe170102570 0x7fe17019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:25.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:25.295+0000 7fe174bde700 1 -- 172.21.15.146:0/4015097971 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe170107660 msgr2=0x7fe17019ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:25.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:25.295+0000 7fe174bde700 1 --2- 172.21.15.146:0/4015097971 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe170107660 0x7fe17019ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:25.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:25.295+0000 7fe174bde700 1 -- 172.21.15.146:0/4015097971 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe170102570 msgr2=0x7fe17019a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:25.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:25.295+0000 7fe174bde700 1 --2- 172.21.15.146:0/4015097971 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe170102570 0x7fe17019a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:25.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:25.295+0000 7fe174bde700 1 -- 172.21.15.146:0/4015097971 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1701a21a0 con 0x7fe170109fc0 2024-04-25T11:19:25.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:25.295+0000 7fe174bde700 1 --2- 172.21.15.146:0/4015097971 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe170109fc0 0x7fe1701a1b20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe16800b070 tx=0x7fe168009300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:25.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:25.295+0000 7fe16d7fa700 1 -- 172.21.15.146:0/4015097971 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe168021070 con 0x7fe170109fc0 2024-04-25T11:19:25.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:25.295+0000 7fe16d7fa700 1 -- 172.21.15.146:0/4015097971 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe168004e10 con 0x7fe170109fc0 2024-04-25T11:19:25.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:25.295+0000 7fe176641700 1 -- 172.21.15.146:0/4015097971 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1701a2430 con 0x7fe170109fc0 2024-04-25T11:19:25.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:25.295+0000 7fe16d7fa700 1 -- 172.21.15.146:0/4015097971 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1680044a0 con 0x7fe170109fc0 2024-04-25T11:19:25.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:25.295+0000 7fe176641700 1 -- 172.21.15.146:0/4015097971 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1701a2880 con 0x7fe170109fc0 2024-04-25T11:19:25.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:25.296+0000 7fe176641700 1 -- 172.21.15.146:0/4015097971 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe170051190 con 0x7fe170109fc0 2024-04-25T11:19:25.297 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:25.297+0000 7fe16d7fa700 1 -- 172.21.15.146:0/4015097971 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fe168004640 con 0x7fe170109fc0 2024-04-25T11:19:25.297 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:25.298+0000 7fe16d7fa700 1 --2- 172.21.15.146:0/4015097971 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fe1580afb80 0x7fe1580b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:25.297 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:25.298+0000 7fe16d7fa700 1 -- 172.21.15.146:0/4015097971 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3669+0+0 (secure 0 0 0) 0x7fe16800f020 con 0x7fe170109fc0 2024-04-25T11:19:25.297 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:25.298+0000 7fe16ffff700 1 --2- 172.21.15.146:0/4015097971 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fe1580afb80 0x7fe1580b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:25.297 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:25.299+0000 7fe16ffff700 1 --2- 172.21.15.146:0/4015097971 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fe1580afb80 0x7fe1580b2090 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe160005950 tx=0x7fe160005eb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:25.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:25.302+0000 7fe16d7fa700 1 -- 172.21.15.146:0/4015097971 <== mon.2 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe16802a020 con 0x7fe170109fc0 2024-04-25T11:19:25.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:25.507+0000 7fe176641700 1 -- 172.21.15.146:0/4015097971 --> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fe170063b90 con 0x7fe1580afb80 2024-04-25T11:19:26.119 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:25 smithi146 ceph-mon[32821]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:19:26.119 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:25 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:19:26.119 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:25 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:19:26.119 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:25 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:19:26.119 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:25 smithi146 ceph-mon[32821]: pgmap v104: 1 pgs: 1 activating+degraded; 577 KiB data, 112 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T11:19:26.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:25 smithi069 ceph-mon[29188]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:19:26.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:25 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:19:26.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:25 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:19:26.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:25 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:19:26.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:25 smithi069 ceph-mon[29188]: pgmap v104: 1 pgs: 1 activating+degraded; 577 KiB data, 112 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T11:19:26.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:25 smithi152 ceph-mon[32701]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:19:26.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:25 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:19:26.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:25 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:19:26.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:25 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:19:26.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:25 smithi152 ceph-mon[32701]: pgmap v104: 1 pgs: 1 activating+degraded; 577 KiB data, 112 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T11:19:27.016 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:26 smithi146 ceph-mon[32821]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-25T11:19:27.197 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:26 smithi069 ceph-mon[29188]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-25T11:19:27.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:26 smithi152 ceph-mon[32701]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-25T11:19:28.197 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:27 smithi069 ceph-mon[29188]: pgmap v105: 1 pgs: 1 activating+degraded; 577 KiB data, 112 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T11:19:28.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:27 smithi152 ceph-mon[32701]: pgmap v105: 1 pgs: 1 activating+degraded; 577 KiB data, 112 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T11:19:28.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:27 smithi146 ceph-mon[32821]: pgmap v105: 1 pgs: 1 activating+degraded; 577 KiB data, 112 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T11:19:29.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:28 smithi069 ceph-mon[29188]: from='client.? 172.21.15.146:0/1013338213' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "273e0592-bbbc-4188-8eeb-46a3879cf632"}]: dispatch 2024-04-25T11:19:29.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:28 smithi069 ceph-mon[29188]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "273e0592-bbbc-4188-8eeb-46a3879cf632"}]: dispatch 2024-04-25T11:19:29.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:28 smithi069 ceph-mon[29188]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "273e0592-bbbc-4188-8eeb-46a3879cf632"}]': finished 2024-04-25T11:19:29.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:28 smithi069 ceph-mon[29188]: osdmap e28: 5 total, 4 up, 5 in 2024-04-25T11:19:29.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:28 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T11:19:29.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:28 smithi152 ceph-mon[32701]: from='client.? 172.21.15.146:0/1013338213' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "273e0592-bbbc-4188-8eeb-46a3879cf632"}]: dispatch 2024-04-25T11:19:29.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:28 smithi152 ceph-mon[32701]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "273e0592-bbbc-4188-8eeb-46a3879cf632"}]: dispatch 2024-04-25T11:19:29.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:28 smithi152 ceph-mon[32701]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "273e0592-bbbc-4188-8eeb-46a3879cf632"}]': finished 2024-04-25T11:19:29.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:28 smithi152 ceph-mon[32701]: osdmap e28: 5 total, 4 up, 5 in 2024-04-25T11:19:29.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:28 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T11:19:29.354 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:28 smithi146 ceph-mon[32821]: from='client.? 172.21.15.146:0/1013338213' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "273e0592-bbbc-4188-8eeb-46a3879cf632"}]: dispatch 2024-04-25T11:19:29.354 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:28 smithi146 ceph-mon[32821]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "273e0592-bbbc-4188-8eeb-46a3879cf632"}]: dispatch 2024-04-25T11:19:29.354 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:28 smithi146 ceph-mon[32821]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "273e0592-bbbc-4188-8eeb-46a3879cf632"}]': finished 2024-04-25T11:19:29.354 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:28 smithi146 ceph-mon[32821]: osdmap e28: 5 total, 4 up, 5 in 2024-04-25T11:19:29.354 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:28 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T11:19:30.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:29 smithi069 ceph-mon[29188]: from='client.? 172.21.15.146:0/96651326' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:19:30.327 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:29 smithi069 ceph-mon[29188]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-04-25T11:19:30.328 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:29 smithi152 ceph-mon[32701]: from='client.? 172.21.15.146:0/96651326' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:19:30.328 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:29 smithi152 ceph-mon[32701]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-04-25T11:19:30.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:29 smithi146 ceph-mon[32821]: from='client.? 172.21.15.146:0/96651326' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:19:30.354 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:29 smithi146 ceph-mon[32821]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-04-25T11:19:31.197 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:30 smithi069 ceph-mon[29188]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-25T11:19:31.197 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:30 smithi069 ceph-mon[29188]: Cluster is now healthy 2024-04-25T11:19:31.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:30 smithi152 ceph-mon[32701]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-25T11:19:31.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:30 smithi152 ceph-mon[32701]: Cluster is now healthy 2024-04-25T11:19:31.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:30 smithi146 ceph-mon[32821]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-25T11:19:31.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:30 smithi146 ceph-mon[32821]: Cluster is now healthy 2024-04-25T11:19:32.197 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:31 smithi069 ceph-mon[29188]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T11:19:32.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:31 smithi152 ceph-mon[32701]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T11:19:32.257 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:31 smithi146 ceph-mon[32821]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T11:19:35.103 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:34 smithi146 ceph-mon[32821]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T11:19:35.170 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:34 smithi069 ceph-mon[29188]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T11:19:35.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:34 smithi152 ceph-mon[32701]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T11:19:35.853 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:35 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T11:19:35.853 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:35 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:19:36.111 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:35 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T11:19:36.111 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:35 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:19:36.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:35 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T11:19:36.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:35 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:19:37.103 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:36 smithi146 ceph-mon[32821]: Deploying daemon osd.4 on smithi146 2024-04-25T11:19:37.104 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:36 smithi146 ceph-mon[32821]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T11:19:37.197 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:36 smithi069 ceph-mon[29188]: Deploying daemon osd.4 on smithi146 2024-04-25T11:19:37.197 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:36 smithi069 ceph-mon[29188]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T11:19:37.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:36 smithi152 ceph-mon[32701]: Deploying daemon osd.4 on smithi146 2024-04-25T11:19:37.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:36 smithi152 ceph-mon[32701]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T11:19:38.986 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:38 smithi146 ceph-mon[32821]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T11:19:39.197 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:38 smithi069 ceph-mon[29188]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T11:19:39.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:38 smithi152 ceph-mon[32701]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T11:19:40.891 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:40 smithi146 ceph-mon[32821]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-04-25T11:19:41.197 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:40 smithi069 ceph-mon[29188]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-04-25T11:19:41.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:40 smithi152 ceph-mon[32701]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-04-25T11:19:42.311 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:42 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:42.311 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:42 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:42.311 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:42 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:19:42.311 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:42 smithi152 ceph-mon[32701]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-25T11:19:42.592 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:42 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:42.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:42 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:42.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:42 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:19:42.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:42 smithi146 ceph-mon[32821]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-25T11:19:42.697 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:42 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:42.698 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:42 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:42.698 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:42 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:19:42.698 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:42 smithi069 ceph-mon[29188]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-25T11:19:45.003 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:44 smithi146 ceph-mon[32821]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-25T11:19:45.003 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:44 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:45.003 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:44 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:45.003 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:44 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:19:45.003 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:44 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:19:45.003 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:44 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:45.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:45.072+0000 7fe16d7fa700 1 -- 172.21.15.146:0/4015097971 <== mgr.14152 v2:172.21.15.69:6800/4220065409 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe170063b90 con 0x7fe1580afb80 2024-04-25T11:19:45.071 INFO:teuthology.orchestra.run.smithi146.stdout:Created osd(s) 4 on host 'smithi146' 2024-04-25T11:19:45.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:45.074+0000 7fe176641700 1 -- 172.21.15.146:0/4015097971 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fe1580afb80 msgr2=0x7fe1580b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:45.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:45.074+0000 7fe176641700 1 --2- 172.21.15.146:0/4015097971 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fe1580afb80 0x7fe1580b2090 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe160005950 tx=0x7fe160005eb0 comp rx=0 tx=0).stop 2024-04-25T11:19:45.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:45.074+0000 7fe176641700 1 -- 172.21.15.146:0/4015097971 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe170109fc0 msgr2=0x7fe1701a1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:45.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:45.074+0000 7fe176641700 1 --2- 172.21.15.146:0/4015097971 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe170109fc0 0x7fe1701a1b20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe16800b070 tx=0x7fe168009300 comp rx=0 tx=0).stop 2024-04-25T11:19:45.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:45.074+0000 7fe176641700 1 -- 172.21.15.146:0/4015097971 shutdown_connections 2024-04-25T11:19:45.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:45.074+0000 7fe176641700 1 --2- 172.21.15.146:0/4015097971 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fe1580afb80 0x7fe1580b2090 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:45.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:45.074+0000 7fe176641700 1 --2- 172.21.15.146:0/4015097971 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe170109fc0 0x7fe1701a1b20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:45.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:45.074+0000 7fe176641700 1 --2- 172.21.15.146:0/4015097971 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe170102570 0x7fe17019a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:45.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:45.074+0000 7fe176641700 1 --2- 172.21.15.146:0/4015097971 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe170107660 0x7fe17019ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:45.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:45.074+0000 7fe176641700 1 -- 172.21.15.146:0/4015097971 >> 172.21.15.146:0/4015097971 conn(0x7fe1700fded0 msgr2=0x7fe1701085a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:45.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:45.074+0000 7fe176641700 1 -- 172.21.15.146:0/4015097971 shutdown_connections 2024-04-25T11:19:45.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:19:45.075+0000 7fe176641700 1 -- 172.21.15.146:0/4015097971 wait complete. 2024-04-25T11:19:45.197 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:44 smithi069 ceph-mon[29188]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-25T11:19:45.197 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:44 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:45.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:44 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:45.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:44 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:19:45.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:44 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:19:45.198 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:44 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:45.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:44 smithi152 ceph-mon[32701]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-25T11:19:45.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:44 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:45.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:44 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:45.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:44 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:19:45.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:44 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:19:45.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:44 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:45.692 DEBUG:teuthology.orchestra.run.smithi146:osd.4> sudo journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@osd.4.service 2024-04-25T11:19:45.694 INFO:tasks.cephadm:Deploying osd.5 on smithi152 with /dev/vg_nvme/lv_3... 2024-04-25T11:19:45.694 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T11:19:45.941 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T11:19:46.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:46 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:46.354 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:46 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:19:46.354 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:46 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:46.354 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:46 smithi146 ceph-mon[32821]: from='osd.4 [v2:172.21.15.146:6808/1898520972,v1:172.21.15.146:6809/1898520972]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T11:19:46.354 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:46 smithi146 ceph-mon[32821]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T11:19:46.354 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:46 smithi146 ceph-mon[32821]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-25T11:19:46.389 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:46 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:46.389 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:46 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:19:46.389 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:46 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:46.389 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:46 smithi152 ceph-mon[32701]: from='osd.4 [v2:172.21.15.146:6808/1898520972,v1:172.21.15.146:6809/1898520972]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T11:19:46.389 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:46 smithi152 ceph-mon[32701]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T11:19:46.389 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:46 smithi152 ceph-mon[32701]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-25T11:19:46.447 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:46 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:46.447 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:46 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:19:46.447 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:46 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:46.447 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:46 smithi069 ceph-mon[29188]: from='osd.4 [v2:172.21.15.146:6808/1898520972,v1:172.21.15.146:6809/1898520972]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T11:19:46.447 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:46 smithi069 ceph-mon[29188]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T11:19:46.447 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:46 smithi069 ceph-mon[29188]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-25T11:19:47.074 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:47 smithi146 ceph-mon[32821]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T11:19:47.074 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:47 smithi146 ceph-mon[32821]: osdmap e29: 5 total, 4 up, 5 in 2024-04-25T11:19:47.074 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:47 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T11:19:47.074 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:47 smithi146 ceph-mon[32821]: from='osd.4 [v2:172.21.15.146:6808/1898520972,v1:172.21.15.146:6809/1898520972]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-04-25T11:19:47.074 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:47 smithi146 ceph-mon[32821]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-04-25T11:19:47.074 INFO:journalctl@ceph.osd.4.smithi146.stdout:Apr 25 11:19:47 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-4[43915]: 2024-04-25T11:19:47.063+0000 7f2eab9c0700 -1 osd.4 0 waiting for initial osdmap 2024-04-25T11:19:47.074 INFO:journalctl@ceph.osd.4.smithi146.stdout:Apr 25 11:19:47 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-4[43915]: 2024-04-25T11:19:47.075+0000 7f2ea7775700 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:19:47.405 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:47 smithi152 ceph-mon[32701]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T11:19:47.406 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:47 smithi152 ceph-mon[32701]: osdmap e29: 5 total, 4 up, 5 in 2024-04-25T11:19:47.406 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:47 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T11:19:47.406 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:47 smithi152 ceph-mon[32701]: from='osd.4 [v2:172.21.15.146:6808/1898520972,v1:172.21.15.146:6809/1898520972]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-04-25T11:19:47.406 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:47 smithi152 ceph-mon[32701]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-04-25T11:19:47.447 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:47 smithi069 ceph-mon[29188]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T11:19:47.447 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:47 smithi069 ceph-mon[29188]: osdmap e29: 5 total, 4 up, 5 in 2024-04-25T11:19:47.448 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:47 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T11:19:47.448 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:47 smithi069 ceph-mon[29188]: from='osd.4 [v2:172.21.15.146:6808/1898520972,v1:172.21.15.146:6809/1898520972]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-04-25T11:19:47.448 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:47 smithi069 ceph-mon[29188]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-04-25T11:19:47.858 INFO:teuthology.orchestra.run.smithi152.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T11:19:47.858 INFO:teuthology.orchestra.run.smithi152.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T11:19:47.858 INFO:teuthology.orchestra.run.smithi152.stderr: stderr: 10+0 records in 2024-04-25T11:19:47.858 INFO:teuthology.orchestra.run.smithi152.stderr:10+0 records out 2024-04-25T11:19:47.858 INFO:teuthology.orchestra.run.smithi152.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0119177 s, 880 MB/s 2024-04-25T11:19:47.858 INFO:teuthology.orchestra.run.smithi152.stderr:--> Zapping successful for: 2024-04-25T11:19:48.165 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:48 smithi152 ceph-mon[32701]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-04-25T11:19:48.166 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:48 smithi152 ceph-mon[32701]: osdmap e30: 5 total, 4 up, 5 in 2024-04-25T11:19:48.166 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:48 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T11:19:48.166 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:48 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T11:19:48.166 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:48 smithi152 ceph-mon[32701]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-25T11:19:48.246 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:48 smithi146 ceph-mon[32821]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-04-25T11:19:48.246 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:48 smithi146 ceph-mon[32821]: osdmap e30: 5 total, 4 up, 5 in 2024-04-25T11:19:48.246 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:48 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T11:19:48.246 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:48 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T11:19:48.246 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:48 smithi146 ceph-mon[32821]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-25T11:19:48.447 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:48 smithi069 ceph-mon[29188]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-04-25T11:19:48.447 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:48 smithi069 ceph-mon[29188]: osdmap e30: 5 total, 4 up, 5 in 2024-04-25T11:19:48.447 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:48 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T11:19:48.447 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:48 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T11:19:48.448 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:48 smithi069 ceph-mon[29188]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-25T11:19:48.587 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi152:vg_nvme/lv_3 2024-04-25T11:19:48.824 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T11:19:49.203 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:49 smithi152 ceph-mon[32701]: purged_snaps scrub starts 2024-04-25T11:19:49.203 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:49 smithi152 ceph-mon[32701]: purged_snaps scrub ok 2024-04-25T11:19:49.204 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:49 smithi152 ceph-mon[32701]: osd.4 [v2:172.21.15.146:6808/1898520972,v1:172.21.15.146:6809/1898520972] boot 2024-04-25T11:19:49.204 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:49 smithi152 ceph-mon[32701]: osdmap e31: 5 total, 5 up, 5 in 2024-04-25T11:19:49.204 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:49 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T11:19:49.204 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:49 smithi152 ceph-mon[32701]: Detected new or changed devices on smithi146 2024-04-25T11:19:49.204 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:49 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:49.204 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:49 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:49.204 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:49 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:19:49.204 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:49 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:19:49.204 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:49 smithi152 ceph-mon[32701]: Adjusting osd_memory_target on smithi146 to 8544M 2024-04-25T11:19:49.204 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:49 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:49.205 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:49 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:19:49.205 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:49 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:19:49.205 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:49 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:49.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:49 smithi146 ceph-mon[32821]: purged_snaps scrub starts 2024-04-25T11:19:49.354 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:49 smithi146 ceph-mon[32821]: purged_snaps scrub ok 2024-04-25T11:19:49.354 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:49 smithi146 ceph-mon[32821]: osd.4 [v2:172.21.15.146:6808/1898520972,v1:172.21.15.146:6809/1898520972] boot 2024-04-25T11:19:49.354 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:49 smithi146 ceph-mon[32821]: osdmap e31: 5 total, 5 up, 5 in 2024-04-25T11:19:49.354 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:49 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T11:19:49.354 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:49 smithi146 ceph-mon[32821]: Detected new or changed devices on smithi146 2024-04-25T11:19:49.354 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:49 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:49.354 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:49 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:49.355 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:49 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:19:49.355 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:49 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:19:49.355 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:49 smithi146 ceph-mon[32821]: Adjusting osd_memory_target on smithi146 to 8544M 2024-04-25T11:19:49.355 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:49 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:49.355 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:49 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:19:49.355 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:49 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:19:49.355 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:49 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:49.447 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:49 smithi069 ceph-mon[29188]: purged_snaps scrub starts 2024-04-25T11:19:49.447 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:49 smithi069 ceph-mon[29188]: purged_snaps scrub ok 2024-04-25T11:19:49.448 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:49 smithi069 ceph-mon[29188]: osd.4 [v2:172.21.15.146:6808/1898520972,v1:172.21.15.146:6809/1898520972] boot 2024-04-25T11:19:49.448 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:49 smithi069 ceph-mon[29188]: osdmap e31: 5 total, 5 up, 5 in 2024-04-25T11:19:49.448 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:49 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T11:19:49.448 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:49 smithi069 ceph-mon[29188]: Detected new or changed devices on smithi146 2024-04-25T11:19:49.448 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:49 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:49.448 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:49 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:49.448 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:49 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:19:49.448 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:49 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:19:49.448 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:49 smithi069 ceph-mon[29188]: Adjusting osd_memory_target on smithi146 to 8544M 2024-04-25T11:19:49.448 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:49 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:49.448 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:49 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:19:49.448 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:49 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:19:49.448 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:49 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:19:50.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:50 smithi152 ceph-mon[32701]: osdmap e32: 5 total, 5 up, 5 in 2024-04-25T11:19:50.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:50 smithi152 ceph-mon[32701]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:19:50.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:50 smithi146 ceph-mon[32821]: osdmap e32: 5 total, 5 up, 5 in 2024-04-25T11:19:50.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:50 smithi146 ceph-mon[32821]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:19:50.447 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:50 smithi069 ceph-mon[29188]: osdmap e32: 5 total, 5 up, 5 in 2024-04-25T11:19:50.447 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:50 smithi069 ceph-mon[29188]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:19:50.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:19:50.493+0000 7f9a586a6700 1 -- 172.21.15.152:0/2631096201 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9a50104210 msgr2=0x7f9a50104620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:50.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:19:50.493+0000 7f9a586a6700 1 --2- 172.21.15.152:0/2631096201 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9a50104210 0x7f9a50104620 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c009990 tx=0x7f9a4c009ca0 comp rx=0 tx=0).stop 2024-04-25T11:19:50.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:19:50.493+0000 7f9a586a6700 1 -- 172.21.15.152:0/2631096201 shutdown_connections 2024-04-25T11:19:50.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:19:50.493+0000 7f9a586a6700 1 --2- 172.21.15.152:0/2631096201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a50105ce0 0x7f9a5010c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:50.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:19:50.493+0000 7f9a586a6700 1 --2- 172.21.15.152:0/2631096201 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9a50104210 0x7f9a50104620 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:50.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:19:50.493+0000 7f9a586a6700 1 --2- 172.21.15.152:0/2631096201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a50105340 0x7f9a50105750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:50.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:19:50.493+0000 7f9a586a6700 1 -- 172.21.15.152:0/2631096201 >> 172.21.15.152:0/2631096201 conn(0x7f9a500ff870 msgr2=0x7f9a50101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:50.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:19:50.493+0000 7f9a586a6700 1 -- 172.21.15.152:0/2631096201 shutdown_connections 2024-04-25T11:19:50.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:19:50.493+0000 7f9a586a6700 1 -- 172.21.15.152:0/2631096201 wait complete. 2024-04-25T11:19:50.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:19:50.494+0000 7f9a586a6700 1 Processor -- start 2024-04-25T11:19:50.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:19:50.494+0000 7f9a586a6700 1 -- start start 2024-04-25T11:19:50.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:19:50.494+0000 7f9a586a6700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9a50104210 0x7f9a5019a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:50.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:19:50.494+0000 7f9a586a6700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a50105340 0x7f9a5019ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:50.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:19:50.494+0000 7f9a586a6700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a50105ce0 0x7f9a501a1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:50.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:19:50.494+0000 7f9a586a6700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5010fbd0 con 0x7f9a50105ce0 2024-04-25T11:19:50.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:19:50.494+0000 7f9a586a6700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5010fa50 con 0x7f9a50105340 2024-04-25T11:19:50.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:19:50.494+0000 7f9a586a6700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5010fd50 con 0x7f9a50104210 2024-04-25T11:19:50.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:19:50.494+0000 7f9a56442700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9a50104210 0x7f9a5019a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:50.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:19:50.494+0000 7f9a56442700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9a50104210 0x7f9a5019a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38980/0 (socket says 172.21.15.152:38980) 2024-04-25T11:19:50.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:19:50.494+0000 7f9a56442700 1 -- 172.21.15.152:0/2896210425 learned_addr learned my addr 172.21.15.152:0/2896210425 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-25T11:19:50.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:19:50.494+0000 7f9a56c43700 1 --2- 172.21.15.152:0/2896210425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a50105ce0 0x7f9a501a1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:50.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:19:50.494+0000 7f9a55c41700 1 --2- 172.21.15.152:0/2896210425 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a50105340 0x7f9a5019ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:50.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:19:50.494+0000 7f9a56442700 1 -- 172.21.15.152:0/2896210425 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a50105340 msgr2=0x7f9a5019ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:50.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:19:50.494+0000 7f9a56442700 1 --2- 172.21.15.152:0/2896210425 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a50105340 0x7f9a5019ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:50.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:19:50.494+0000 7f9a56442700 1 -- 172.21.15.152:0/2896210425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a50105ce0 msgr2=0x7f9a501a1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:50.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:19:50.494+0000 7f9a56442700 1 --2- 172.21.15.152:0/2896210425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a50105ce0 0x7f9a501a1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:50.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:19:50.494+0000 7f9a56442700 1 -- 172.21.15.152:0/2896210425 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a501a22e0 con 0x7f9a50104210 2024-04-25T11:19:50.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:19:50.494+0000 7f9a55c41700 1 --2- 172.21.15.152:0/2896210425 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a50105340 0x7f9a5019ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:50.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:19:50.495+0000 7f9a56442700 1 --2- 172.21.15.152:0/2896210425 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9a50104210 0x7f9a5019a7d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c003820 tx=0x7f9a4c0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:50.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:19:50.495+0000 7f9a477fe700 1 -- 172.21.15.152:0/2896210425 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c026070 con 0x7f9a50104210 2024-04-25T11:19:50.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:19:50.495+0000 7f9a586a6700 1 -- 172.21.15.152:0/2896210425 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a500769d0 con 0x7f9a50104210 2024-04-25T11:19:50.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:19:50.495+0000 7f9a477fe700 1 -- 172.21.15.152:0/2896210425 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9a4c004020 con 0x7f9a50104210 2024-04-25T11:19:50.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:19:50.495+0000 7f9a477fe700 1 -- 172.21.15.152:0/2896210425 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c00fc60 con 0x7f9a50104210 2024-04-25T11:19:50.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:19:50.495+0000 7f9a586a6700 1 -- 172.21.15.152:0/2896210425 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a50076f30 con 0x7f9a50104210 2024-04-25T11:19:50.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:19:50.496+0000 7f9a586a6700 1 -- 172.21.15.152:0/2896210425 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a50051190 con 0x7f9a50104210 2024-04-25T11:19:50.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:19:50.497+0000 7f9a477fe700 1 -- 172.21.15.152:0/2896210425 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f9a4c029070 con 0x7f9a50104210 2024-04-25T11:19:50.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:19:50.498+0000 7f9a477fe700 1 --2- 172.21.15.152:0/2896210425 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f9a3c0afb80 0x7f9a3c0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:50.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:19:50.498+0000 7f9a477fe700 1 -- 172.21.15.152:0/2896210425 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4101+0+0 (secure 0 0 0) 0x7f9a4c0dedf0 con 0x7f9a50104210 2024-04-25T11:19:50.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:19:50.498+0000 7f9a55c41700 1 --2- 172.21.15.152:0/2896210425 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f9a3c0afb80 0x7f9a3c0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:50.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:19:50.499+0000 7f9a55c41700 1 --2- 172.21.15.152:0/2896210425 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f9a3c0afb80 0x7f9a3c0b2090 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9a5019b830 tx=0x7f9a40008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:50.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:19:50.502+0000 7f9a477fe700 1 -- 172.21.15.152:0/2896210425 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9a4c013070 con 0x7f9a50104210 2024-04-25T11:19:50.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:19:50.706+0000 7f9a586a6700 1 -- 172.21.15.152:0/2896210425 --> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f9a50063ad0 con 0x7f9a3c0afb80 2024-04-25T11:19:51.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:51 smithi146 ceph-mon[32821]: from='client.24208 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:19:51.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:51 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:19:51.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:51 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:19:51.354 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:51 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:19:51.447 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:51 smithi069 ceph-mon[29188]: from='client.24208 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:19:51.447 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:51 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:19:51.447 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:51 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:19:51.447 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:51 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:19:51.448 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:51 smithi152 ceph-mon[32701]: from='client.24208 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:19:51.449 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:51 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:19:51.449 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:51 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:19:51.449 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:51 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:19:52.343 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:52 smithi152 ceph-mon[32701]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:19:52.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:52 smithi146 ceph-mon[32821]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:19:52.447 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:52 smithi069 ceph-mon[29188]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:19:53.853 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:53 smithi146 ceph-mon[32821]: from='client.? 172.21.15.152:0/3932106848' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bbeae945-9fa2-48f9-a1d6-8bf18dcaef19"}]: dispatch 2024-04-25T11:19:53.853 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:53 smithi146 ceph-mon[32821]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bbeae945-9fa2-48f9-a1d6-8bf18dcaef19"}]: dispatch 2024-04-25T11:19:53.854 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:53 smithi146 ceph-mon[32821]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bbeae945-9fa2-48f9-a1d6-8bf18dcaef19"}]': finished 2024-04-25T11:19:53.854 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:53 smithi146 ceph-mon[32821]: osdmap e33: 6 total, 5 up, 6 in 2024-04-25T11:19:53.854 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:53 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T11:19:53.890 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:53 smithi152 ceph-mon[32701]: from='client.? 172.21.15.152:0/3932106848' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bbeae945-9fa2-48f9-a1d6-8bf18dcaef19"}]: dispatch 2024-04-25T11:19:53.890 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:53 smithi152 ceph-mon[32701]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bbeae945-9fa2-48f9-a1d6-8bf18dcaef19"}]: dispatch 2024-04-25T11:19:53.890 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:53 smithi152 ceph-mon[32701]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bbeae945-9fa2-48f9-a1d6-8bf18dcaef19"}]': finished 2024-04-25T11:19:53.890 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:53 smithi152 ceph-mon[32701]: osdmap e33: 6 total, 5 up, 6 in 2024-04-25T11:19:53.890 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:53 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T11:19:53.947 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:53 smithi069 ceph-mon[29188]: from='client.? 172.21.15.152:0/3932106848' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bbeae945-9fa2-48f9-a1d6-8bf18dcaef19"}]: dispatch 2024-04-25T11:19:53.947 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:53 smithi069 ceph-mon[29188]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bbeae945-9fa2-48f9-a1d6-8bf18dcaef19"}]: dispatch 2024-04-25T11:19:53.947 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:53 smithi069 ceph-mon[29188]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bbeae945-9fa2-48f9-a1d6-8bf18dcaef19"}]': finished 2024-04-25T11:19:53.947 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:53 smithi069 ceph-mon[29188]: osdmap e33: 6 total, 5 up, 6 in 2024-04-25T11:19:53.947 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:53 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T11:19:54.853 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:54 smithi146 ceph-mon[32821]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:19:54.853 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:54 smithi146 ceph-mon[32821]: from='client.? 172.21.15.152:0/2020581696' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:19:54.947 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:54 smithi069 ceph-mon[29188]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:19:54.947 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:54 smithi069 ceph-mon[29188]: from='client.? 172.21.15.152:0/2020581696' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:19:54.985 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:54 smithi152 ceph-mon[32701]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:19:54.985 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:54 smithi152 ceph-mon[32701]: from='client.? 172.21.15.152:0/2020581696' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:19:57.103 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:56 smithi146 ceph-mon[32821]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:19:57.197 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:56 smithi069 ceph-mon[29188]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:19:57.218 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:56 smithi152 ceph-mon[32701]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:19:59.042 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:19:58 smithi152 ceph-mon[32701]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:19:59.103 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:19:58 smithi146 ceph-mon[32821]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:19:59.197 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:19:58 smithi069 ceph-mon[29188]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:20:00.984 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:00 smithi152 ceph-mon[32701]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:20:00.984 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:00 smithi152 ceph-mon[32701]: overall HEALTH_OK 2024-04-25T11:20:00.984 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:00 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T11:20:00.985 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:00 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:20:01.103 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:00 smithi146 ceph-mon[32821]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:20:01.103 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:00 smithi146 ceph-mon[32821]: overall HEALTH_OK 2024-04-25T11:20:01.103 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:00 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T11:20:01.103 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:00 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:20:01.197 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:00 smithi069 ceph-mon[29188]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:20:01.197 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:00 smithi069 ceph-mon[29188]: overall HEALTH_OK 2024-04-25T11:20:01.197 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:00 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T11:20:01.197 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:00 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:20:01.984 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:01 smithi152 ceph-mon[32701]: Deploying daemon osd.5 on smithi152 2024-04-25T11:20:02.103 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:01 smithi146 ceph-mon[32821]: Deploying daemon osd.5 on smithi152 2024-04-25T11:20:02.197 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:01 smithi069 ceph-mon[29188]: Deploying daemon osd.5 on smithi152 2024-04-25T11:20:02.938 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:02 smithi152 ceph-mon[32701]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:20:03.103 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:02 smithi146 ceph-mon[32821]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:20:03.197 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:02 smithi069 ceph-mon[29188]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:20:05.093 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:04 smithi152 ceph-mon[32701]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:20:05.103 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:04 smithi146 ceph-mon[32821]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:20:05.197 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:04 smithi069 ceph-mon[29188]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:20:06.984 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:06 smithi152 ceph-mon[32701]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:20:06.984 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:06 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:06.984 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:06 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:06.985 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:06 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:20:07.103 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:06 smithi146 ceph-mon[32821]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:20:07.103 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:06 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:07.103 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:06 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:07.103 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:06 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:20:07.196 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:06 smithi069 ceph-mon[29188]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:20:07.196 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:06 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:07.197 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:06 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:07.197 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:06 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:20:09.103 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:08 smithi146 ceph-mon[32821]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:20:09.197 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:08 smithi069 ceph-mon[29188]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:20:09.200 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:08 smithi152 ceph-mon[32701]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:20:09.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:20:09.920+0000 7f9a477fe700 1 -- 172.21.15.152:0/2896210425 <== mgr.14152 v2:172.21.15.69:6800/4220065409 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9a50063ad0 con 0x7f9a3c0afb80 2024-04-25T11:20:09.918 INFO:teuthology.orchestra.run.smithi152.stdout:Created osd(s) 5 on host 'smithi152' 2024-04-25T11:20:09.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:20:09.922+0000 7f9a586a6700 1 -- 172.21.15.152:0/2896210425 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f9a3c0afb80 msgr2=0x7f9a3c0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:09.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:20:09.922+0000 7f9a586a6700 1 --2- 172.21.15.152:0/2896210425 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f9a3c0afb80 0x7f9a3c0b2090 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9a5019b830 tx=0x7f9a40008040 comp rx=0 tx=0).stop 2024-04-25T11:20:09.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:20:09.922+0000 7f9a586a6700 1 -- 172.21.15.152:0/2896210425 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9a50104210 msgr2=0x7f9a5019a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:09.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:20:09.922+0000 7f9a586a6700 1 --2- 172.21.15.152:0/2896210425 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9a50104210 0x7f9a5019a7d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c003820 tx=0x7f9a4c0047b0 comp rx=0 tx=0).stop 2024-04-25T11:20:09.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:20:09.922+0000 7f9a586a6700 1 -- 172.21.15.152:0/2896210425 shutdown_connections 2024-04-25T11:20:09.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:20:09.922+0000 7f9a586a6700 1 --2- 172.21.15.152:0/2896210425 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f9a3c0afb80 0x7f9a3c0b2090 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:09.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:20:09.922+0000 7f9a586a6700 1 --2- 172.21.15.152:0/2896210425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a50105ce0 0x7f9a501a1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:09.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:20:09.922+0000 7f9a586a6700 1 --2- 172.21.15.152:0/2896210425 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9a50104210 0x7f9a5019a7d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:09.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:20:09.922+0000 7f9a586a6700 1 --2- 172.21.15.152:0/2896210425 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9a50105340 0x7f9a5019ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:09.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:20:09.922+0000 7f9a586a6700 1 -- 172.21.15.152:0/2896210425 >> 172.21.15.152:0/2896210425 conn(0x7f9a500ff870 msgr2=0x7f9a50108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:09.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:20:09.922+0000 7f9a586a6700 1 -- 172.21.15.152:0/2896210425 shutdown_connections 2024-04-25T11:20:09.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:20:09.922+0000 7f9a586a6700 1 -- 172.21.15.152:0/2896210425 wait complete. 2024-04-25T11:20:10.504 DEBUG:teuthology.orchestra.run.smithi152:osd.5> sudo journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@osd.5.service 2024-04-25T11:20:10.507 INFO:tasks.cephadm:Deploying osd.6 on smithi069 with /dev/vg_nvme/lv_2... 2024-04-25T11:20:10.507 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T11:20:10.754 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:20:11.103 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:10 smithi146 ceph-mon[32821]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:20:11.103 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:10 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:11.103 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:10 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:11.103 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:10 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:11.104 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:10 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:11.104 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:10 smithi146 ceph-mon[32821]: from='osd.5 [v2:172.21.15.152:6808/1996555108,v1:172.21.15.152:6809/1996555108]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T11:20:11.104 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:10 smithi146 ceph-mon[32821]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T11:20:11.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:10 smithi069 ceph-mon[29188]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:20:11.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:10 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:11.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:10 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:11.111 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:10 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:11.111 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:10 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:11.111 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:10 smithi069 ceph-mon[29188]: from='osd.5 [v2:172.21.15.152:6808/1996555108,v1:172.21.15.152:6809/1996555108]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T11:20:11.111 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:10 smithi069 ceph-mon[29188]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T11:20:11.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:10 smithi152 ceph-mon[32701]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:20:11.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:10 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:11.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:10 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:11.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:10 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:11.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:10 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:11.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:10 smithi152 ceph-mon[32701]: from='osd.5 [v2:172.21.15.152:6808/1996555108,v1:172.21.15.152:6809/1996555108]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T11:20:11.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:10 smithi152 ceph-mon[32701]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T11:20:12.169 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:12 smithi069 ceph-mon[29188]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T11:20:12.169 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:12 smithi069 ceph-mon[29188]: osdmap e34: 6 total, 5 up, 6 in 2024-04-25T11:20:12.169 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:12 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T11:20:12.169 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:12 smithi069 ceph-mon[29188]: from='osd.5 [v2:172.21.15.152:6808/1996555108,v1:172.21.15.152:6809/1996555108]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T11:20:12.169 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:12 smithi069 ceph-mon[29188]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T11:20:12.169 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:12 smithi069 ceph-mon[29188]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:20:12.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:12 smithi152 ceph-mon[32701]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T11:20:12.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:12 smithi152 ceph-mon[32701]: osdmap e34: 6 total, 5 up, 6 in 2024-04-25T11:20:12.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:12 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T11:20:12.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:12 smithi152 ceph-mon[32701]: from='osd.5 [v2:172.21.15.152:6808/1996555108,v1:172.21.15.152:6809/1996555108]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T11:20:12.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:12 smithi152 ceph-mon[32701]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T11:20:12.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:12 smithi152 ceph-mon[32701]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:20:12.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:12 smithi146 ceph-mon[32821]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T11:20:12.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:12 smithi146 ceph-mon[32821]: osdmap e34: 6 total, 5 up, 6 in 2024-04-25T11:20:12.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:12 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T11:20:12.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:12 smithi146 ceph-mon[32821]: from='osd.5 [v2:172.21.15.152:6808/1996555108,v1:172.21.15.152:6809/1996555108]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T11:20:12.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:12 smithi146 ceph-mon[32821]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T11:20:12.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:12 smithi146 ceph-mon[32821]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:20:12.937 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T11:20:12.937 INFO:teuthology.orchestra.run.smithi069.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T11:20:12.938 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10+0 records in 2024-04-25T11:20:12.938 INFO:teuthology.orchestra.run.smithi069.stderr:10+0 records out 2024-04-25T11:20:12.938 INFO:teuthology.orchestra.run.smithi069.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0129518 s, 810 MB/s 2024-04-25T11:20:12.938 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping successful for: 2024-04-25T11:20:13.036 INFO:journalctl@ceph.osd.5.smithi152.stdout:Apr 25 11:20:12 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-5[44355]: 2024-04-25T11:20:12.764+0000 7f26f80b0700 -1 osd.5 0 waiting for initial osdmap 2024-04-25T11:20:13.036 INFO:journalctl@ceph.osd.5.smithi152.stdout:Apr 25 11:20:12 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-5[44355]: 2024-04-25T11:20:12.774+0000 7f26ef65c700 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:20:13.196 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:13 smithi069 ceph-mon[29188]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-04-25T11:20:13.197 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:13 smithi069 ceph-mon[29188]: osdmap e35: 6 total, 5 up, 6 in 2024-04-25T11:20:13.197 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:13 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T11:20:13.197 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:13 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T11:20:13.197 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:13 smithi069 ceph-mon[29188]: from='osd.5 ' entity='osd.5' 2024-04-25T11:20:13.197 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:13 smithi069 ceph-mon[29188]: osd.5 [v2:172.21.15.152:6808/1996555108,v1:172.21.15.152:6809/1996555108] boot 2024-04-25T11:20:13.197 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:13 smithi069 ceph-mon[29188]: osdmap e36: 6 total, 6 up, 6 in 2024-04-25T11:20:13.197 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:13 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T11:20:13.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:13 smithi146 ceph-mon[32821]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-04-25T11:20:13.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:13 smithi146 ceph-mon[32821]: osdmap e35: 6 total, 5 up, 6 in 2024-04-25T11:20:13.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:13 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T11:20:13.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:13 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T11:20:13.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:13 smithi146 ceph-mon[32821]: from='osd.5 ' entity='osd.5' 2024-04-25T11:20:13.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:13 smithi146 ceph-mon[32821]: osd.5 [v2:172.21.15.152:6808/1996555108,v1:172.21.15.152:6809/1996555108] boot 2024-04-25T11:20:13.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:13 smithi146 ceph-mon[32821]: osdmap e36: 6 total, 6 up, 6 in 2024-04-25T11:20:13.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:13 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T11:20:13.358 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:13 smithi152 ceph-mon[32701]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-04-25T11:20:13.358 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:13 smithi152 ceph-mon[32701]: osdmap e35: 6 total, 5 up, 6 in 2024-04-25T11:20:13.358 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:13 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T11:20:13.358 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:13 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T11:20:13.358 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:13 smithi152 ceph-mon[32701]: from='osd.5 ' entity='osd.5' 2024-04-25T11:20:13.359 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:13 smithi152 ceph-mon[32701]: osd.5 [v2:172.21.15.152:6808/1996555108,v1:172.21.15.152:6809/1996555108] boot 2024-04-25T11:20:13.359 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:13 smithi152 ceph-mon[32701]: osdmap e36: 6 total, 6 up, 6 in 2024-04-25T11:20:13.359 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:13 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T11:20:13.654 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi069:vg_nvme/lv_2 2024-04-25T11:20:13.911 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:20:14.301 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:14 smithi069 ceph-mon[29188]: purged_snaps scrub starts 2024-04-25T11:20:14.301 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:14 smithi069 ceph-mon[29188]: purged_snaps scrub ok 2024-04-25T11:20:14.301 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:14 smithi069 ceph-mon[29188]: Detected new or changed devices on smithi152 2024-04-25T11:20:14.301 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:14 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:14.301 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:14 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:14.301 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:14 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:20:14.302 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:14 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:20:14.302 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:14 smithi069 ceph-mon[29188]: Adjusting osd_memory_target on smithi152 to 8544M 2024-04-25T11:20:14.302 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:14 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:14.302 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:14 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:20:14.302 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:14 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:20:14.302 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:14 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:14.302 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:14 smithi069 ceph-mon[29188]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 445 MiB used, 536 GiB / 536 GiB avail 2024-04-25T11:20:14.302 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:14 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:20:14.302 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:14 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:20:14.302 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:14 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:20:14.302 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:14 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:14.302 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:14 smithi069 ceph-mon[29188]: osdmap e37: 6 total, 6 up, 6 in 2024-04-25T11:20:14.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:14 smithi146 ceph-mon[32821]: purged_snaps scrub starts 2024-04-25T11:20:14.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:14 smithi146 ceph-mon[32821]: purged_snaps scrub ok 2024-04-25T11:20:14.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:14 smithi146 ceph-mon[32821]: Detected new or changed devices on smithi152 2024-04-25T11:20:14.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:14 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:14.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:14 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:14.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:14 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:20:14.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:14 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:20:14.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:14 smithi146 ceph-mon[32821]: Adjusting osd_memory_target on smithi152 to 8544M 2024-04-25T11:20:14.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:14 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:14.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:14 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:20:14.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:14 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:20:14.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:14 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:14.354 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:14 smithi146 ceph-mon[32821]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 445 MiB used, 536 GiB / 536 GiB avail 2024-04-25T11:20:14.354 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:14 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:20:14.354 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:14 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:20:14.354 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:14 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:20:14.354 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:14 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:14.354 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:14 smithi146 ceph-mon[32821]: osdmap e37: 6 total, 6 up, 6 in 2024-04-25T11:20:14.485 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:14 smithi152 ceph-mon[32701]: purged_snaps scrub starts 2024-04-25T11:20:14.485 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:14 smithi152 ceph-mon[32701]: purged_snaps scrub ok 2024-04-25T11:20:14.485 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:14 smithi152 ceph-mon[32701]: Detected new or changed devices on smithi152 2024-04-25T11:20:14.485 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:14 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:14.485 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:14 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:14.485 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:14 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:20:14.485 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:14 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:20:14.485 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:14 smithi152 ceph-mon[32701]: Adjusting osd_memory_target on smithi152 to 8544M 2024-04-25T11:20:14.486 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:14 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:14.486 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:14 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:20:14.486 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:14 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:20:14.486 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:14 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:14.486 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:14 smithi152 ceph-mon[32701]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 445 MiB used, 536 GiB / 536 GiB avail 2024-04-25T11:20:14.486 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:14 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:20:14.486 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:14 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:20:14.486 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:14 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:20:14.486 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:14 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:14.486 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:14 smithi152 ceph-mon[32701]: osdmap e37: 6 total, 6 up, 6 in 2024-04-25T11:20:15.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:15.641+0000 7f8b6edfc700 1 -- 172.21.15.69:0/1882653354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b68104380 msgr2=0x7f8b6810ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:15.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:15.641+0000 7f8b6edfc700 1 --2- 172.21.15.69:0/1882653354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b68104380 0x7f8b6810ac30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000b0a0 tx=0x7f8b6000b3b0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:15.641+0000 7f8b6edfc700 1 -- 172.21.15.69:0/1882653354 shutdown_connections 2024-04-25T11:20:15.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:15.641+0000 7f8b6edfc700 1 --2- 172.21.15.69:0/1882653354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b68104380 0x7f8b6810ac30 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:15.641+0000 7f8b6edfc700 1 --2- 172.21.15.69:0/1882653354 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b681028b0 0x7f8b68102cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:15.641+0000 7f8b6edfc700 1 --2- 172.21.15.69:0/1882653354 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b681039e0 0x7f8b68103df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:15.641+0000 7f8b6edfc700 1 -- 172.21.15.69:0/1882653354 >> 172.21.15.69:0/1882653354 conn(0x7f8b680fdef0 msgr2=0x7f8b681003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:15.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:15.641+0000 7f8b6edfc700 1 -- 172.21.15.69:0/1882653354 shutdown_connections 2024-04-25T11:20:15.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:15.641+0000 7f8b6edfc700 1 -- 172.21.15.69:0/1882653354 wait complete. 2024-04-25T11:20:15.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:15.642+0000 7f8b6edfc700 1 Processor -- start 2024-04-25T11:20:15.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:15.642+0000 7f8b6edfc700 1 -- start start 2024-04-25T11:20:15.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:15.642+0000 7f8b6edfc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b681028b0 0x7f8b68199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:15.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:15.642+0000 7f8b6edfc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b681039e0 0x7f8b68199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:15.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:15.642+0000 7f8b6edfc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b68104380 0x7f8b681a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:15.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:15.642+0000 7f8b6edfc700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6810e1d0 con 0x7f8b681039e0 2024-04-25T11:20:15.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:15.642+0000 7f8b6edfc700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6810e050 con 0x7f8b681028b0 2024-04-25T11:20:15.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:15.642+0000 7f8b6edfc700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6810e350 con 0x7f8b68104380 2024-04-25T11:20:15.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:15.642+0000 7f8b67fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b681039e0 0x7f8b68199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:15.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:15.642+0000 7f8b67fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b681039e0 0x7f8b68199ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51182/0 (socket says 172.21.15.69:51182) 2024-04-25T11:20:15.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:15.642+0000 7f8b67fff700 1 -- 172.21.15.69:0/2219207297 learned_addr learned my addr 172.21.15.69:0/2219207297 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:20:15.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:15.642+0000 7f8b6cb98700 1 --2- 172.21.15.69:0/2219207297 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b681028b0 0x7f8b68199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:15.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:15.642+0000 7f8b6d399700 1 --2- 172.21.15.69:0/2219207297 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b68104380 0x7f8b681a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:15.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:15.642+0000 7f8b67fff700 1 -- 172.21.15.69:0/2219207297 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b68104380 msgr2=0x7f8b681a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:15.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:15.642+0000 7f8b67fff700 1 --2- 172.21.15.69:0/2219207297 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b68104380 0x7f8b681a0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:15.642+0000 7f8b67fff700 1 -- 172.21.15.69:0/2219207297 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b681028b0 msgr2=0x7f8b68199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:15.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:15.642+0000 7f8b67fff700 1 --2- 172.21.15.69:0/2219207297 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b681028b0 0x7f8b68199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:15.642+0000 7f8b67fff700 1 -- 172.21.15.69:0/2219207297 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b681a10e0 con 0x7f8b681039e0 2024-04-25T11:20:15.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:15.642+0000 7f8b67fff700 1 --2- 172.21.15.69:0/2219207297 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b681039e0 0x7f8b68199ac0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8b5400c920 tx=0x7f8b5400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:15.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:15.643+0000 7f8b65ffb700 1 -- 172.21.15.69:0/2219207297 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b54007b00 con 0x7f8b681039e0 2024-04-25T11:20:15.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:15.643+0000 7f8b6edfc700 1 -- 172.21.15.69:0/2219207297 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b681a13d0 con 0x7f8b681039e0 2024-04-25T11:20:15.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:15.643+0000 7f8b65ffb700 1 -- 172.21.15.69:0/2219207297 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8b54004cc0 con 0x7f8b681039e0 2024-04-25T11:20:15.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:15.643+0000 7f8b65ffb700 1 -- 172.21.15.69:0/2219207297 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b540054a0 con 0x7f8b681039e0 2024-04-25T11:20:15.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:15.643+0000 7f8b6edfc700 1 -- 172.21.15.69:0/2219207297 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b681a1960 con 0x7f8b681039e0 2024-04-25T11:20:15.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:15.644+0000 7f8b6edfc700 1 -- 172.21.15.69:0/2219207297 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b68051190 con 0x7f8b681039e0 2024-04-25T11:20:15.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:15.645+0000 7f8b65ffb700 1 -- 172.21.15.69:0/2219207297 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f8b54004750 con 0x7f8b681039e0 2024-04-25T11:20:15.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:15.646+0000 7f8b65ffb700 1 --2- 172.21.15.69:0/2219207297 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f8b580afb80 0x7f8b580b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:15.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:15.646+0000 7f8b6cb98700 1 --2- 172.21.15.69:0/2219207297 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f8b580afb80 0x7f8b580b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:15.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:15.646+0000 7f8b65ffb700 1 -- 172.21.15.69:0/2219207297 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4533+0+0 (secure 0 0 0) 0x7f8b540dd9f0 con 0x7f8b681039e0 2024-04-25T11:20:15.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:15.646+0000 7f8b6cb98700 1 --2- 172.21.15.69:0/2219207297 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f8b580afb80 0x7f8b580b2090 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c005fd0 tx=0x7f8b5c005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:15.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:15.651+0000 7f8b65ffb700 1 -- 172.21.15.69:0/2219207297 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8b54014030 con 0x7f8b681039e0 2024-04-25T11:20:15.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:15.858+0000 7f8b6edfc700 1 -- 172.21.15.69:0/2219207297 --> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f8b68063ad0 con 0x7f8b580afb80 2024-04-25T11:20:16.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:16 smithi146 ceph-mon[32821]: osdmap e38: 6 total, 6 up, 6 in 2024-04-25T11:20:16.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:16 smithi146 ceph-mon[32821]: pgmap v140: 1 pgs: 1 remapped+peering; 577 KiB data, 446 MiB used, 536 GiB / 536 GiB avail 2024-04-25T11:20:16.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:16 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:20:16.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:16 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:20:16.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:16 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:20:16.446 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:16 smithi069 ceph-mon[29188]: osdmap e38: 6 total, 6 up, 6 in 2024-04-25T11:20:16.447 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:16 smithi069 ceph-mon[29188]: pgmap v140: 1 pgs: 1 remapped+peering; 577 KiB data, 446 MiB used, 536 GiB / 536 GiB avail 2024-04-25T11:20:16.447 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:16 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:20:16.447 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:16 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:20:16.447 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:16 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:20:16.484 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:16 smithi152 ceph-mon[32701]: osdmap e38: 6 total, 6 up, 6 in 2024-04-25T11:20:16.484 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:16 smithi152 ceph-mon[32701]: pgmap v140: 1 pgs: 1 remapped+peering; 577 KiB data, 446 MiB used, 536 GiB / 536 GiB avail 2024-04-25T11:20:16.484 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:16 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:20:16.484 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:16 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:20:16.485 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:16 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:20:17.196 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:17 smithi069 ceph-mon[29188]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:20:17.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:17 smithi146 ceph-mon[32821]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:20:17.484 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:17 smithi152 ceph-mon[32701]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:20:18.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:18 smithi146 ceph-mon[32821]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 446 MiB used, 536 GiB / 536 GiB avail 2024-04-25T11:20:18.446 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:18 smithi069 ceph-mon[29188]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 446 MiB used, 536 GiB / 536 GiB avail 2024-04-25T11:20:18.484 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:18 smithi152 ceph-mon[32701]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 446 MiB used, 536 GiB / 536 GiB avail 2024-04-25T11:20:19.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:19 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/2724101134' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dbb61253-30bc-418e-bd75-118c80bbf931"}]: dispatch 2024-04-25T11:20:19.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:19 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/2724101134' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dbb61253-30bc-418e-bd75-118c80bbf931"}]': finished 2024-04-25T11:20:19.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:19 smithi146 ceph-mon[32821]: osdmap e39: 7 total, 6 up, 7 in 2024-04-25T11:20:19.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:19 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T11:20:19.446 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:19 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2724101134' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dbb61253-30bc-418e-bd75-118c80bbf931"}]: dispatch 2024-04-25T11:20:19.447 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:19 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2724101134' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dbb61253-30bc-418e-bd75-118c80bbf931"}]': finished 2024-04-25T11:20:19.447 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:19 smithi069 ceph-mon[29188]: osdmap e39: 7 total, 6 up, 7 in 2024-04-25T11:20:19.447 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:19 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T11:20:19.484 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:19 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/2724101134' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dbb61253-30bc-418e-bd75-118c80bbf931"}]: dispatch 2024-04-25T11:20:19.484 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:19 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/2724101134' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dbb61253-30bc-418e-bd75-118c80bbf931"}]': finished 2024-04-25T11:20:19.484 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:19 smithi152 ceph-mon[32701]: osdmap e39: 7 total, 6 up, 7 in 2024-04-25T11:20:19.484 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:19 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T11:20:20.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:20 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/4165041815' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:20:20.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:20 smithi146 ceph-mon[32821]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T11:20:20.446 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:20 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/4165041815' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:20:20.447 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:20 smithi069 ceph-mon[29188]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T11:20:20.484 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:20 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/4165041815' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:20:20.484 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:20 smithi152 ceph-mon[32701]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T11:20:23.071 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:22 smithi146 ceph-mon[32821]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-25T11:20:23.132 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:22 smithi069 ceph-mon[29188]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-25T11:20:23.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:22 smithi152 ceph-mon[32701]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-25T11:20:25.102 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:24 smithi146 ceph-mon[32821]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-04-25T11:20:25.125 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:24 smithi069 ceph-mon[29188]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-04-25T11:20:25.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:24 smithi152 ceph-mon[32701]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-04-25T11:20:27.084 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:26 smithi069 ceph-mon[29188]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T11:20:27.084 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:26 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T11:20:27.084 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:26 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:20:27.102 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:26 smithi146 ceph-mon[32821]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T11:20:27.102 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:26 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T11:20:27.103 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:26 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:20:27.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:26 smithi152 ceph-mon[32701]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T11:20:27.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:26 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T11:20:27.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:26 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:20:28.102 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:27 smithi146 ceph-mon[32821]: Deploying daemon osd.6 on smithi069 2024-04-25T11:20:28.120 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:27 smithi069 ceph-mon[29188]: Deploying daemon osd.6 on smithi069 2024-04-25T11:20:28.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:27 smithi152 ceph-mon[32701]: Deploying daemon osd.6 on smithi069 2024-04-25T11:20:28.946 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:28 smithi069 ceph-mon[29188]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T11:20:29.102 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:28 smithi146 ceph-mon[32821]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T11:20:29.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:28 smithi152 ceph-mon[32701]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T11:20:31.102 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:30 smithi146 ceph-mon[32821]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-04-25T11:20:31.196 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:30 smithi069 ceph-mon[29188]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-04-25T11:20:31.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:30 smithi152 ceph-mon[32701]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-04-25T11:20:32.911 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:32 smithi069 ceph-mon[29188]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-25T11:20:32.911 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:32 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:32.911 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:32 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:20:32.911 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:32 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:33.102 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:32 smithi146 ceph-mon[32821]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-25T11:20:33.103 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:32 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:33.103 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:32 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:20:33.103 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:32 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:33.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:32 smithi152 ceph-mon[32701]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-25T11:20:33.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:32 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:33.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:32 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:20:33.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:32 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:35.098 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:34 smithi069 ceph-mon[29188]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-25T11:20:35.102 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:34 smithi146 ceph-mon[32821]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-25T11:20:35.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:34 smithi152 ceph-mon[32701]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-25T11:20:36.478 INFO:teuthology.orchestra.run.smithi069.stdout:Created osd(s) 6 on host 'smithi069' 2024-04-25T11:20:36.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:36.480+0000 7f8b65ffb700 1 -- 172.21.15.69:0/2219207297 <== mgr.14152 v2:172.21.15.69:6800/4220065409 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8b68063ad0 con 0x7f8b580afb80 2024-04-25T11:20:36.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:36.483+0000 7f8b6edfc700 1 -- 172.21.15.69:0/2219207297 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f8b580afb80 msgr2=0x7f8b580b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:36.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:36.483+0000 7f8b6edfc700 1 --2- 172.21.15.69:0/2219207297 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f8b580afb80 0x7f8b580b2090 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c005fd0 tx=0x7f8b5c005dd0 comp rx=0 tx=0).stop 2024-04-25T11:20:36.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:36.483+0000 7f8b6edfc700 1 -- 172.21.15.69:0/2219207297 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b681039e0 msgr2=0x7f8b68199ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:36.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:36.483+0000 7f8b6edfc700 1 --2- 172.21.15.69:0/2219207297 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b681039e0 0x7f8b68199ac0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8b5400c920 tx=0x7f8b5400cce0 comp rx=0 tx=0).stop 2024-04-25T11:20:36.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:36.483+0000 7f8b6edfc700 1 -- 172.21.15.69:0/2219207297 shutdown_connections 2024-04-25T11:20:36.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:36.483+0000 7f8b6edfc700 1 --2- 172.21.15.69:0/2219207297 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f8b580afb80 0x7f8b580b2090 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:36.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:36.483+0000 7f8b6edfc700 1 --2- 172.21.15.69:0/2219207297 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b68104380 0x7f8b681a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:36.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:36.483+0000 7f8b6edfc700 1 --2- 172.21.15.69:0/2219207297 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b681028b0 0x7f8b68199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:36.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:36.483+0000 7f8b6edfc700 1 --2- 172.21.15.69:0/2219207297 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b681039e0 0x7f8b68199ac0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:36.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:36.483+0000 7f8b6edfc700 1 -- 172.21.15.69:0/2219207297 >> 172.21.15.69:0/2219207297 conn(0x7f8b680fdef0 msgr2=0x7f8b68100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:36.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:36.483+0000 7f8b6edfc700 1 -- 172.21.15.69:0/2219207297 shutdown_connections 2024-04-25T11:20:36.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:20:36.483+0000 7f8b6edfc700 1 -- 172.21.15.69:0/2219207297 wait complete. 2024-04-25T11:20:36.696 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:36 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:36.696 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:36 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:36.696 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:36 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:20:36.696 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:36 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:20:36.696 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:36 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:36.696 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:36 smithi069 ceph-mon[29188]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-25T11:20:36.697 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:36 smithi069 ceph-mon[29188]: from='osd.6 [v2:172.21.15.69:6818/491215762,v1:172.21.15.69:6819/491215762]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T11:20:36.734 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:36 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:36.734 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:36 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:36.734 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:36 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:20:36.734 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:36 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:20:36.734 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:36 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:36.734 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:36 smithi152 ceph-mon[32701]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-25T11:20:36.734 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:36 smithi152 ceph-mon[32701]: from='osd.6 [v2:172.21.15.69:6818/491215762,v1:172.21.15.69:6819/491215762]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T11:20:36.852 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:36 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:36.853 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:36 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:36.853 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:36 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:20:36.853 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:36 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:20:36.853 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:36 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:36.853 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:36 smithi146 ceph-mon[32821]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-25T11:20:36.853 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:36 smithi146 ceph-mon[32821]: from='osd.6 [v2:172.21.15.69:6818/491215762,v1:172.21.15.69:6819/491215762]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T11:20:37.132 DEBUG:teuthology.orchestra.run.smithi069:osd.6> sudo journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@osd.6.service 2024-04-25T11:20:37.135 INFO:tasks.cephadm:Deploying osd.7 on smithi146 with /dev/vg_nvme/lv_2... 2024-04-25T11:20:37.135 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T11:20:37.383 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T11:20:37.734 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:37 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:37.734 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:37 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:37.734 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:37 smithi152 ceph-mon[32701]: from='osd.6 [v2:172.21.15.69:6818/491215762,v1:172.21.15.69:6819/491215762]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T11:20:37.734 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:37 smithi152 ceph-mon[32701]: osdmap e40: 7 total, 6 up, 7 in 2024-04-25T11:20:37.734 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:37 smithi152 ceph-mon[32701]: from='osd.6 [v2:172.21.15.69:6818/491215762,v1:172.21.15.69:6819/491215762]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-25T11:20:37.734 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:37 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T11:20:37.734 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:37 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:20:37.813 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:37 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:37.813 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:37 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:37.813 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:37 smithi146 ceph-mon[32821]: from='osd.6 [v2:172.21.15.69:6818/491215762,v1:172.21.15.69:6819/491215762]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T11:20:37.813 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:37 smithi146 ceph-mon[32821]: osdmap e40: 7 total, 6 up, 7 in 2024-04-25T11:20:37.813 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:37 smithi146 ceph-mon[32821]: from='osd.6 [v2:172.21.15.69:6818/491215762,v1:172.21.15.69:6819/491215762]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-25T11:20:37.813 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:37 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T11:20:37.813 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:37 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:20:37.946 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:37 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:37.946 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:37 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:37.946 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:37 smithi069 ceph-mon[29188]: from='osd.6 [v2:172.21.15.69:6818/491215762,v1:172.21.15.69:6819/491215762]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T11:20:37.946 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:37 smithi069 ceph-mon[29188]: osdmap e40: 7 total, 6 up, 7 in 2024-04-25T11:20:37.946 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:37 smithi069 ceph-mon[29188]: from='osd.6 [v2:172.21.15.69:6818/491215762,v1:172.21.15.69:6819/491215762]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-25T11:20:37.946 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:37 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T11:20:37.947 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:37 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:20:38.446 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 25 11:20:38 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-6[57622]: 2024-04-25T11:20:38.167+0000 7fda79285700 -1 osd.6 0 waiting for initial osdmap 2024-04-25T11:20:38.447 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 25 11:20:38 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-6[57622]: 2024-04-25T11:20:38.179+0000 7fda71032700 -1 osd.6 41 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:20:38.778 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:38 smithi069 ceph-mon[29188]: from='osd.6 [v2:172.21.15.69:6818/491215762,v1:172.21.15.69:6819/491215762]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-04-25T11:20:38.778 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:38 smithi069 ceph-mon[29188]: osdmap e41: 7 total, 6 up, 7 in 2024-04-25T11:20:38.778 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:38 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T11:20:38.778 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:38 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T11:20:38.778 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:38 smithi069 ceph-mon[29188]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-25T11:20:38.778 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:38 smithi069 ceph-mon[29188]: from='osd.6 [v2:172.21.15.69:6818/491215762,v1:172.21.15.69:6819/491215762]' entity='osd.6' 2024-04-25T11:20:38.801 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:38 smithi146 ceph-mon[32821]: from='osd.6 [v2:172.21.15.69:6818/491215762,v1:172.21.15.69:6819/491215762]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-04-25T11:20:38.802 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:38 smithi146 ceph-mon[32821]: osdmap e41: 7 total, 6 up, 7 in 2024-04-25T11:20:38.802 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:38 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T11:20:38.802 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:38 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T11:20:38.802 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:38 smithi146 ceph-mon[32821]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-25T11:20:38.802 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:38 smithi146 ceph-mon[32821]: from='osd.6 [v2:172.21.15.69:6818/491215762,v1:172.21.15.69:6819/491215762]' entity='osd.6' 2024-04-25T11:20:38.984 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:38 smithi152 ceph-mon[32701]: from='osd.6 [v2:172.21.15.69:6818/491215762,v1:172.21.15.69:6819/491215762]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-04-25T11:20:38.984 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:38 smithi152 ceph-mon[32701]: osdmap e41: 7 total, 6 up, 7 in 2024-04-25T11:20:38.984 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:38 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T11:20:38.984 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:38 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T11:20:38.984 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:38 smithi152 ceph-mon[32701]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-04-25T11:20:38.984 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:38 smithi152 ceph-mon[32701]: from='osd.6 [v2:172.21.15.69:6818/491215762,v1:172.21.15.69:6819/491215762]' entity='osd.6' 2024-04-25T11:20:39.412 INFO:teuthology.orchestra.run.smithi146.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T11:20:39.412 INFO:teuthology.orchestra.run.smithi146.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T11:20:39.412 INFO:teuthology.orchestra.run.smithi146.stderr: stderr: 10+0 records in 2024-04-25T11:20:39.412 INFO:teuthology.orchestra.run.smithi146.stderr:10+0 records out 2024-04-25T11:20:39.413 INFO:teuthology.orchestra.run.smithi146.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113855 s, 921 MB/s 2024-04-25T11:20:39.413 INFO:teuthology.orchestra.run.smithi146.stderr:--> Zapping successful for: 2024-04-25T11:20:39.755 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:39 smithi146 ceph-mon[32821]: purged_snaps scrub starts 2024-04-25T11:20:39.755 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:39 smithi146 ceph-mon[32821]: purged_snaps scrub ok 2024-04-25T11:20:39.755 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:39 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T11:20:39.756 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:39 smithi146 ceph-mon[32821]: osd.6 [v2:172.21.15.69:6818/491215762,v1:172.21.15.69:6819/491215762] boot 2024-04-25T11:20:39.756 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:39 smithi146 ceph-mon[32821]: osdmap e42: 7 total, 7 up, 7 in 2024-04-25T11:20:39.756 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:39 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T11:20:39.829 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:39 smithi069 ceph-mon[29188]: purged_snaps scrub starts 2024-04-25T11:20:39.829 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:39 smithi069 ceph-mon[29188]: purged_snaps scrub ok 2024-04-25T11:20:39.829 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:39 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T11:20:39.830 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:39 smithi069 ceph-mon[29188]: osd.6 [v2:172.21.15.69:6818/491215762,v1:172.21.15.69:6819/491215762] boot 2024-04-25T11:20:39.830 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:39 smithi069 ceph-mon[29188]: osdmap e42: 7 total, 7 up, 7 in 2024-04-25T11:20:39.830 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:39 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T11:20:39.983 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:39 smithi152 ceph-mon[32701]: purged_snaps scrub starts 2024-04-25T11:20:39.984 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:39 smithi152 ceph-mon[32701]: purged_snaps scrub ok 2024-04-25T11:20:39.984 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:39 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T11:20:39.984 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:39 smithi152 ceph-mon[32701]: osd.6 [v2:172.21.15.69:6818/491215762,v1:172.21.15.69:6819/491215762] boot 2024-04-25T11:20:39.984 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:39 smithi152 ceph-mon[32701]: osdmap e42: 7 total, 7 up, 7 in 2024-04-25T11:20:39.984 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:39 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T11:20:40.058 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi146:vg_nvme/lv_2 2024-04-25T11:20:40.303 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T11:20:40.602 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:40 smithi146 ceph-mon[32821]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:20:40.946 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:40 smithi069 ceph-mon[29188]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:20:40.984 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:40 smithi152 ceph-mon[32701]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:20:41.716 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:41 smithi146 ceph-mon[32821]: osdmap e43: 7 total, 7 up, 7 in 2024-04-25T11:20:41.716 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:41 smithi146 ceph-mon[32821]: Detected new or changed devices on smithi069 2024-04-25T11:20:41.716 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:41 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:41.716 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:41 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:41.716 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:41 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:20:41.716 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:41 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:20:41.717 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:41 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:20:41.717 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:41 smithi146 ceph-mon[32821]: Adjusting osd_memory_target on smithi069 to 5696M 2024-04-25T11:20:41.717 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:41 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:41.717 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:41 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:20:41.717 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:41 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:20:41.717 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:41 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:41.885 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:41 smithi152 ceph-mon[32701]: osdmap e43: 7 total, 7 up, 7 in 2024-04-25T11:20:41.885 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:41 smithi152 ceph-mon[32701]: Detected new or changed devices on smithi069 2024-04-25T11:20:41.885 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:41 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:41.885 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:41 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:41.885 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:41 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:20:41.885 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:41 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:20:41.885 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:41 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:20:41.886 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:41 smithi152 ceph-mon[32701]: Adjusting osd_memory_target on smithi069 to 5696M 2024-04-25T11:20:41.886 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:41 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:41.886 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:41 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:20:41.886 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:41 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:20:41.886 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:41 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:41.946 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:41 smithi069 ceph-mon[29188]: osdmap e43: 7 total, 7 up, 7 in 2024-04-25T11:20:41.946 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:41 smithi069 ceph-mon[29188]: Detected new or changed devices on smithi069 2024-04-25T11:20:41.946 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:41 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:41.946 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:41 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:41.947 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:41 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:20:41.947 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:41 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:20:41.947 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:41 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:20:41.947 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:41 smithi069 ceph-mon[29188]: Adjusting osd_memory_target on smithi069 to 5696M 2024-04-25T11:20:41.947 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:41 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:41.947 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:41 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:20:41.947 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:41 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:20:41.947 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:41 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:41.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:20:41.958+0000 7fa8d9ad1700 1 -- 172.21.15.146:0/3578384987 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8d4103120 msgr2=0x7fa8d4103530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:41.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:20:41.958+0000 7fa8d9ad1700 1 --2- 172.21.15.146:0/3578384987 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8d4103120 0x7fa8d4103530 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4009990 tx=0x7fa8c4009ca0 comp rx=0 tx=0).stop 2024-04-25T11:20:41.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:20:41.958+0000 7fa8d9ad1700 1 -- 172.21.15.146:0/3578384987 shutdown_connections 2024-04-25T11:20:41.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:20:41.958+0000 7fa8d9ad1700 1 --2- 172.21.15.146:0/3578384987 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa8d4104bf0 0x7fa8d410b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:41.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:20:41.958+0000 7fa8d9ad1700 1 --2- 172.21.15.146:0/3578384987 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8d4103120 0x7fa8d4103530 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:41.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:20:41.958+0000 7fa8d9ad1700 1 --2- 172.21.15.146:0/3578384987 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8d4104250 0x7fa8d4104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:41.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:20:41.958+0000 7fa8d9ad1700 1 -- 172.21.15.146:0/3578384987 >> 172.21.15.146:0/3578384987 conn(0x7fa8d40fe760 msgr2=0x7fa8d4100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:41.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:20:41.958+0000 7fa8d9ad1700 1 -- 172.21.15.146:0/3578384987 shutdown_connections 2024-04-25T11:20:41.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:20:41.958+0000 7fa8d9ad1700 1 -- 172.21.15.146:0/3578384987 wait complete. 2024-04-25T11:20:41.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:20:41.958+0000 7fa8d9ad1700 1 Processor -- start 2024-04-25T11:20:41.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:20:41.959+0000 7fa8d9ad1700 1 -- start start 2024-04-25T11:20:41.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:20:41.959+0000 7fa8d9ad1700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa8d4103120 0x7fa8d410da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:41.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:20:41.959+0000 7fa8d9ad1700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8d4104250 0x7fa8d410c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:41.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:20:41.959+0000 7fa8d9ad1700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8d4104bf0 0x7fa8d410c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:41.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:20:41.959+0000 7fa8d9ad1700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d410fbb0 con 0x7fa8d4104bf0 2024-04-25T11:20:41.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:20:41.959+0000 7fa8d9ad1700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d410fa30 con 0x7fa8d4104250 2024-04-25T11:20:41.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:20:41.959+0000 7fa8d9ad1700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d410fd30 con 0x7fa8d4103120 2024-04-25T11:20:41.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:20:41.959+0000 7fa8d2ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8d4104250 0x7fa8d410c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:41.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:20:41.959+0000 7fa8d2ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8d4104250 0x7fa8d410c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:43220/0 (socket says 172.21.15.146:43220) 2024-04-25T11:20:41.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:20:41.959+0000 7fa8d2ffd700 1 -- 172.21.15.146:0/1564111658 learned_addr learned my addr 172.21.15.146:0/1564111658 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-04-25T11:20:41.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:20:41.959+0000 7fa8d37fe700 1 --2- 172.21.15.146:0/1564111658 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa8d4103120 0x7fa8d410da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:41.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:20:41.959+0000 7fa8d3fff700 1 --2- 172.21.15.146:0/1564111658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8d4104bf0 0x7fa8d410c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:41.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:20:41.959+0000 7fa8d2ffd700 1 -- 172.21.15.146:0/1564111658 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa8d4103120 msgr2=0x7fa8d410da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:41.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:20:41.959+0000 7fa8d2ffd700 1 --2- 172.21.15.146:0/1564111658 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa8d4103120 0x7fa8d410da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:41.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:20:41.959+0000 7fa8d2ffd700 1 -- 172.21.15.146:0/1564111658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8d4104bf0 msgr2=0x7fa8d410c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:41.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:20:41.959+0000 7fa8d2ffd700 1 --2- 172.21.15.146:0/1564111658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8d4104bf0 0x7fa8d410c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:41.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:20:41.959+0000 7fa8d2ffd700 1 -- 172.21.15.146:0/1564111658 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8d410cdb0 con 0x7fa8d4104250 2024-04-25T11:20:41.959 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:20:41.959+0000 7fa8d3fff700 1 --2- 172.21.15.146:0/1564111658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8d4104bf0 0x7fa8d410c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:41.959 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:20:41.960+0000 7fa8d2ffd700 1 --2- 172.21.15.146:0/1564111658 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8d4104250 0x7fa8d410c110 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa8c800c920 tx=0x7fa8c800cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:41.959 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:20:41.960+0000 7fa8d0ff9700 1 -- 172.21.15.146:0/1564111658 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c8007b00 con 0x7fa8d4104250 2024-04-25T11:20:41.959 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:20:41.960+0000 7fa8d9ad1700 1 -- 172.21.15.146:0/1564111658 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8d41a47f0 con 0x7fa8d4104250 2024-04-25T11:20:41.959 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:20:41.960+0000 7fa8d0ff9700 1 -- 172.21.15.146:0/1564111658 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8c8004cc0 con 0x7fa8d4104250 2024-04-25T11:20:41.959 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:20:41.960+0000 7fa8d0ff9700 1 -- 172.21.15.146:0/1564111658 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c80053f0 con 0x7fa8d4104250 2024-04-25T11:20:41.959 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:20:41.960+0000 7fa8d9ad1700 1 -- 172.21.15.146:0/1564111658 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8d41a4cd0 con 0x7fa8d4104250 2024-04-25T11:20:41.960 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:20:41.961+0000 7fa8d9ad1700 1 -- 172.21.15.146:0/1564111658 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8d4051190 con 0x7fa8d4104250 2024-04-25T11:20:41.960 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:20:41.962+0000 7fa8d0ff9700 1 -- 172.21.15.146:0/1564111658 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fa8c8004750 con 0x7fa8d4104250 2024-04-25T11:20:41.960 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:20:41.963+0000 7fa8d0ff9700 1 --2- 172.21.15.146:0/1564111658 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fa8bc0afb80 0x7fa8bc0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:41.960 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:20:41.963+0000 7fa8d0ff9700 1 -- 172.21.15.146:0/1564111658 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4965+0+0 (secure 0 0 0) 0x7fa8c80ddab0 con 0x7fa8d4104250 2024-04-25T11:20:41.960 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:20:41.964+0000 7fa8d37fe700 1 --2- 172.21.15.146:0/1564111658 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fa8bc0afb80 0x7fa8bc0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:41.961 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:20:41.964+0000 7fa8d37fe700 1 --2- 172.21.15.146:0/1564111658 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fa8bc0afb80 0x7fa8bc0b2090 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4006010 tx=0x7fa8c401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:41.965 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:20:41.968+0000 7fa8d0ff9700 1 -- 172.21.15.146:0/1564111658 <== mon.2 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa8c8014030 con 0x7fa8d4104250 2024-04-25T11:20:42.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:20:42.178+0000 7fa8d9ad1700 1 -- 172.21.15.146:0/1564111658 --> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fa8d4063ad0 con 0x7fa8bc0afb80 2024-04-25T11:20:42.789 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:42 smithi146 ceph-mon[32821]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:20:42.789 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:42 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:20:42.789 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:42 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:20:42.789 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:42 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:20:42.901 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:42 smithi152 ceph-mon[32701]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:20:42.901 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:42 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:20:42.901 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:42 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:20:42.901 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:42 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:20:42.946 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:42 smithi069 ceph-mon[29188]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:20:42.946 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:42 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:20:42.946 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:42 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:20:42.946 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:42 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:20:43.798 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:43 smithi146 ceph-mon[32821]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:20:43.946 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:43 smithi069 ceph-mon[29188]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:20:43.983 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:43 smithi152 ceph-mon[32701]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:20:44.852 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:44 smithi146 ceph-mon[32821]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:20:44.946 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:44 smithi069 ceph-mon[29188]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:20:44.983 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:44 smithi152 ceph-mon[32701]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:20:45.602 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:45 smithi146 ceph-mon[32821]: from='client.? 172.21.15.146:0/1204931837' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "75f23854-e283-4e3e-94cf-7b4079d83876"}]: dispatch 2024-04-25T11:20:45.603 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:45 smithi146 ceph-mon[32821]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "75f23854-e283-4e3e-94cf-7b4079d83876"}]: dispatch 2024-04-25T11:20:45.603 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:45 smithi146 ceph-mon[32821]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "75f23854-e283-4e3e-94cf-7b4079d83876"}]': finished 2024-04-25T11:20:45.603 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:45 smithi146 ceph-mon[32821]: osdmap e44: 8 total, 7 up, 8 in 2024-04-25T11:20:45.603 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:45 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T11:20:45.848 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:45 smithi069 ceph-mon[29188]: from='client.? 172.21.15.146:0/1204931837' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "75f23854-e283-4e3e-94cf-7b4079d83876"}]: dispatch 2024-04-25T11:20:45.848 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:45 smithi069 ceph-mon[29188]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "75f23854-e283-4e3e-94cf-7b4079d83876"}]: dispatch 2024-04-25T11:20:45.848 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:45 smithi069 ceph-mon[29188]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "75f23854-e283-4e3e-94cf-7b4079d83876"}]': finished 2024-04-25T11:20:45.848 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:45 smithi069 ceph-mon[29188]: osdmap e44: 8 total, 7 up, 8 in 2024-04-25T11:20:45.848 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:45 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T11:20:45.984 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:45 smithi152 ceph-mon[32701]: from='client.? 172.21.15.146:0/1204931837' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "75f23854-e283-4e3e-94cf-7b4079d83876"}]: dispatch 2024-04-25T11:20:45.984 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:45 smithi152 ceph-mon[32701]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "75f23854-e283-4e3e-94cf-7b4079d83876"}]: dispatch 2024-04-25T11:20:45.984 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:45 smithi152 ceph-mon[32701]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "75f23854-e283-4e3e-94cf-7b4079d83876"}]': finished 2024-04-25T11:20:45.984 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:45 smithi152 ceph-mon[32701]: osdmap e44: 8 total, 7 up, 8 in 2024-04-25T11:20:45.984 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:45 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T11:20:46.696 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:46 smithi069 ceph-mon[29188]: from='client.? 172.21.15.146:0/2457319686' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:20:46.696 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:46 smithi069 ceph-mon[29188]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:20:46.852 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:46 smithi146 ceph-mon[32821]: from='client.? 172.21.15.146:0/2457319686' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:20:46.852 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:46 smithi146 ceph-mon[32821]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:20:46.983 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:46 smithi152 ceph-mon[32701]: from='client.? 172.21.15.146:0/2457319686' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:20:46.984 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:46 smithi152 ceph-mon[32701]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:20:49.007 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:48 smithi146 ceph-mon[32821]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:20:49.196 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:48 smithi069 ceph-mon[29188]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:20:49.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:48 smithi152 ceph-mon[32701]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:20:51.047 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:50 smithi146 ceph-mon[32821]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:20:51.196 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:50 smithi069 ceph-mon[29188]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:20:51.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:50 smithi152 ceph-mon[32701]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:20:53.048 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:52 smithi146 ceph-mon[32821]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:20:53.048 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:52 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T11:20:53.048 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:52 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:20:53.196 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:52 smithi069 ceph-mon[29188]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:20:53.196 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:52 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T11:20:53.196 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:52 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:20:53.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:52 smithi152 ceph-mon[32701]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:20:53.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:52 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T11:20:53.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:52 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:20:53.983 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:53 smithi152 ceph-mon[32701]: Deploying daemon osd.7 on smithi146 2024-04-25T11:20:54.102 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:53 smithi146 ceph-mon[32821]: Deploying daemon osd.7 on smithi146 2024-04-25T11:20:54.196 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:53 smithi069 ceph-mon[29188]: Deploying daemon osd.7 on smithi146 2024-04-25T11:20:54.852 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:54 smithi146 ceph-mon[32821]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:20:55.196 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:54 smithi069 ceph-mon[29188]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:20:55.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:54 smithi152 ceph-mon[32701]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:20:57.102 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:56 smithi146 ceph-mon[32821]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:20:57.196 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:56 smithi069 ceph-mon[29188]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:20:57.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:56 smithi152 ceph-mon[32701]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:20:59.102 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:58 smithi146 ceph-mon[32821]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:20:59.102 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:58 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:59.102 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:58 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:59.102 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:20:58 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:20:59.196 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:58 smithi069 ceph-mon[29188]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:20:59.196 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:58 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:59.196 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:58 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:59.196 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:20:58 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:20:59.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:58 smithi152 ceph-mon[32701]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:20:59.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:58 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:59.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:58 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:20:59.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:20:58 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:21:01.102 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:00 smithi146 ceph-mon[32821]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:21:01.196 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:00 smithi069 ceph-mon[29188]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:21:01.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:00 smithi152 ceph-mon[32701]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:21:01.942 INFO:teuthology.orchestra.run.smithi146.stdout:Created osd(s) 7 on host 'smithi146' 2024-04-25T11:21:01.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:21:01.945+0000 7fa8d0ff9700 1 -- 172.21.15.146:0/1564111658 <== mgr.14152 v2:172.21.15.69:6800/4220065409 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa8d4063ad0 con 0x7fa8bc0afb80 2024-04-25T11:21:01.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:21:01.948+0000 7fa8d9ad1700 1 -- 172.21.15.146:0/1564111658 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fa8bc0afb80 msgr2=0x7fa8bc0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:01.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:21:01.948+0000 7fa8d9ad1700 1 --2- 172.21.15.146:0/1564111658 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fa8bc0afb80 0x7fa8bc0b2090 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4006010 tx=0x7fa8c401a040 comp rx=0 tx=0).stop 2024-04-25T11:21:01.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:21:01.948+0000 7fa8d9ad1700 1 -- 172.21.15.146:0/1564111658 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8d4104250 msgr2=0x7fa8d410c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:01.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:21:01.948+0000 7fa8d9ad1700 1 --2- 172.21.15.146:0/1564111658 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8d4104250 0x7fa8d410c110 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa8c800c920 tx=0x7fa8c800cc30 comp rx=0 tx=0).stop 2024-04-25T11:21:01.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:21:01.948+0000 7fa8d9ad1700 1 -- 172.21.15.146:0/1564111658 shutdown_connections 2024-04-25T11:21:01.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:21:01.949+0000 7fa8d9ad1700 1 --2- 172.21.15.146:0/1564111658 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fa8bc0afb80 0x7fa8bc0b2090 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:01.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:21:01.949+0000 7fa8d9ad1700 1 --2- 172.21.15.146:0/1564111658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8d4104bf0 0x7fa8d410c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:01.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:21:01.949+0000 7fa8d9ad1700 1 --2- 172.21.15.146:0/1564111658 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa8d4103120 0x7fa8d410da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:01.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:21:01.949+0000 7fa8d9ad1700 1 --2- 172.21.15.146:0/1564111658 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa8d4104250 0x7fa8d410c110 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:01.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:21:01.949+0000 7fa8d9ad1700 1 -- 172.21.15.146:0/1564111658 >> 172.21.15.146:0/1564111658 conn(0x7fa8d40fe760 msgr2=0x7fa8d41074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:01.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:21:01.949+0000 7fa8d9ad1700 1 -- 172.21.15.146:0/1564111658 shutdown_connections 2024-04-25T11:21:01.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:21:01.949+0000 7fa8d9ad1700 1 -- 172.21.15.146:0/1564111658 wait complete. 2024-04-25T11:21:02.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:01 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:02.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:01 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:02.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:01 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:21:02.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:01 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:21:02.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:01 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:02.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:01 smithi152 ceph-mon[32701]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:21:02.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:01 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:02.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:01 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:02.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:01 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:21:02.294 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:01 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:02.294 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:01 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:02.294 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:01 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:21:02.294 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:01 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:21:02.294 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:01 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:02.294 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:01 smithi146 ceph-mon[32821]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:21:02.294 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:01 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:02.294 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:01 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:02.294 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:01 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:21:02.446 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:01 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:02.446 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:01 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:02.446 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:01 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:21:02.446 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:01 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:21:02.446 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:01 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:02.446 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:01 smithi069 ceph-mon[29188]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:21:02.446 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:01 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:02.446 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:01 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:02.446 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:01 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:21:02.629 DEBUG:teuthology.orchestra.run.smithi146:osd.7> sudo journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@osd.7.service 2024-04-25T11:21:02.632 INFO:tasks.cephadm:Deploying osd.8 on smithi152 with /dev/vg_nvme/lv_2... 2024-04-25T11:21:02.632 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T11:21:02.889 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T11:21:03.221 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:02 smithi152 ceph-mon[32701]: from='osd.7 [v2:172.21.15.146:6816/2330294690,v1:172.21.15.146:6817/2330294690]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T11:21:03.221 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:02 smithi152 ceph-mon[32701]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T11:21:03.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:02 smithi146 ceph-mon[32821]: from='osd.7 [v2:172.21.15.146:6816/2330294690,v1:172.21.15.146:6817/2330294690]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T11:21:03.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:02 smithi146 ceph-mon[32821]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T11:21:03.446 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:02 smithi069 ceph-mon[29188]: from='osd.7 [v2:172.21.15.146:6816/2330294690,v1:172.21.15.146:6817/2330294690]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T11:21:03.446 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:02 smithi069 ceph-mon[29188]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T11:21:04.064 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:03 smithi146 ceph-mon[32821]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T11:21:04.065 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:03 smithi146 ceph-mon[32821]: from='osd.7 [v2:172.21.15.146:6816/2330294690,v1:172.21.15.146:6817/2330294690]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-04-25T11:21:04.065 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:03 smithi146 ceph-mon[32821]: osdmap e45: 8 total, 7 up, 8 in 2024-04-25T11:21:04.065 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:03 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T11:21:04.065 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:03 smithi146 ceph-mon[32821]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-04-25T11:21:04.065 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:03 smithi146 ceph-mon[32821]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:21:04.065 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:21:03 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:21:03.991+0000 7f8fdeb96700 -1 osd.7 0 waiting for initial osdmap 2024-04-25T11:21:04.065 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:21:04 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:21:04.001+0000 7f8fda94b700 -1 osd.7 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:21:04.262 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:03 smithi152 ceph-mon[32701]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T11:21:04.262 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:03 smithi152 ceph-mon[32701]: from='osd.7 [v2:172.21.15.146:6816/2330294690,v1:172.21.15.146:6817/2330294690]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-04-25T11:21:04.262 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:03 smithi152 ceph-mon[32701]: osdmap e45: 8 total, 7 up, 8 in 2024-04-25T11:21:04.262 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:03 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T11:21:04.262 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:03 smithi152 ceph-mon[32701]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-04-25T11:21:04.262 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:03 smithi152 ceph-mon[32701]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:21:04.446 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:03 smithi069 ceph-mon[29188]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T11:21:04.446 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:03 smithi069 ceph-mon[29188]: from='osd.7 [v2:172.21.15.146:6816/2330294690,v1:172.21.15.146:6817/2330294690]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-04-25T11:21:04.446 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:03 smithi069 ceph-mon[29188]: osdmap e45: 8 total, 7 up, 8 in 2024-04-25T11:21:04.446 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:03 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T11:21:04.446 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:03 smithi069 ceph-mon[29188]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-04-25T11:21:04.446 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:03 smithi069 ceph-mon[29188]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:21:04.966 INFO:teuthology.orchestra.run.smithi152.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T11:21:04.967 INFO:teuthology.orchestra.run.smithi152.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T11:21:04.967 INFO:teuthology.orchestra.run.smithi152.stderr: stderr: 10+0 records in 2024-04-25T11:21:04.967 INFO:teuthology.orchestra.run.smithi152.stderr:10+0 records out 2024-04-25T11:21:04.967 INFO:teuthology.orchestra.run.smithi152.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115417 s, 909 MB/s 2024-04-25T11:21:04.967 INFO:teuthology.orchestra.run.smithi152.stderr:--> Zapping successful for: 2024-04-25T11:21:05.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:04 smithi152 ceph-mon[32701]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-04-25T11:21:05.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:04 smithi152 ceph-mon[32701]: osdmap e46: 8 total, 7 up, 8 in 2024-04-25T11:21:05.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:04 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T11:21:05.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:04 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T11:21:05.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:04 smithi152 ceph-mon[32701]: osd.7 [v2:172.21.15.146:6816/2330294690,v1:172.21.15.146:6817/2330294690] boot 2024-04-25T11:21:05.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:04 smithi152 ceph-mon[32701]: osdmap e47: 8 total, 8 up, 8 in 2024-04-25T11:21:05.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:04 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T11:21:05.317 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:04 smithi146 ceph-mon[32821]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-04-25T11:21:05.317 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:04 smithi146 ceph-mon[32821]: osdmap e46: 8 total, 7 up, 8 in 2024-04-25T11:21:05.317 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:04 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T11:21:05.317 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:04 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T11:21:05.317 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:04 smithi146 ceph-mon[32821]: osd.7 [v2:172.21.15.146:6816/2330294690,v1:172.21.15.146:6817/2330294690] boot 2024-04-25T11:21:05.317 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:04 smithi146 ceph-mon[32821]: osdmap e47: 8 total, 8 up, 8 in 2024-04-25T11:21:05.317 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:04 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T11:21:05.347 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:04 smithi069 ceph-mon[29188]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-04-25T11:21:05.347 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:04 smithi069 ceph-mon[29188]: osdmap e46: 8 total, 7 up, 8 in 2024-04-25T11:21:05.347 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:04 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T11:21:05.347 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:04 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T11:21:05.347 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:04 smithi069 ceph-mon[29188]: osd.7 [v2:172.21.15.146:6816/2330294690,v1:172.21.15.146:6817/2330294690] boot 2024-04-25T11:21:05.348 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:04 smithi069 ceph-mon[29188]: osdmap e47: 8 total, 8 up, 8 in 2024-04-25T11:21:05.348 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:04 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T11:21:05.607 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi152:vg_nvme/lv_2 2024-04-25T11:21:05.852 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T11:21:06.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:05 smithi152 ceph-mon[32701]: purged_snaps scrub starts 2024-04-25T11:21:06.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:05 smithi152 ceph-mon[32701]: purged_snaps scrub ok 2024-04-25T11:21:06.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:05 smithi152 ceph-mon[32701]: Detected new or changed devices on smithi146 2024-04-25T11:21:06.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:05 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:06.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:05 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:06.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:05 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:21:06.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:05 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:21:06.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:05 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:21:06.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:05 smithi152 ceph-mon[32701]: Adjusting osd_memory_target on smithi146 to 5696M 2024-04-25T11:21:06.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:05 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:06.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:05 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:21:06.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:05 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:21:06.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:05 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:06.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:05 smithi152 ceph-mon[32701]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:21:06.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:05 smithi152 ceph-mon[32701]: osdmap e48: 8 total, 8 up, 8 in 2024-04-25T11:21:06.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:05 smithi146 ceph-mon[32821]: purged_snaps scrub starts 2024-04-25T11:21:06.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:05 smithi146 ceph-mon[32821]: purged_snaps scrub ok 2024-04-25T11:21:06.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:05 smithi146 ceph-mon[32821]: Detected new or changed devices on smithi146 2024-04-25T11:21:06.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:05 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:06.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:05 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:06.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:05 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:21:06.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:05 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:21:06.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:05 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:21:06.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:05 smithi146 ceph-mon[32821]: Adjusting osd_memory_target on smithi146 to 5696M 2024-04-25T11:21:06.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:05 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:06.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:05 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:21:06.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:05 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:21:06.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:06 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:06.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:06 smithi146 ceph-mon[32821]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:21:06.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:06 smithi146 ceph-mon[32821]: osdmap e48: 8 total, 8 up, 8 in 2024-04-25T11:21:06.446 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:05 smithi069 ceph-mon[29188]: purged_snaps scrub starts 2024-04-25T11:21:06.446 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:05 smithi069 ceph-mon[29188]: purged_snaps scrub ok 2024-04-25T11:21:06.446 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:05 smithi069 ceph-mon[29188]: Detected new or changed devices on smithi146 2024-04-25T11:21:06.446 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:05 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:06.446 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:05 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:06.446 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:05 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:21:06.446 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:05 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:21:06.446 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:05 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:21:06.447 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:05 smithi069 ceph-mon[29188]: Adjusting osd_memory_target on smithi146 to 5696M 2024-04-25T11:21:06.447 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:05 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:06.447 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:05 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:21:06.447 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:05 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:21:06.447 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:05 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:06.447 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:05 smithi069 ceph-mon[29188]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:21:06.447 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:05 smithi069 ceph-mon[29188]: osdmap e48: 8 total, 8 up, 8 in 2024-04-25T11:21:07.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:07.581+0000 7f8430fc9700 1 -- 172.21.15.152:0/141735324 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f842c102890 msgr2=0x7f842c102ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:07.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:07.581+0000 7f8430fc9700 1 --2- 172.21.15.152:0/141735324 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f842c102890 0x7f842c102ca0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f841c009990 tx=0x7f841c009ca0 comp rx=0 tx=0).stop 2024-04-25T11:21:07.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:07.581+0000 7f8430fc9700 1 -- 172.21.15.152:0/141735324 shutdown_connections 2024-04-25T11:21:07.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:07.581+0000 7f8430fc9700 1 --2- 172.21.15.152:0/141735324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f842c104360 0x7f842c10ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:07.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:07.581+0000 7f8430fc9700 1 --2- 172.21.15.152:0/141735324 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f842c102890 0x7f842c102ca0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:07.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:07.581+0000 7f8430fc9700 1 --2- 172.21.15.152:0/141735324 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f842c1039c0 0x7f842c103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:07.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:07.581+0000 7f8430fc9700 1 -- 172.21.15.152:0/141735324 >> 172.21.15.152:0/141735324 conn(0x7f842c0fded0 msgr2=0x7f842c100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:07.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:07.581+0000 7f8430fc9700 1 -- 172.21.15.152:0/141735324 shutdown_connections 2024-04-25T11:21:07.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:07.581+0000 7f8430fc9700 1 -- 172.21.15.152:0/141735324 wait complete. 2024-04-25T11:21:07.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:07.582+0000 7f8430fc9700 1 Processor -- start 2024-04-25T11:21:07.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:07.582+0000 7f8430fc9700 1 -- start start 2024-04-25T11:21:07.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:07.582+0000 7f8430fc9700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f842c102890 0x7f842c19a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:07.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:07.582+0000 7f8430fc9700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f842c1039c0 0x7f842c19ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:07.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:07.582+0000 7f8430fc9700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f842c104360 0x7f842c1a1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:07.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:07.582+0000 7f8430fc9700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f842c10ea00 con 0x7f842c102890 2024-04-25T11:21:07.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:07.582+0000 7f8430fc9700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f842c10e880 con 0x7f842c1039c0 2024-04-25T11:21:07.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:07.582+0000 7f8430fc9700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f842c10eb80 con 0x7f842c104360 2024-04-25T11:21:07.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:07.582+0000 7f842ad9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f842c104360 0x7f842c1a1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:07.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:07.582+0000 7f842ad9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f842c104360 0x7f842c1a1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59574/0 (socket says 172.21.15.152:59574) 2024-04-25T11:21:07.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:07.582+0000 7f842ad9d700 1 -- 172.21.15.152:0/814778534 learned_addr learned my addr 172.21.15.152:0/814778534 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-25T11:21:07.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:07.582+0000 7f842a59c700 1 --2- 172.21.15.152:0/814778534 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f842c102890 0x7f842c19a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:07.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:07.582+0000 7f8429d9b700 1 --2- 172.21.15.152:0/814778534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f842c1039c0 0x7f842c19ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:07.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:07.583+0000 7f842ad9d700 1 -- 172.21.15.152:0/814778534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f842c1039c0 msgr2=0x7f842c19ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:07.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:07.583+0000 7f842ad9d700 1 --2- 172.21.15.152:0/814778534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f842c1039c0 0x7f842c19ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:07.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:07.583+0000 7f842ad9d700 1 -- 172.21.15.152:0/814778534 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f842c102890 msgr2=0x7f842c19a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:07.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:07.583+0000 7f842ad9d700 1 --2- 172.21.15.152:0/814778534 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f842c102890 0x7f842c19a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:07.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:07.583+0000 7f842ad9d700 1 -- 172.21.15.152:0/814778534 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f842c1a2190 con 0x7f842c104360 2024-04-25T11:21:07.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:07.583+0000 7f842ad9d700 1 --2- 172.21.15.152:0/814778534 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f842c104360 0x7f842c1a1ba0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f842400eb80 tx=0x7f842400ee90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:07.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:07.583+0000 7f841b7fe700 1 -- 172.21.15.152:0/814778534 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84240092e0 con 0x7f842c104360 2024-04-25T11:21:07.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:07.583+0000 7f841b7fe700 1 -- 172.21.15.152:0/814778534 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f842400c5f0 con 0x7f842c104360 2024-04-25T11:21:07.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:07.583+0000 7f8430fc9700 1 -- 172.21.15.152:0/814778534 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f842c1a2420 con 0x7f842c104360 2024-04-25T11:21:07.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:07.583+0000 7f841b7fe700 1 -- 172.21.15.152:0/814778534 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8424004280 con 0x7f842c104360 2024-04-25T11:21:07.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:07.583+0000 7f8430fc9700 1 -- 172.21.15.152:0/814778534 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f842c1a2900 con 0x7f842c104360 2024-04-25T11:21:07.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:07.584+0000 7f8430fc9700 1 -- 172.21.15.152:0/814778534 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f842c051190 con 0x7f842c104360 2024-04-25T11:21:07.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:07.586+0000 7f841b7fe700 1 -- 172.21.15.152:0/814778534 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f842401b030 con 0x7f842c104360 2024-04-25T11:21:07.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:07.587+0000 7f841b7fe700 1 --2- 172.21.15.152:0/814778534 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f84140afb80 0x7f84140b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:07.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:07.587+0000 7f841b7fe700 1 -- 172.21.15.152:0/814778534 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5397+0+0 (secure 0 0 0) 0x7f84240de4d0 con 0x7f842c104360 2024-04-25T11:21:07.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:07.587+0000 7f842a59c700 1 --2- 172.21.15.152:0/814778534 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f84140afb80 0x7f84140b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:07.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:07.588+0000 7f842a59c700 1 --2- 172.21.15.152:0/814778534 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f84140afb80 0x7f84140b2090 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f841c0098e0 tx=0x7f841c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:07.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:07.591+0000 7f841b7fe700 1 -- 172.21.15.152:0/814778534 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8424022020 con 0x7f842c104360 2024-04-25T11:21:07.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:07.795+0000 7f8430fc9700 1 -- 172.21.15.152:0/814778534 --> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f842c0fff80 con 0x7f84140afb80 2024-04-25T11:21:08.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:08 smithi152 ceph-mon[32701]: osdmap e49: 8 total, 8 up, 8 in 2024-04-25T11:21:08.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:08 smithi152 ceph-mon[32701]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:21:08.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:08 smithi152 ceph-mon[32701]: from='client.24256 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:21:08.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:08 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:21:08.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:08 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:21:08.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:08 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:21:08.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:08 smithi146 ceph-mon[32821]: osdmap e49: 8 total, 8 up, 8 in 2024-04-25T11:21:08.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:08 smithi146 ceph-mon[32821]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:21:08.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:08 smithi146 ceph-mon[32821]: from='client.24256 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:21:08.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:08 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:21:08.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:08 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:21:08.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:08 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:21:08.446 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:08 smithi069 ceph-mon[29188]: osdmap e49: 8 total, 8 up, 8 in 2024-04-25T11:21:08.446 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:08 smithi069 ceph-mon[29188]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:21:08.446 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:08 smithi069 ceph-mon[29188]: from='client.24256 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:21:08.446 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:08 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:21:08.446 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:08 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:21:08.446 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:08 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:21:11.102 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:10 smithi146 ceph-mon[32821]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:21:11.102 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:10 smithi146 ceph-mon[32821]: from='client.? 172.21.15.152:0/3977883786' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1aa1060c-0c14-4abd-be56-5ebfeb78759c"}]: dispatch 2024-04-25T11:21:11.102 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:10 smithi146 ceph-mon[32821]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1aa1060c-0c14-4abd-be56-5ebfeb78759c"}]: dispatch 2024-04-25T11:21:11.102 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:10 smithi146 ceph-mon[32821]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1aa1060c-0c14-4abd-be56-5ebfeb78759c"}]': finished 2024-04-25T11:21:11.102 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:10 smithi146 ceph-mon[32821]: osdmap e50: 9 total, 8 up, 9 in 2024-04-25T11:21:11.102 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:10 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T11:21:11.196 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:10 smithi069 ceph-mon[29188]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:21:11.196 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:10 smithi069 ceph-mon[29188]: from='client.? 172.21.15.152:0/3977883786' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1aa1060c-0c14-4abd-be56-5ebfeb78759c"}]: dispatch 2024-04-25T11:21:11.196 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:10 smithi069 ceph-mon[29188]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1aa1060c-0c14-4abd-be56-5ebfeb78759c"}]: dispatch 2024-04-25T11:21:11.196 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:10 smithi069 ceph-mon[29188]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1aa1060c-0c14-4abd-be56-5ebfeb78759c"}]': finished 2024-04-25T11:21:11.196 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:10 smithi069 ceph-mon[29188]: osdmap e50: 9 total, 8 up, 9 in 2024-04-25T11:21:11.196 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:10 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T11:21:11.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:10 smithi152 ceph-mon[32701]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:21:11.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:10 smithi152 ceph-mon[32701]: from='client.? 172.21.15.152:0/3977883786' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1aa1060c-0c14-4abd-be56-5ebfeb78759c"}]: dispatch 2024-04-25T11:21:11.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:10 smithi152 ceph-mon[32701]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1aa1060c-0c14-4abd-be56-5ebfeb78759c"}]: dispatch 2024-04-25T11:21:11.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:10 smithi152 ceph-mon[32701]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1aa1060c-0c14-4abd-be56-5ebfeb78759c"}]': finished 2024-04-25T11:21:11.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:10 smithi152 ceph-mon[32701]: osdmap e50: 9 total, 8 up, 9 in 2024-04-25T11:21:11.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:10 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T11:21:12.102 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:11 smithi146 ceph-mon[32821]: from='client.? 172.21.15.152:0/2557582837' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:21:12.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:11 smithi069 ceph-mon[29188]: from='client.? 172.21.15.152:0/2557582837' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:21:12.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:11 smithi152 ceph-mon[32701]: from='client.? 172.21.15.152:0/2557582837' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:21:13.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:12 smithi146 ceph-mon[32821]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T11:21:13.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:12 smithi069 ceph-mon[29188]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T11:21:13.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:12 smithi152 ceph-mon[32701]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T11:21:14.940 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:14 smithi152 ceph-mon[32701]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-25T11:21:15.102 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:14 smithi146 ceph-mon[32821]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-25T11:21:15.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:14 smithi069 ceph-mon[29188]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-25T11:21:16.932 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:16 smithi152 ceph-mon[32701]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-04-25T11:21:16.945 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:16 smithi069 ceph-mon[29188]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-04-25T11:21:17.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:16 smithi146 ceph-mon[32821]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-04-25T11:21:19.042 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:18 smithi152 ceph-mon[32701]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T11:21:19.042 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:18 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T11:21:19.042 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:18 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:21:19.102 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:18 smithi146 ceph-mon[32821]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T11:21:19.102 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:18 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T11:21:19.102 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:18 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:21:19.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:18 smithi069 ceph-mon[29188]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T11:21:19.196 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:18 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T11:21:19.196 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:18 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:21:20.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:19 smithi146 ceph-mon[32821]: Deploying daemon osd.8 on smithi152 2024-04-25T11:21:20.111 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:19 smithi152 ceph-mon[32701]: Deploying daemon osd.8 on smithi152 2024-04-25T11:21:20.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:19 smithi069 ceph-mon[29188]: Deploying daemon osd.8 on smithi152 2024-04-25T11:21:21.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:20 smithi146 ceph-mon[32821]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T11:21:21.146 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:20 smithi152 ceph-mon[32701]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T11:21:21.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:20 smithi069 ceph-mon[29188]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T11:21:23.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:22 smithi146 ceph-mon[32821]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:21:23.179 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:22 smithi152 ceph-mon[32701]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:21:23.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:22 smithi069 ceph-mon[29188]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:21:25.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:24 smithi146 ceph-mon[32821]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:21:25.102 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:24 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:25.102 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:24 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:25.102 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:24 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:21:25.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:24 smithi069 ceph-mon[29188]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:21:25.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:24 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:25.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:24 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:25.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:24 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:21:25.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:24 smithi152 ceph-mon[32701]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:21:25.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:24 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:25.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:24 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:25.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:24 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:21:27.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:26 smithi146 ceph-mon[32821]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:21:27.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:26 smithi069 ceph-mon[29188]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:21:27.207 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:26 smithi152 ceph-mon[32701]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:21:27.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:27.909+0000 7f841b7fe700 1 -- 172.21.15.152:0/814778534 <== mgr.14152 v2:172.21.15.69:6800/4220065409 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f842c0fff80 con 0x7f84140afb80 2024-04-25T11:21:27.907 INFO:teuthology.orchestra.run.smithi152.stdout:Created osd(s) 8 on host 'smithi152' 2024-04-25T11:21:27.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:27.912+0000 7f8430fc9700 1 -- 172.21.15.152:0/814778534 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f84140afb80 msgr2=0x7f84140b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:27.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:27.912+0000 7f8430fc9700 1 --2- 172.21.15.152:0/814778534 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f84140afb80 0x7f84140b2090 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f841c0098e0 tx=0x7f841c01a040 comp rx=0 tx=0).stop 2024-04-25T11:21:27.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:27.912+0000 7f8430fc9700 1 -- 172.21.15.152:0/814778534 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f842c104360 msgr2=0x7f842c1a1ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:27.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:27.912+0000 7f8430fc9700 1 --2- 172.21.15.152:0/814778534 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f842c104360 0x7f842c1a1ba0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f842400eb80 tx=0x7f842400ee90 comp rx=0 tx=0).stop 2024-04-25T11:21:27.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:27.912+0000 7f8430fc9700 1 -- 172.21.15.152:0/814778534 shutdown_connections 2024-04-25T11:21:27.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:27.912+0000 7f8430fc9700 1 --2- 172.21.15.152:0/814778534 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f84140afb80 0x7f84140b2090 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:27.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:27.912+0000 7f8430fc9700 1 --2- 172.21.15.152:0/814778534 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f842c104360 0x7f842c1a1ba0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:27.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:27.912+0000 7f8430fc9700 1 --2- 172.21.15.152:0/814778534 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f842c102890 0x7f842c19a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:27.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:27.912+0000 7f8430fc9700 1 --2- 172.21.15.152:0/814778534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f842c1039c0 0x7f842c19ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:27.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:27.912+0000 7f8430fc9700 1 -- 172.21.15.152:0/814778534 >> 172.21.15.152:0/814778534 conn(0x7f842c0fded0 msgr2=0x7f842c0ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:27.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:27.912+0000 7f8430fc9700 1 -- 172.21.15.152:0/814778534 shutdown_connections 2024-04-25T11:21:27.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:21:27.912+0000 7f8430fc9700 1 -- 172.21.15.152:0/814778534 wait complete. 2024-04-25T11:21:28.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:28 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:28.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:28 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:28.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:28 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:21:28.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:28 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:21:28.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:28 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:28.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:28 smithi152 ceph-mon[32701]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:21:28.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:28 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:28.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:28 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:28.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:28 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:21:28.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:28 smithi152 ceph-mon[32701]: from='osd.8 [v2:172.21.15.152:6816/950669381,v1:172.21.15.152:6817/950669381]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T11:21:28.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:28 smithi152 ceph-mon[32701]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T11:21:28.351 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:28 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:28.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:28 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:28.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:28 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:21:28.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:28 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:21:28.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:28 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:28.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:28 smithi146 ceph-mon[32821]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:21:28.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:28 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:28.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:28 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:28.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:28 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:21:28.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:28 smithi146 ceph-mon[32821]: from='osd.8 [v2:172.21.15.152:6816/950669381,v1:172.21.15.152:6817/950669381]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T11:21:28.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:28 smithi146 ceph-mon[32821]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T11:21:28.445 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:28 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:28.445 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:28 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:28.446 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:28 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:21:28.446 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:28 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:21:28.446 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:28 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:28.446 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:28 smithi069 ceph-mon[29188]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:21:28.446 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:28 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:28.446 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:28 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:28.446 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:28 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:21:28.446 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:28 smithi069 ceph-mon[29188]: from='osd.8 [v2:172.21.15.152:6816/950669381,v1:172.21.15.152:6817/950669381]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T11:21:28.446 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:28 smithi069 ceph-mon[29188]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T11:21:28.635 DEBUG:teuthology.orchestra.run.smithi152:osd.8> sudo journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@osd.8.service 2024-04-25T11:21:28.637 INFO:tasks.cephadm:Deploying osd.9 on smithi069 with /dev/vg_nvme/lv_1... 2024-04-25T11:21:28.637 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T11:21:28.883 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:21:30.138 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:29 smithi152 ceph-mon[32701]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T11:21:30.139 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:29 smithi152 ceph-mon[32701]: osdmap e51: 9 total, 8 up, 9 in 2024-04-25T11:21:30.139 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:29 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T11:21:30.139 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:29 smithi152 ceph-mon[32701]: from='osd.8 [v2:172.21.15.152:6816/950669381,v1:172.21.15.152:6817/950669381]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T11:21:30.139 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:29 smithi152 ceph-mon[32701]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T11:21:30.139 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:29 smithi152 ceph-mon[32701]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:21:30.147 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:29 smithi069 ceph-mon[29188]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T11:21:30.147 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:29 smithi069 ceph-mon[29188]: osdmap e51: 9 total, 8 up, 9 in 2024-04-25T11:21:30.147 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:29 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T11:21:30.147 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:29 smithi069 ceph-mon[29188]: from='osd.8 [v2:172.21.15.152:6816/950669381,v1:172.21.15.152:6817/950669381]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T11:21:30.147 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:29 smithi069 ceph-mon[29188]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T11:21:30.147 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:29 smithi069 ceph-mon[29188]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:21:30.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:29 smithi146 ceph-mon[32821]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T11:21:30.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:29 smithi146 ceph-mon[32821]: osdmap e51: 9 total, 8 up, 9 in 2024-04-25T11:21:30.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:29 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T11:21:30.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:29 smithi146 ceph-mon[32821]: from='osd.8 [v2:172.21.15.152:6816/950669381,v1:172.21.15.152:6817/950669381]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T11:21:30.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:29 smithi146 ceph-mon[32821]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T11:21:30.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:29 smithi146 ceph-mon[32821]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:21:30.733 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:21:30 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:21:30.623+0000 7f29d2256700 -1 osd.8 0 waiting for initial osdmap 2024-04-25T11:21:30.733 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:21:30 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:21:30.634+0000 7f29ca003700 -1 osd.8 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:21:31.062 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T11:21:31.062 INFO:teuthology.orchestra.run.smithi069.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T11:21:31.062 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10+0 records in 2024-04-25T11:21:31.063 INFO:teuthology.orchestra.run.smithi069.stderr:10+0 records out 2024-04-25T11:21:31.063 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0118211 s, 887 MB/s 2024-04-25T11:21:31.063 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping successful for: 2024-04-25T11:21:31.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:30 smithi069 ceph-mon[29188]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-04-25T11:21:31.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:30 smithi069 ceph-mon[29188]: osdmap e52: 9 total, 8 up, 9 in 2024-04-25T11:21:31.196 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:30 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T11:21:31.196 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:30 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T11:21:31.196 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:30 smithi069 ceph-mon[29188]: from='osd.8 ' entity='osd.8' 2024-04-25T11:21:31.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:30 smithi152 ceph-mon[32701]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-04-25T11:21:31.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:30 smithi152 ceph-mon[32701]: osdmap e52: 9 total, 8 up, 9 in 2024-04-25T11:21:31.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:30 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T11:21:31.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:30 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T11:21:31.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:30 smithi152 ceph-mon[32701]: from='osd.8 ' entity='osd.8' 2024-04-25T11:21:31.351 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:30 smithi146 ceph-mon[32821]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-04-25T11:21:31.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:30 smithi146 ceph-mon[32821]: osdmap e52: 9 total, 8 up, 9 in 2024-04-25T11:21:31.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:30 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T11:21:31.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:30 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T11:21:31.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:30 smithi146 ceph-mon[32821]: from='osd.8 ' entity='osd.8' 2024-04-25T11:21:31.776 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi069:vg_nvme/lv_1 2024-04-25T11:21:32.033 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:21:32.156 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:31 smithi069 ceph-mon[29188]: purged_snaps scrub starts 2024-04-25T11:21:32.156 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:31 smithi069 ceph-mon[29188]: purged_snaps scrub ok 2024-04-25T11:21:32.156 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:31 smithi069 ceph-mon[29188]: osd.8 [v2:172.21.15.152:6816/950669381,v1:172.21.15.152:6817/950669381] boot 2024-04-25T11:21:32.156 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:31 smithi069 ceph-mon[29188]: osdmap e53: 9 total, 9 up, 9 in 2024-04-25T11:21:32.157 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:31 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T11:21:32.157 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:31 smithi069 ceph-mon[29188]: pgmap v193: 1 pgs: 1 peering; 577 KiB data, 471 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:21:32.157 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:31 smithi069 ceph-mon[29188]: Detected new or changed devices on smithi152 2024-04-25T11:21:32.157 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:31 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:32.157 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:31 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:32.157 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:31 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:21:32.157 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:31 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:21:32.157 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:31 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:21:32.157 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:31 smithi069 ceph-mon[29188]: Adjusting osd_memory_target on smithi152 to 5696M 2024-04-25T11:21:32.157 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:31 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:32.157 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:31 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:21:32.157 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:31 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:21:32.157 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:31 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:32.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:31 smithi152 ceph-mon[32701]: purged_snaps scrub starts 2024-04-25T11:21:32.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:31 smithi152 ceph-mon[32701]: purged_snaps scrub ok 2024-04-25T11:21:32.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:31 smithi152 ceph-mon[32701]: osd.8 [v2:172.21.15.152:6816/950669381,v1:172.21.15.152:6817/950669381] boot 2024-04-25T11:21:32.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:31 smithi152 ceph-mon[32701]: osdmap e53: 9 total, 9 up, 9 in 2024-04-25T11:21:32.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:31 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T11:21:32.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:31 smithi152 ceph-mon[32701]: pgmap v193: 1 pgs: 1 peering; 577 KiB data, 471 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:21:32.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:31 smithi152 ceph-mon[32701]: Detected new or changed devices on smithi152 2024-04-25T11:21:32.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:31 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:32.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:31 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:32.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:31 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:21:32.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:31 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:21:32.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:31 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:21:32.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:31 smithi152 ceph-mon[32701]: Adjusting osd_memory_target on smithi152 to 5696M 2024-04-25T11:21:32.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:31 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:32.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:31 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:21:32.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:31 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:21:32.236 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:31 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:32.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:31 smithi146 ceph-mon[32821]: purged_snaps scrub starts 2024-04-25T11:21:32.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:31 smithi146 ceph-mon[32821]: purged_snaps scrub ok 2024-04-25T11:21:32.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:31 smithi146 ceph-mon[32821]: osd.8 [v2:172.21.15.152:6816/950669381,v1:172.21.15.152:6817/950669381] boot 2024-04-25T11:21:32.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:31 smithi146 ceph-mon[32821]: osdmap e53: 9 total, 9 up, 9 in 2024-04-25T11:21:32.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:31 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T11:21:32.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:31 smithi146 ceph-mon[32821]: pgmap v193: 1 pgs: 1 peering; 577 KiB data, 471 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:21:32.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:31 smithi146 ceph-mon[32821]: Detected new or changed devices on smithi152 2024-04-25T11:21:32.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:31 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:32.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:31 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:32.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:31 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:21:32.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:31 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:21:32.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:31 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:21:32.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:31 smithi146 ceph-mon[32821]: Adjusting osd_memory_target on smithi152 to 5696M 2024-04-25T11:21:32.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:31 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:32.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:31 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:21:32.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:31 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:21:32.353 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:31 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:33.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:32 smithi069 ceph-mon[29188]: osdmap e54: 9 total, 9 up, 9 in 2024-04-25T11:21:33.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:32 smithi152 ceph-mon[32701]: osdmap e54: 9 total, 9 up, 9 in 2024-04-25T11:21:33.351 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:32 smithi146 ceph-mon[32821]: osdmap e54: 9 total, 9 up, 9 in 2024-04-25T11:21:33.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:33.867+0000 7f064148e700 1 -- 172.21.15.69:0/1535608460 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f063c104220 msgr2=0x7f063c104630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:33.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:33.867+0000 7f064148e700 1 --2- 172.21.15.69:0/1535608460 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f063c104220 0x7f063c104630 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0624009990 tx=0x7f0624009ca0 comp rx=0 tx=0).stop 2024-04-25T11:21:33.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:33.867+0000 7f064148e700 1 -- 172.21.15.69:0/1535608460 shutdown_connections 2024-04-25T11:21:33.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:33.867+0000 7f064148e700 1 --2- 172.21.15.69:0/1535608460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f063c104bc0 0x7f063c10b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:33.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:33.867+0000 7f064148e700 1 --2- 172.21.15.69:0/1535608460 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f063c1030f0 0x7f063c103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:33.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:33.867+0000 7f064148e700 1 --2- 172.21.15.69:0/1535608460 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f063c104220 0x7f063c104630 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:33.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:33.867+0000 7f064148e700 1 -- 172.21.15.69:0/1535608460 >> 172.21.15.69:0/1535608460 conn(0x7f063c0fe730 msgr2=0x7f063c100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:33.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:33.867+0000 7f064148e700 1 -- 172.21.15.69:0/1535608460 shutdown_connections 2024-04-25T11:21:33.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:33.867+0000 7f064148e700 1 -- 172.21.15.69:0/1535608460 wait complete. 2024-04-25T11:21:33.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:33.868+0000 7f064148e700 1 Processor -- start 2024-04-25T11:21:33.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:33.868+0000 7f064148e700 1 -- start start 2024-04-25T11:21:33.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:33.868+0000 7f064148e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f063c1030f0 0x7f063c076340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:33.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:33.868+0000 7f064148e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f063c104220 0x7f063c19f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:33.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:33.868+0000 7f064148e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f063c104bc0 0x7f063c1a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:33.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:33.868+0000 7f064148e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f063c10ea30 con 0x7f063c1030f0 2024-04-25T11:21:33.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:33.868+0000 7f064148e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f063c10e8b0 con 0x7f063c104bc0 2024-04-25T11:21:33.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:33.868+0000 7f064148e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f063c10ebb0 con 0x7f063c104220 2024-04-25T11:21:33.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:33.868+0000 7f063affd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f063c1030f0 0x7f063c076340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:33.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:33.868+0000 7f063affd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f063c1030f0 0x7f063c076340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37474/0 (socket says 172.21.15.69:37474) 2024-04-25T11:21:33.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:33.868+0000 7f063affd700 1 -- 172.21.15.69:0/1824585781 learned_addr learned my addr 172.21.15.69:0/1824585781 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:21:33.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:33.868+0000 7f06327fc700 1 --2- 172.21.15.69:0/1824585781 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f063c104220 0x7f063c19f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:33.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:33.869+0000 7f063b7fe700 1 --2- 172.21.15.69:0/1824585781 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f063c104bc0 0x7f063c1a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:33.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:33.869+0000 7f063affd700 1 -- 172.21.15.69:0/1824585781 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f063c104220 msgr2=0x7f063c19f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:33.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:33.869+0000 7f063affd700 1 --2- 172.21.15.69:0/1824585781 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f063c104220 0x7f063c19f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:33.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:33.869+0000 7f063affd700 1 -- 172.21.15.69:0/1824585781 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f063c104bc0 msgr2=0x7f063c1a1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:33.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:33.869+0000 7f063affd700 1 --2- 172.21.15.69:0/1824585781 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f063c104bc0 0x7f063c1a1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:33.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:33.869+0000 7f063affd700 1 -- 172.21.15.69:0/1824585781 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f063c1a21d0 con 0x7f063c1030f0 2024-04-25T11:21:33.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:33.869+0000 7f063affd700 1 --2- 172.21.15.69:0/1824585781 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f063c1030f0 0x7f063c076340 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f062c00c970 tx=0x7f062c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:33.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:33.869+0000 7f0638ff9700 1 -- 172.21.15.69:0/1824585781 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062c007b00 con 0x7f063c1030f0 2024-04-25T11:21:33.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:33.869+0000 7f064148e700 1 -- 172.21.15.69:0/1824585781 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f063c1a24c0 con 0x7f063c1030f0 2024-04-25T11:21:33.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:33.869+0000 7f0638ff9700 1 -- 172.21.15.69:0/1824585781 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f062c007ca0 con 0x7f063c1030f0 2024-04-25T11:21:33.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:33.869+0000 7f0638ff9700 1 -- 172.21.15.69:0/1824585781 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062c018660 con 0x7f063c1030f0 2024-04-25T11:21:33.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:33.869+0000 7f064148e700 1 -- 172.21.15.69:0/1824585781 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f063c1a2a50 con 0x7f063c1030f0 2024-04-25T11:21:33.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:33.870+0000 7f064148e700 1 -- 172.21.15.69:0/1824585781 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f063c051190 con 0x7f063c1030f0 2024-04-25T11:21:33.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:33.872+0000 7f0638ff9700 1 -- 172.21.15.69:0/1824585781 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f062c00f450 con 0x7f063c1030f0 2024-04-25T11:21:33.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:33.872+0000 7f0638ff9700 1 --2- 172.21.15.69:0/1824585781 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f06280afb80 0x7f06280b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:33.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:33.872+0000 7f06327fc700 1 --2- 172.21.15.69:0/1824585781 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f06280afb80 0x7f06280b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:33.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:33.873+0000 7f0638ff9700 1 -- 172.21.15.69:0/1824585781 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5829+0+0 (secure 0 0 0) 0x7f062c0de2e0 con 0x7f063c1030f0 2024-04-25T11:21:33.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:33.873+0000 7f06327fc700 1 --2- 172.21.15.69:0/1824585781 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f06280afb80 0x7f06280b2090 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f062400b5c0 tx=0x7f062401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:33.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:33.876+0000 7f0638ff9700 1 -- 172.21.15.69:0/1824585781 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f062c052ff0 con 0x7f063c1030f0 2024-04-25T11:21:34.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:34.084+0000 7f064148e700 1 -- 172.21.15.69:0/1824585781 --> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f063c063ad0 con 0x7f06280afb80 2024-04-25T11:21:34.082 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:33 smithi069 ceph-mon[29188]: osdmap e55: 9 total, 9 up, 9 in 2024-04-25T11:21:34.082 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:33 smithi069 ceph-mon[29188]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 471 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:21:34.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:33 smithi152 ceph-mon[32701]: osdmap e55: 9 total, 9 up, 9 in 2024-04-25T11:21:34.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:33 smithi152 ceph-mon[32701]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 471 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:21:34.351 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:33 smithi146 ceph-mon[32821]: osdmap e55: 9 total, 9 up, 9 in 2024-04-25T11:21:34.351 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:33 smithi146 ceph-mon[32821]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 471 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:21:34.945 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:34 smithi069 ceph-mon[29188]: from='client.14403 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:21:34.945 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:34 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:21:34.945 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:34 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:21:34.946 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:34 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:21:35.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:34 smithi152 ceph-mon[32701]: from='client.14403 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:21:35.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:34 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:21:35.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:34 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:21:35.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:34 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:21:35.351 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:34 smithi146 ceph-mon[32821]: from='client.14403 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:21:35.351 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:34 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:21:35.351 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:34 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:21:35.351 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:34 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:21:36.015 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:35 smithi069 ceph-mon[29188]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 472 MiB used, 804 GiB / 805 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-04-25T11:21:36.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:35 smithi152 ceph-mon[32701]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 472 MiB used, 804 GiB / 805 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-04-25T11:21:36.351 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:35 smithi146 ceph-mon[32821]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 472 MiB used, 804 GiB / 805 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-04-25T11:21:37.434 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:37 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/3155699766' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f4bf800-4090-4138-9bb3-d001f13f450c"}]: dispatch 2024-04-25T11:21:37.434 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:37 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/3155699766' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3f4bf800-4090-4138-9bb3-d001f13f450c"}]': finished 2024-04-25T11:21:37.434 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:37 smithi069 ceph-mon[29188]: osdmap e56: 10 total, 9 up, 10 in 2024-04-25T11:21:37.434 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:37 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T11:21:37.482 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:37 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/3155699766' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f4bf800-4090-4138-9bb3-d001f13f450c"}]: dispatch 2024-04-25T11:21:37.483 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:37 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/3155699766' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3f4bf800-4090-4138-9bb3-d001f13f450c"}]': finished 2024-04-25T11:21:37.483 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:37 smithi152 ceph-mon[32701]: osdmap e56: 10 total, 9 up, 10 in 2024-04-25T11:21:37.483 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:37 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T11:21:37.601 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:37 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/3155699766' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f4bf800-4090-4138-9bb3-d001f13f450c"}]: dispatch 2024-04-25T11:21:37.602 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:37 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/3155699766' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3f4bf800-4090-4138-9bb3-d001f13f450c"}]': finished 2024-04-25T11:21:37.602 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:37 smithi146 ceph-mon[32821]: osdmap e56: 10 total, 9 up, 10 in 2024-04-25T11:21:37.602 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:37 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T11:21:38.445 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:38 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/3022385338' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:21:38.445 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:38 smithi069 ceph-mon[29188]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T11:21:38.483 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:38 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/3022385338' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:21:38.483 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:38 smithi152 ceph-mon[32701]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T11:21:38.601 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:38 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/3022385338' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:21:38.601 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:38 smithi146 ceph-mon[32821]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T11:21:41.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:40 smithi146 ceph-mon[32821]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-25T11:21:41.150 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:40 smithi069 ceph-mon[29188]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-25T11:21:41.186 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:40 smithi152 ceph-mon[32701]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-25T11:21:43.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:42 smithi146 ceph-mon[32821]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-25T11:21:43.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:42 smithi069 ceph-mon[29188]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-25T11:21:43.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:42 smithi152 ceph-mon[32701]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-25T11:21:45.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:44 smithi146 ceph-mon[32821]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T11:21:45.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:44 smithi069 ceph-mon[29188]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T11:21:45.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:44 smithi152 ceph-mon[32701]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T11:21:46.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:45 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T11:21:46.102 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:45 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:21:46.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:45 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T11:21:46.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:45 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:21:46.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:45 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T11:21:46.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:45 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:21:46.945 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:46 smithi069 ceph-mon[29188]: Deploying daemon osd.9 on smithi069 2024-04-25T11:21:46.945 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:46 smithi069 ceph-mon[29188]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:21:47.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:46 smithi146 ceph-mon[32821]: Deploying daemon osd.9 on smithi069 2024-04-25T11:21:47.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:46 smithi146 ceph-mon[32821]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:21:47.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:46 smithi152 ceph-mon[32701]: Deploying daemon osd.9 on smithi069 2024-04-25T11:21:47.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:46 smithi152 ceph-mon[32701]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:21:49.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:48 smithi146 ceph-mon[32821]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:21:49.135 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:48 smithi069 ceph-mon[29188]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:21:49.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:48 smithi152 ceph-mon[32701]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:21:51.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:50 smithi146 ceph-mon[32821]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:21:51.177 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:50 smithi069 ceph-mon[29188]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:21:51.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:50 smithi152 ceph-mon[32701]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:21:52.445 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:52 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:52.445 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:52 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:21:52.445 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:52 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:52.445 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:52 smithi069 ceph-mon[29188]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:21:52.482 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:52 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:52.483 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:52 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:21:52.483 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:52 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:52.483 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:52 smithi152 ceph-mon[32701]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:21:52.601 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:52 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:52.601 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:52 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:21:52.601 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:52 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:52.601 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:52 smithi146 ceph-mon[32821]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:21:55.054 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:54 smithi069 ceph-mon[29188]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:21:55.054 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:54 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:55.054 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:54 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:55.055 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:54 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:21:55.055 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:54 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:21:55.055 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:54 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:55.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:54 smithi146 ceph-mon[32821]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:21:55.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:54 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:55.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:54 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:55.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:54 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:21:55.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:54 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:21:55.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:54 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:55.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:54 smithi152 ceph-mon[32701]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:21:55.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:54 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:55.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:54 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:55.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:54 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:21:55.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:54 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:21:55.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:54 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:55.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:55.302+0000 7f0638ff9700 1 -- 172.21.15.69:0/1824585781 <== mgr.14152 v2:172.21.15.69:6800/4220065409 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f063c063ad0 con 0x7f06280afb80 2024-04-25T11:21:55.298 INFO:teuthology.orchestra.run.smithi069.stdout:Created osd(s) 9 on host 'smithi069' 2024-04-25T11:21:55.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:55.305+0000 7f064148e700 1 -- 172.21.15.69:0/1824585781 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f06280afb80 msgr2=0x7f06280b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:55.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:55.305+0000 7f064148e700 1 --2- 172.21.15.69:0/1824585781 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f06280afb80 0x7f06280b2090 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f062400b5c0 tx=0x7f062401a040 comp rx=0 tx=0).stop 2024-04-25T11:21:55.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:55.305+0000 7f064148e700 1 -- 172.21.15.69:0/1824585781 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f063c1030f0 msgr2=0x7f063c076340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:55.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:55.305+0000 7f064148e700 1 --2- 172.21.15.69:0/1824585781 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f063c1030f0 0x7f063c076340 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f062c00c970 tx=0x7f062c00cc80 comp rx=0 tx=0).stop 2024-04-25T11:21:55.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:55.305+0000 7f064148e700 1 -- 172.21.15.69:0/1824585781 shutdown_connections 2024-04-25T11:21:55.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:55.305+0000 7f064148e700 1 --2- 172.21.15.69:0/1824585781 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f06280afb80 0x7f06280b2090 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:55.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:55.305+0000 7f064148e700 1 --2- 172.21.15.69:0/1824585781 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f063c104bc0 0x7f063c1a1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:55.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:55.305+0000 7f064148e700 1 --2- 172.21.15.69:0/1824585781 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f063c1030f0 0x7f063c076340 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:55.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:55.305+0000 7f064148e700 1 --2- 172.21.15.69:0/1824585781 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f063c104220 0x7f063c19f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:55.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:55.305+0000 7f064148e700 1 -- 172.21.15.69:0/1824585781 >> 172.21.15.69:0/1824585781 conn(0x7f063c0fe730 msgr2=0x7f063c107450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:55.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:55.305+0000 7f064148e700 1 -- 172.21.15.69:0/1824585781 shutdown_connections 2024-04-25T11:21:55.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:21:55.305+0000 7f064148e700 1 -- 172.21.15.69:0/1824585781 wait complete. 2024-04-25T11:21:55.945 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:55 smithi069 ceph-mon[29188]: from='osd.9 [v2:172.21.15.69:6826/2414186758,v1:172.21.15.69:6827/2414186758]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T11:21:55.945 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:55 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:55.945 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:55 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:55.945 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:55 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:21:56.090 DEBUG:teuthology.orchestra.run.smithi069:osd.9> sudo journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@osd.9.service 2024-04-25T11:21:56.092 INFO:tasks.cephadm:Deploying osd.10 on smithi146 with /dev/vg_nvme/lv_1... 2024-04-25T11:21:56.092 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T11:21:56.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:55 smithi146 ceph-mon[32821]: from='osd.9 [v2:172.21.15.69:6826/2414186758,v1:172.21.15.69:6827/2414186758]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T11:21:56.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:55 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:56.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:55 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:56.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:55 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:21:56.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:55 smithi152 ceph-mon[32701]: from='osd.9 [v2:172.21.15.69:6826/2414186758,v1:172.21.15.69:6827/2414186758]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T11:21:56.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:55 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:56.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:55 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:21:56.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:55 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:21:56.341 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T11:21:56.945 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:56 smithi069 ceph-mon[29188]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:21:56.945 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:56 smithi069 ceph-mon[29188]: from='osd.9 [v2:172.21.15.69:6826/2414186758,v1:172.21.15.69:6827/2414186758]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T11:21:56.945 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:56 smithi069 ceph-mon[29188]: osdmap e57: 10 total, 9 up, 10 in 2024-04-25T11:21:56.945 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:56 smithi069 ceph-mon[29188]: from='osd.9 [v2:172.21.15.69:6826/2414186758,v1:172.21.15.69:6827/2414186758]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-25T11:21:56.945 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:56 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T11:21:57.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:56 smithi146 ceph-mon[32821]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:21:57.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:56 smithi146 ceph-mon[32821]: from='osd.9 [v2:172.21.15.69:6826/2414186758,v1:172.21.15.69:6827/2414186758]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T11:21:57.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:56 smithi146 ceph-mon[32821]: osdmap e57: 10 total, 9 up, 10 in 2024-04-25T11:21:57.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:56 smithi146 ceph-mon[32821]: from='osd.9 [v2:172.21.15.69:6826/2414186758,v1:172.21.15.69:6827/2414186758]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-25T11:21:57.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:56 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T11:21:57.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:56 smithi152 ceph-mon[32701]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:21:57.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:56 smithi152 ceph-mon[32701]: from='osd.9 [v2:172.21.15.69:6826/2414186758,v1:172.21.15.69:6827/2414186758]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T11:21:57.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:56 smithi152 ceph-mon[32701]: osdmap e57: 10 total, 9 up, 10 in 2024-04-25T11:21:57.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:56 smithi152 ceph-mon[32701]: from='osd.9 [v2:172.21.15.69:6826/2414186758,v1:172.21.15.69:6827/2414186758]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-25T11:21:57.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:56 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T11:21:57.615 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:21:57 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:21:57.541+0000 7f4f3c836700 -1 osd.9 0 waiting for initial osdmap 2024-04-25T11:21:57.616 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:21:57 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:21:57.551+0000 7f4f33de2700 -1 osd.9 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:21:57.945 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:57 smithi069 ceph-mon[29188]: from='osd.9 [v2:172.21.15.69:6826/2414186758,v1:172.21.15.69:6827/2414186758]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-04-25T11:21:57.945 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:57 smithi069 ceph-mon[29188]: osdmap e58: 10 total, 9 up, 10 in 2024-04-25T11:21:57.945 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:57 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T11:21:57.945 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:57 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T11:21:57.945 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:57 smithi069 ceph-mon[29188]: from='osd.9 [v2:172.21.15.69:6826/2414186758,v1:172.21.15.69:6827/2414186758]' entity='osd.9' 2024-04-25T11:21:57.945 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:57 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T11:21:58.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:57 smithi146 ceph-mon[32821]: from='osd.9 [v2:172.21.15.69:6826/2414186758,v1:172.21.15.69:6827/2414186758]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-04-25T11:21:58.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:57 smithi146 ceph-mon[32821]: osdmap e58: 10 total, 9 up, 10 in 2024-04-25T11:21:58.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:57 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T11:21:58.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:57 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T11:21:58.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:57 smithi146 ceph-mon[32821]: from='osd.9 [v2:172.21.15.69:6826/2414186758,v1:172.21.15.69:6827/2414186758]' entity='osd.9' 2024-04-25T11:21:58.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:57 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T11:21:58.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:57 smithi152 ceph-mon[32701]: from='osd.9 [v2:172.21.15.69:6826/2414186758,v1:172.21.15.69:6827/2414186758]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-04-25T11:21:58.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:57 smithi152 ceph-mon[32701]: osdmap e58: 10 total, 9 up, 10 in 2024-04-25T11:21:58.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:57 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T11:21:58.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:57 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T11:21:58.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:57 smithi152 ceph-mon[32701]: from='osd.9 [v2:172.21.15.69:6826/2414186758,v1:172.21.15.69:6827/2414186758]' entity='osd.9' 2024-04-25T11:21:58.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:57 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T11:21:58.472 INFO:teuthology.orchestra.run.smithi146.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T11:21:58.473 INFO:teuthology.orchestra.run.smithi146.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T11:21:58.473 INFO:teuthology.orchestra.run.smithi146.stderr: stderr: 10+0 records in 2024-04-25T11:21:58.473 INFO:teuthology.orchestra.run.smithi146.stderr:10+0 records out 2024-04-25T11:21:58.473 INFO:teuthology.orchestra.run.smithi146.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0117888 s, 889 MB/s 2024-04-25T11:21:58.473 INFO:teuthology.orchestra.run.smithi146.stderr:--> Zapping successful for: 2024-04-25T11:21:59.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:58 smithi146 ceph-mon[32821]: purged_snaps scrub starts 2024-04-25T11:21:59.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:58 smithi146 ceph-mon[32821]: purged_snaps scrub ok 2024-04-25T11:21:59.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:58 smithi146 ceph-mon[32821]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:21:59.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:58 smithi146 ceph-mon[32821]: osd.9 [v2:172.21.15.69:6826/2414186758,v1:172.21.15.69:6827/2414186758] boot 2024-04-25T11:21:59.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:58 smithi146 ceph-mon[32821]: osdmap e59: 10 total, 10 up, 10 in 2024-04-25T11:21:59.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:21:58 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T11:21:59.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:58 smithi069 ceph-mon[29188]: purged_snaps scrub starts 2024-04-25T11:21:59.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:58 smithi069 ceph-mon[29188]: purged_snaps scrub ok 2024-04-25T11:21:59.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:58 smithi069 ceph-mon[29188]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:21:59.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:58 smithi069 ceph-mon[29188]: osd.9 [v2:172.21.15.69:6826/2414186758,v1:172.21.15.69:6827/2414186758] boot 2024-04-25T11:21:59.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:58 smithi069 ceph-mon[29188]: osdmap e59: 10 total, 10 up, 10 in 2024-04-25T11:21:59.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:21:58 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T11:21:59.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:58 smithi152 ceph-mon[32701]: purged_snaps scrub starts 2024-04-25T11:21:59.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:58 smithi152 ceph-mon[32701]: purged_snaps scrub ok 2024-04-25T11:21:59.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:58 smithi152 ceph-mon[32701]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:21:59.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:58 smithi152 ceph-mon[32701]: osd.9 [v2:172.21.15.69:6826/2414186758,v1:172.21.15.69:6827/2414186758] boot 2024-04-25T11:21:59.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:58 smithi152 ceph-mon[32701]: osdmap e59: 10 total, 10 up, 10 in 2024-04-25T11:21:59.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:21:58 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T11:21:59.283 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi146:vg_nvme/lv_1 2024-04-25T11:21:59.565 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T11:22:00.601 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:00 smithi146 ceph-mon[32821]: Detected new or changed devices on smithi069 2024-04-25T11:22:00.601 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:00 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:00.601 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:00 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:00.601 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:00 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:22:00.601 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:00 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:22:00.601 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:00 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:22:00.601 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:00 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:22:00.602 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:00 smithi146 ceph-mon[32821]: Adjusting osd_memory_target on smithi069 to 4272M 2024-04-25T11:22:00.602 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:00 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:00.602 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:00 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:22:00.602 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:00 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:22:00.602 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:00 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:00.602 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:00 smithi146 ceph-mon[32821]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:22:00.602 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:00 smithi146 ceph-mon[32821]: osdmap e60: 10 total, 10 up, 10 in 2024-04-25T11:22:00.695 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:00 smithi069 ceph-mon[29188]: Detected new or changed devices on smithi069 2024-04-25T11:22:00.695 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:00 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:00.695 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:00 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:00.695 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:00 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:22:00.695 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:00 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:22:00.695 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:00 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:22:00.695 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:00 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:22:00.695 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:00 smithi069 ceph-mon[29188]: Adjusting osd_memory_target on smithi069 to 4272M 2024-04-25T11:22:00.696 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:00 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:00.696 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:00 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:22:00.696 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:00 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:22:00.696 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:00 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:00.696 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:00 smithi069 ceph-mon[29188]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:22:00.696 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:00 smithi069 ceph-mon[29188]: osdmap e60: 10 total, 10 up, 10 in 2024-04-25T11:22:00.733 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:00 smithi152 ceph-mon[32701]: Detected new or changed devices on smithi069 2024-04-25T11:22:00.733 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:00 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:00.733 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:00 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:00.733 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:00 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:22:00.733 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:00 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:22:00.733 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:00 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:22:00.733 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:00 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:22:00.733 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:00 smithi152 ceph-mon[32701]: Adjusting osd_memory_target on smithi069 to 4272M 2024-04-25T11:22:00.733 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:00 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:00.733 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:00 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:22:00.733 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:00 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:22:00.734 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:00 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:00.734 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:00 smithi152 ceph-mon[32701]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:22:00.734 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:00 smithi152 ceph-mon[32701]: osdmap e60: 10 total, 10 up, 10 in 2024-04-25T11:22:01.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:01.276+0000 7f49140cf700 1 -- 172.21.15.146:0/224124843 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f490c109e90 msgr2=0x7f490c10c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:01.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:01.276+0000 7f49140cf700 1 --2- 172.21.15.146:0/224124843 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f490c109e90 0x7f490c10c3a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f490400b0a0 tx=0x7f490400b3b0 comp rx=0 tx=0).stop 2024-04-25T11:22:01.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:01.276+0000 7f49140cf700 1 -- 172.21.15.146:0/224124843 shutdown_connections 2024-04-25T11:22:01.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:01.276+0000 7f49140cf700 1 --2- 172.21.15.146:0/224124843 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f490c109e90 0x7f490c10c3a0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:01.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:01.276+0000 7f49140cf700 1 --2- 172.21.15.146:0/224124843 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f490c076560 0x7f490c106d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:01.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:01.276+0000 7f49140cf700 1 --2- 172.21.15.146:0/224124843 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f490c107340 0x7f490c109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:01.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:01.276+0000 7f49140cf700 1 -- 172.21.15.146:0/224124843 >> 172.21.15.146:0/224124843 conn(0x7f490c0fe740 msgr2=0x7f490c100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:01.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:01.276+0000 7f49140cf700 1 -- 172.21.15.146:0/224124843 shutdown_connections 2024-04-25T11:22:01.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:01.276+0000 7f49140cf700 1 -- 172.21.15.146:0/224124843 wait complete. 2024-04-25T11:22:01.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:01.276+0000 7f49140cf700 1 Processor -- start 2024-04-25T11:22:01.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:01.276+0000 7f49140cf700 1 -- start start 2024-04-25T11:22:01.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:01.277+0000 7f49140cf700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f490c076560 0x7f490c1045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:01.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:01.277+0000 7f49140cf700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f490c107340 0x7f490c102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:01.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:01.277+0000 7f49140cf700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f490c109e90 0x7f490c1031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:01.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:01.277+0000 7f49140cf700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f490c10fb30 con 0x7f490c109e90 2024-04-25T11:22:01.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:01.277+0000 7f49140cf700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f490c10f9b0 con 0x7f490c076560 2024-04-25T11:22:01.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:01.277+0000 7f49140cf700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f490c10fcb0 con 0x7f490c107340 2024-04-25T11:22:01.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:01.277+0000 7f4911e6b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f490c076560 0x7f490c1045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:01.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:01.277+0000 7f4911e6b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f490c076560 0x7f490c1045c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:41322/0 (socket says 172.21.15.146:41322) 2024-04-25T11:22:01.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:01.277+0000 7f4911e6b700 1 -- 172.21.15.146:0/3545222816 learned_addr learned my addr 172.21.15.146:0/3545222816 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-04-25T11:22:01.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:01.277+0000 7f491166a700 1 --2- 172.21.15.146:0/3545222816 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f490c107340 0x7f490c102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:01.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:01.277+0000 7f491266c700 1 --2- 172.21.15.146:0/3545222816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f490c109e90 0x7f490c1031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:01.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:01.277+0000 7f4911e6b700 1 -- 172.21.15.146:0/3545222816 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f490c107340 msgr2=0x7f490c102c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:01.280 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:01.277+0000 7f4911e6b700 1 --2- 172.21.15.146:0/3545222816 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f490c107340 0x7f490c102c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:01.280 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:01.277+0000 7f4911e6b700 1 -- 172.21.15.146:0/3545222816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f490c109e90 msgr2=0x7f490c1031b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:01.280 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:01.277+0000 7f4911e6b700 1 --2- 172.21.15.146:0/3545222816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f490c109e90 0x7f490c1031b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:01.280 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:01.277+0000 7f4911e6b700 1 -- 172.21.15.146:0/3545222816 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f490c103910 con 0x7f490c076560 2024-04-25T11:22:01.280 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:01.277+0000 7f491266c700 1 --2- 172.21.15.146:0/3545222816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f490c109e90 0x7f490c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:22:01.280 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:01.277+0000 7f4911e6b700 1 --2- 172.21.15.146:0/3545222816 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f490c076560 0x7f490c1045c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f490800c8f0 tx=0x7f490800ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:01.280 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:01.277+0000 7f4902ffd700 1 -- 172.21.15.146:0/3545222816 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4908007b00 con 0x7f490c076560 2024-04-25T11:22:01.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:01.277+0000 7f4902ffd700 1 -- 172.21.15.146:0/3545222816 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4908007ca0 con 0x7f490c076560 2024-04-25T11:22:01.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:01.277+0000 7f49140cf700 1 -- 172.21.15.146:0/3545222816 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f490c1a4740 con 0x7f490c076560 2024-04-25T11:22:01.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:01.277+0000 7f4902ffd700 1 -- 172.21.15.146:0/3545222816 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4908018660 con 0x7f490c076560 2024-04-25T11:22:01.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:01.277+0000 7f49140cf700 1 -- 172.21.15.146:0/3545222816 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f490c1a4c70 con 0x7f490c076560 2024-04-25T11:22:01.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:01.278+0000 7f49140cf700 1 -- 172.21.15.146:0/3545222816 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f490c051190 con 0x7f490c076560 2024-04-25T11:22:01.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:01.280+0000 7f4902ffd700 1 -- 172.21.15.146:0/3545222816 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f490800f450 con 0x7f490c076560 2024-04-25T11:22:01.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:01.280+0000 7f4902ffd700 1 --2- 172.21.15.146:0/3545222816 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f48f80afc00 0x7f48f80b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:01.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:01.281+0000 7f4902ffd700 1 -- 172.21.15.146:0/3545222816 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6261+0+0 (secure 0 0 0) 0x7f49080de460 con 0x7f490c076560 2024-04-25T11:22:01.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:01.281+0000 7f491166a700 1 --2- 172.21.15.146:0/3545222816 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f48f80afc00 0x7f48f80b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:01.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:01.281+0000 7f491166a700 1 --2- 172.21.15.146:0/3545222816 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f48f80afc00 0x7f48f80b2110 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f48fc005950 tx=0x7f48fc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:01.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:01.284+0000 7f4902ffd700 1 -- 172.21.15.146:0/3545222816 <== mon.2 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4908014030 con 0x7f490c076560 2024-04-25T11:22:01.491 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:01.495+0000 7f49140cf700 1 -- 172.21.15.146:0/3545222816 --> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f490c02e940 con 0x7f48f80afc00 2024-04-25T11:22:01.851 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:01 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:22:01.851 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:01 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:22:01.851 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:01 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:22:01.945 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:01 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:22:01.945 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:01 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:22:01.945 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:01 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:22:01.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:01 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:22:01.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:01 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:22:01.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:01 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:22:02.851 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:02 smithi146 ceph-mon[32821]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:22:02.852 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:02 smithi146 ceph-mon[32821]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:22:02.945 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:02 smithi069 ceph-mon[29188]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:22:02.945 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:02 smithi069 ceph-mon[29188]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:22:02.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:02 smithi152 ceph-mon[32701]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:22:02.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:02 smithi152 ceph-mon[32701]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:22:05.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:04 smithi146 ceph-mon[32821]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:22:05.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:04 smithi146 ceph-mon[32821]: from='client.? 172.21.15.146:0/2839148154' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "19e967e5-51ce-4506-80c9-0a95441baf99"}]: dispatch 2024-04-25T11:22:05.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:04 smithi146 ceph-mon[32821]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "19e967e5-51ce-4506-80c9-0a95441baf99"}]: dispatch 2024-04-25T11:22:05.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:04 smithi146 ceph-mon[32821]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "19e967e5-51ce-4506-80c9-0a95441baf99"}]': finished 2024-04-25T11:22:05.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:04 smithi146 ceph-mon[32821]: osdmap e61: 11 total, 10 up, 11 in 2024-04-25T11:22:05.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:04 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T11:22:05.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:04 smithi069 ceph-mon[29188]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:22:05.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:04 smithi069 ceph-mon[29188]: from='client.? 172.21.15.146:0/2839148154' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "19e967e5-51ce-4506-80c9-0a95441baf99"}]: dispatch 2024-04-25T11:22:05.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:04 smithi069 ceph-mon[29188]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "19e967e5-51ce-4506-80c9-0a95441baf99"}]: dispatch 2024-04-25T11:22:05.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:04 smithi069 ceph-mon[29188]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "19e967e5-51ce-4506-80c9-0a95441baf99"}]': finished 2024-04-25T11:22:05.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:04 smithi069 ceph-mon[29188]: osdmap e61: 11 total, 10 up, 11 in 2024-04-25T11:22:05.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:04 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T11:22:05.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:04 smithi152 ceph-mon[32701]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:22:05.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:04 smithi152 ceph-mon[32701]: from='client.? 172.21.15.146:0/2839148154' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "19e967e5-51ce-4506-80c9-0a95441baf99"}]: dispatch 2024-04-25T11:22:05.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:04 smithi152 ceph-mon[32701]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "19e967e5-51ce-4506-80c9-0a95441baf99"}]: dispatch 2024-04-25T11:22:05.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:04 smithi152 ceph-mon[32701]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "19e967e5-51ce-4506-80c9-0a95441baf99"}]': finished 2024-04-25T11:22:05.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:04 smithi152 ceph-mon[32701]: osdmap e61: 11 total, 10 up, 11 in 2024-04-25T11:22:05.233 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:04 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T11:22:06.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:05 smithi146 ceph-mon[32821]: from='client.? 172.21.15.146:0/711651395' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:22:06.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:05 smithi069 ceph-mon[29188]: from='client.? 172.21.15.146:0/711651395' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:22:06.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:05 smithi152 ceph-mon[32701]: from='client.? 172.21.15.146:0/711651395' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:22:07.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:06 smithi146 ceph-mon[32821]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:22:07.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:06 smithi069 ceph-mon[29188]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:22:07.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:06 smithi152 ceph-mon[32701]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:22:09.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:08 smithi146 ceph-mon[32821]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:22:09.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:08 smithi069 ceph-mon[29188]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:22:09.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:08 smithi152 ceph-mon[32701]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:22:11.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:10 smithi146 ceph-mon[32821]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:22:11.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:10 smithi069 ceph-mon[29188]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:22:11.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:10 smithi152 ceph-mon[32701]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:22:12.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:11 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T11:22:12.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:11 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:22:12.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:11 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T11:22:12.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:11 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:22:12.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:11 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T11:22:12.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:11 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:22:13.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:12 smithi146 ceph-mon[32821]: Deploying daemon osd.10 on smithi146 2024-04-25T11:22:13.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:12 smithi146 ceph-mon[32821]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:22:13.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:12 smithi069 ceph-mon[29188]: Deploying daemon osd.10 on smithi146 2024-04-25T11:22:13.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:12 smithi069 ceph-mon[29188]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:22:13.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:12 smithi152 ceph-mon[32701]: Deploying daemon osd.10 on smithi146 2024-04-25T11:22:13.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:12 smithi152 ceph-mon[32701]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:22:15.040 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:14 smithi146 ceph-mon[32821]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:22:15.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:14 smithi069 ceph-mon[29188]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:22:15.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:14 smithi152 ceph-mon[32701]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:22:17.071 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:16 smithi146 ceph-mon[32821]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:22:17.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:16 smithi069 ceph-mon[29188]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:22:17.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:16 smithi152 ceph-mon[32701]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:22:18.480 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:18 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:18.480 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:18 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:18.480 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:18 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:22:18.480 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:18 smithi146 ceph-mon[32821]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:22:18.694 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:18 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:18.695 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:18 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:18.695 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:18 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:22:18.695 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:18 smithi069 ceph-mon[29188]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:22:18.732 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:18 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:18.732 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:18 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:18.732 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:18 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:22:18.732 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:18 smithi152 ceph-mon[32701]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:22:21.086 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:20 smithi146 ceph-mon[32821]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:22:21.086 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:20 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:21.086 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:20 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:21.086 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:20 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:22:21.086 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:20 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:22:21.086 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:20 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:21.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:20 smithi069 ceph-mon[29188]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:22:21.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:20 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:21.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:20 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:21.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:20 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:22:21.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:20 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:22:21.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:20 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:21.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:20 smithi152 ceph-mon[32701]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:22:21.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:20 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:21.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:20 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:21.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:20 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:22:21.234 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:20 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:22:21.235 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:20 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:21.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:21.461+0000 7f4902ffd700 1 -- 172.21.15.146:0/3545222816 <== mgr.14152 v2:172.21.15.69:6800/4220065409 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f490c02e940 con 0x7f48f80afc00 2024-04-25T11:22:21.457 INFO:teuthology.orchestra.run.smithi146.stdout:Created osd(s) 10 on host 'smithi146' 2024-04-25T11:22:21.459 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:21.464+0000 7f49140cf700 1 -- 172.21.15.146:0/3545222816 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f48f80afc00 msgr2=0x7f48f80b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:21.459 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:21.464+0000 7f49140cf700 1 --2- 172.21.15.146:0/3545222816 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f48f80afc00 0x7f48f80b2110 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f48fc005950 tx=0x7f48fc0058e0 comp rx=0 tx=0).stop 2024-04-25T11:22:21.459 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:21.464+0000 7f49140cf700 1 -- 172.21.15.146:0/3545222816 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f490c076560 msgr2=0x7f490c1045c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:21.459 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:21.464+0000 7f49140cf700 1 --2- 172.21.15.146:0/3545222816 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f490c076560 0x7f490c1045c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f490800c8f0 tx=0x7f490800ccb0 comp rx=0 tx=0).stop 2024-04-25T11:22:21.459 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:21.464+0000 7f49140cf700 1 -- 172.21.15.146:0/3545222816 shutdown_connections 2024-04-25T11:22:21.459 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:21.464+0000 7f49140cf700 1 --2- 172.21.15.146:0/3545222816 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f48f80afc00 0x7f48f80b2110 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:21.459 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:21.464+0000 7f49140cf700 1 --2- 172.21.15.146:0/3545222816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f490c109e90 0x7f490c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:21.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:21.464+0000 7f49140cf700 1 --2- 172.21.15.146:0/3545222816 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f490c076560 0x7f490c1045c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:21.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:21.464+0000 7f49140cf700 1 --2- 172.21.15.146:0/3545222816 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f490c107340 0x7f490c102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:21.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:21.464+0000 7f49140cf700 1 -- 172.21.15.146:0/3545222816 >> 172.21.15.146:0/3545222816 conn(0x7f490c0fe740 msgr2=0x7f490c105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:21.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:21.464+0000 7f49140cf700 1 -- 172.21.15.146:0/3545222816 shutdown_connections 2024-04-25T11:22:21.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-04-25T11:22:21.464+0000 7f49140cf700 1 -- 172.21.15.146:0/3545222816 wait complete. 2024-04-25T11:22:22.092 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:21 smithi146 ceph-mon[32821]: from='osd.10 [v2:172.21.15.146:6824/707256400,v1:172.21.15.146:6825/707256400]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T11:22:22.092 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:21 smithi146 ceph-mon[32821]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T11:22:22.092 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:21 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:22.093 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:21 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:22.093 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:21 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:22:22.121 DEBUG:teuthology.orchestra.run.smithi146:osd.10> sudo journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@osd.10.service 2024-04-25T11:22:22.124 INFO:tasks.cephadm:Deploying osd.11 on smithi152 with /dev/vg_nvme/lv_1... 2024-04-25T11:22:22.124 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T11:22:22.150 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:21 smithi152 ceph-mon[32701]: from='osd.10 [v2:172.21.15.146:6824/707256400,v1:172.21.15.146:6825/707256400]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T11:22:22.150 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:21 smithi152 ceph-mon[32701]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T11:22:22.150 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:21 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:22.150 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:21 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:22.150 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:21 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:22:22.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:21 smithi069 ceph-mon[29188]: from='osd.10 [v2:172.21.15.146:6824/707256400,v1:172.21.15.146:6825/707256400]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T11:22:22.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:21 smithi069 ceph-mon[29188]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T11:22:22.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:21 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:22.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:21 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:22.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:21 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:22:22.378 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T11:22:23.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:22 smithi146 ceph-mon[32821]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:22:23.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:22 smithi146 ceph-mon[32821]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T11:22:23.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:22 smithi146 ceph-mon[32821]: from='osd.10 [v2:172.21.15.146:6824/707256400,v1:172.21.15.146:6825/707256400]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-04-25T11:22:23.102 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:22 smithi146 ceph-mon[32821]: osdmap e62: 11 total, 10 up, 11 in 2024-04-25T11:22:23.102 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:22 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T11:22:23.102 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:22 smithi146 ceph-mon[32821]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-04-25T11:22:23.102 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:22:22 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:22:22.831+0000 7f2628f10700 -1 osd.10 0 waiting for initial osdmap 2024-04-25T11:22:23.102 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:22:22 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:22:22.843+0000 7f2624cc5700 -1 osd.10 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:22:23.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:22 smithi069 ceph-mon[29188]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:22:23.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:22 smithi069 ceph-mon[29188]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T11:22:23.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:22 smithi069 ceph-mon[29188]: from='osd.10 [v2:172.21.15.146:6824/707256400,v1:172.21.15.146:6825/707256400]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-04-25T11:22:23.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:22 smithi069 ceph-mon[29188]: osdmap e62: 11 total, 10 up, 11 in 2024-04-25T11:22:23.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:22 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T11:22:23.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:22 smithi069 ceph-mon[29188]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-04-25T11:22:23.205 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:22 smithi152 ceph-mon[32701]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:22:23.205 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:22 smithi152 ceph-mon[32701]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T11:22:23.205 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:22 smithi152 ceph-mon[32701]: from='osd.10 [v2:172.21.15.146:6824/707256400,v1:172.21.15.146:6825/707256400]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-04-25T11:22:23.205 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:22 smithi152 ceph-mon[32701]: osdmap e62: 11 total, 10 up, 11 in 2024-04-25T11:22:23.205 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:22 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T11:22:23.205 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:22 smithi152 ceph-mon[32701]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-04-25T11:22:23.850 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:23 smithi146 ceph-mon[32821]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-04-25T11:22:23.851 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:23 smithi146 ceph-mon[32821]: osdmap e63: 11 total, 10 up, 11 in 2024-04-25T11:22:23.851 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:23 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T11:22:23.851 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:23 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T11:22:23.851 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:23 smithi146 ceph-mon[32821]: osd.10 [v2:172.21.15.146:6824/707256400,v1:172.21.15.146:6825/707256400] boot 2024-04-25T11:22:23.851 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:23 smithi146 ceph-mon[32821]: osdmap e64: 11 total, 11 up, 11 in 2024-04-25T11:22:23.851 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:23 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T11:22:24.076 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:23 smithi152 ceph-mon[32701]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-04-25T11:22:24.076 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:23 smithi152 ceph-mon[32701]: osdmap e63: 11 total, 10 up, 11 in 2024-04-25T11:22:24.076 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:23 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T11:22:24.076 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:23 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T11:22:24.077 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:23 smithi152 ceph-mon[32701]: osd.10 [v2:172.21.15.146:6824/707256400,v1:172.21.15.146:6825/707256400] boot 2024-04-25T11:22:24.077 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:23 smithi152 ceph-mon[32701]: osdmap e64: 11 total, 11 up, 11 in 2024-04-25T11:22:24.077 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:23 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T11:22:24.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:23 smithi069 ceph-mon[29188]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-04-25T11:22:24.248 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:23 smithi069 ceph-mon[29188]: osdmap e63: 11 total, 10 up, 11 in 2024-04-25T11:22:24.249 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:23 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T11:22:24.249 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:23 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T11:22:24.249 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:23 smithi069 ceph-mon[29188]: osd.10 [v2:172.21.15.146:6824/707256400,v1:172.21.15.146:6825/707256400] boot 2024-04-25T11:22:24.249 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:23 smithi069 ceph-mon[29188]: osdmap e64: 11 total, 11 up, 11 in 2024-04-25T11:22:24.249 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:23 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T11:22:24.422 INFO:teuthology.orchestra.run.smithi152.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T11:22:24.422 INFO:teuthology.orchestra.run.smithi152.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T11:22:24.422 INFO:teuthology.orchestra.run.smithi152.stderr: stderr: 10+0 records in 2024-04-25T11:22:24.422 INFO:teuthology.orchestra.run.smithi152.stderr:10+0 records out 2024-04-25T11:22:24.422 INFO:teuthology.orchestra.run.smithi152.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0116336 s, 901 MB/s 2024-04-25T11:22:24.422 INFO:teuthology.orchestra.run.smithi152.stderr:--> Zapping successful for: 2024-04-25T11:22:25.033 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:24 smithi152 ceph-mon[32701]: purged_snaps scrub starts 2024-04-25T11:22:25.033 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:24 smithi152 ceph-mon[32701]: purged_snaps scrub ok 2024-04-25T11:22:25.033 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:24 smithi152 ceph-mon[32701]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:22:25.033 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:24 smithi152 ceph-mon[32701]: osdmap e65: 11 total, 11 up, 11 in 2024-04-25T11:22:25.063 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi152:vg_nvme/lv_1 2024-04-25T11:22:25.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:24 smithi146 ceph-mon[32821]: purged_snaps scrub starts 2024-04-25T11:22:25.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:24 smithi146 ceph-mon[32821]: purged_snaps scrub ok 2024-04-25T11:22:25.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:24 smithi146 ceph-mon[32821]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:22:25.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:24 smithi146 ceph-mon[32821]: osdmap e65: 11 total, 11 up, 11 in 2024-04-25T11:22:25.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:24 smithi069 ceph-mon[29188]: purged_snaps scrub starts 2024-04-25T11:22:25.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:24 smithi069 ceph-mon[29188]: purged_snaps scrub ok 2024-04-25T11:22:25.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:24 smithi069 ceph-mon[29188]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:22:25.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:24 smithi069 ceph-mon[29188]: osdmap e65: 11 total, 11 up, 11 in 2024-04-25T11:22:25.346 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T11:22:26.351 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:26 smithi146 ceph-mon[32821]: osdmap e66: 11 total, 11 up, 11 in 2024-04-25T11:22:26.351 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:26 smithi146 ceph-mon[32821]: Detected new or changed devices on smithi146 2024-04-25T11:22:26.351 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:26 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:26.351 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:26 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:26.351 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:26 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:22:26.351 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:26 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:22:26.351 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:26 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:22:26.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:26 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:22:26.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:26 smithi146 ceph-mon[32821]: Adjusting osd_memory_target on smithi146 to 4272M 2024-04-25T11:22:26.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:26 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:26.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:26 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:22:26.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:26 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:22:26.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:26 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:26.352 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:26 smithi146 ceph-mon[32821]: pgmap v233: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-25T11:22:26.445 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:26 smithi069 ceph-mon[29188]: osdmap e66: 11 total, 11 up, 11 in 2024-04-25T11:22:26.445 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:26 smithi069 ceph-mon[29188]: Detected new or changed devices on smithi146 2024-04-25T11:22:26.445 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:26 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:26.445 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:26 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:26.445 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:26 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:22:26.445 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:26 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:22:26.445 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:26 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:22:26.445 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:26 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:22:26.445 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:26 smithi069 ceph-mon[29188]: Adjusting osd_memory_target on smithi146 to 4272M 2024-04-25T11:22:26.445 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:26 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:26.445 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:26 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:22:26.445 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:26 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:22:26.446 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:26 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:26.446 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:26 smithi069 ceph-mon[29188]: pgmap v233: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-25T11:22:26.482 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:26 smithi152 ceph-mon[32701]: osdmap e66: 11 total, 11 up, 11 in 2024-04-25T11:22:26.482 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:26 smithi152 ceph-mon[32701]: Detected new or changed devices on smithi146 2024-04-25T11:22:26.482 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:26 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:26.483 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:26 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:26.483 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:26 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:22:26.483 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:26 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:22:26.483 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:26 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:22:26.483 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:26 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:22:26.483 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:26 smithi152 ceph-mon[32701]: Adjusting osd_memory_target on smithi146 to 4272M 2024-04-25T11:22:26.483 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:26 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:26.483 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:26 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:22:26.483 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:26 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:22:26.483 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:26 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:26.483 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:26 smithi152 ceph-mon[32701]: pgmap v233: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-25T11:22:27.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:27.178+0000 7fc1bd930700 1 -- 172.21.15.152:0/4118644317 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1b8104bb0 msgr2=0x7fc1b810b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:27.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:27.178+0000 7fc1bd930700 1 --2- 172.21.15.152:0/4118644317 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1b8104bb0 0x7fc1b810b460 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc1b000b0a0 tx=0x7fc1b000b3b0 comp rx=0 tx=0).stop 2024-04-25T11:22:27.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:27.178+0000 7fc1bd930700 1 -- 172.21.15.152:0/4118644317 shutdown_connections 2024-04-25T11:22:27.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:27.178+0000 7fc1bd930700 1 --2- 172.21.15.152:0/4118644317 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1b8104bb0 0x7fc1b810b460 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:27.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:27.178+0000 7fc1bd930700 1 --2- 172.21.15.152:0/4118644317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1b81030e0 0x7fc1b81034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:27.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:27.178+0000 7fc1bd930700 1 --2- 172.21.15.152:0/4118644317 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1b8104210 0x7fc1b8104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:27.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:27.178+0000 7fc1bd930700 1 -- 172.21.15.152:0/4118644317 >> 172.21.15.152:0/4118644317 conn(0x7fc1b80fe740 msgr2=0x7fc1b8100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:27.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:27.178+0000 7fc1bd930700 1 -- 172.21.15.152:0/4118644317 shutdown_connections 2024-04-25T11:22:27.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:27.178+0000 7fc1bd930700 1 -- 172.21.15.152:0/4118644317 wait complete. 2024-04-25T11:22:27.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:27.179+0000 7fc1bd930700 1 Processor -- start 2024-04-25T11:22:27.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:27.179+0000 7fc1bd930700 1 -- start start 2024-04-25T11:22:27.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:27.179+0000 7fc1bd930700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1b81030e0 0x7fc1b8199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:27.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:27.179+0000 7fc1bd930700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1b8104210 0x7fc1b8199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:27.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:27.179+0000 7fc1bd930700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1b8104bb0 0x7fc1b81a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:27.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:27.179+0000 7fc1bd930700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b810ea10 con 0x7fc1b8104210 2024-04-25T11:22:27.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:27.179+0000 7fc1bd930700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b810e890 con 0x7fc1b81030e0 2024-04-25T11:22:27.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:27.179+0000 7fc1bd930700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b810eb90 con 0x7fc1b8104bb0 2024-04-25T11:22:27.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:27.179+0000 7fc1b77fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1b8104bb0 0x7fc1b81a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:27.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:27.179+0000 7fc1b77fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1b8104bb0 0x7fc1b81a0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:53830/0 (socket says 172.21.15.152:53830) 2024-04-25T11:22:27.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:27.179+0000 7fc1b77fe700 1 -- 172.21.15.152:0/1759750611 learned_addr learned my addr 172.21.15.152:0/1759750611 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-25T11:22:27.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:27.179+0000 7fc1b6ffd700 1 --2- 172.21.15.152:0/1759750611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1b81030e0 0x7fc1b8199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:27.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:27.180+0000 7fc1ae7fc700 1 --2- 172.21.15.152:0/1759750611 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1b8104210 0x7fc1b8199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:27.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:27.180+0000 7fc1b77fe700 1 -- 172.21.15.152:0/1759750611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1b81030e0 msgr2=0x7fc1b8199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:27.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:27.180+0000 7fc1b77fe700 1 --2- 172.21.15.152:0/1759750611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1b81030e0 0x7fc1b8199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:27.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:27.180+0000 7fc1b77fe700 1 -- 172.21.15.152:0/1759750611 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1b8104210 msgr2=0x7fc1b8199aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:27.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:27.180+0000 7fc1b77fe700 1 --2- 172.21.15.152:0/1759750611 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1b8104210 0x7fc1b8199aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:27.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:27.180+0000 7fc1b77fe700 1 -- 172.21.15.152:0/1759750611 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1b81a1030 con 0x7fc1b8104bb0 2024-04-25T11:22:27.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:27.180+0000 7fc1ae7fc700 1 --2- 172.21.15.152:0/1759750611 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1b8104210 0x7fc1b8199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:22:27.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:27.180+0000 7fc1b77fe700 1 --2- 172.21.15.152:0/1759750611 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1b8104bb0 0x7fc1b81a0a40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0009030 tx=0x7fc1b00097f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:27.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:27.180+0000 7fc1bc92e700 1 -- 172.21.15.152:0/1759750611 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b001f030 con 0x7fc1b8104bb0 2024-04-25T11:22:27.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:27.180+0000 7fc1bc92e700 1 -- 172.21.15.152:0/1759750611 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc1b0005040 con 0x7fc1b8104bb0 2024-04-25T11:22:27.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:27.180+0000 7fc1bd930700 1 -- 172.21.15.152:0/1759750611 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1b81a12c0 con 0x7fc1b8104bb0 2024-04-25T11:22:27.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:27.180+0000 7fc1bc92e700 1 -- 172.21.15.152:0/1759750611 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b001b690 con 0x7fc1b8104bb0 2024-04-25T11:22:27.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:27.180+0000 7fc1bd930700 1 -- 172.21.15.152:0/1759750611 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1b81a17c0 con 0x7fc1b8104bb0 2024-04-25T11:22:27.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:27.181+0000 7fc1bd930700 1 -- 172.21.15.152:0/1759750611 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1b8051190 con 0x7fc1b8104bb0 2024-04-25T11:22:27.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:27.183+0000 7fc1bc92e700 1 -- 172.21.15.152:0/1759750611 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fc1b0022030 con 0x7fc1b8104bb0 2024-04-25T11:22:27.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:27.183+0000 7fc1bc92e700 1 --2- 172.21.15.152:0/1759750611 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc1a40afc00 0x7fc1a40b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:27.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:27.183+0000 7fc1bc92e700 1 -- 172.21.15.152:0/1759750611 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6694+0+0 (secure 0 0 0) 0x7fc1b000f020 con 0x7fc1b8104bb0 2024-04-25T11:22:27.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:27.184+0000 7fc1b6ffd700 1 --2- 172.21.15.152:0/1759750611 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc1a40afc00 0x7fc1a40b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:27.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:27.184+0000 7fc1b6ffd700 1 --2- 172.21.15.152:0/1759750611 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc1a40afc00 0x7fc1a40b2110 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8104110 tx=0x7fc1a800a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:27.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:27.187+0000 7fc1bc92e700 1 -- 172.21.15.152:0/1759750611 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc1b0029020 con 0x7fc1b8104bb0 2024-04-25T11:22:27.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:27.393+0000 7fc1bd930700 1 -- 172.21.15.152:0/1759750611 --> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fc1b8063ad0 con 0x7fc1a40afc00 2024-04-25T11:22:27.694 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:27 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:22:27.695 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:27 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:22:27.695 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:27 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:22:27.732 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:27 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:22:27.732 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:27 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:22:27.732 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:27 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:22:27.850 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:27 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:22:27.851 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:27 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:22:27.851 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:27 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:22:28.732 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:28 smithi152 ceph-mon[32701]: from='client.24328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:22:28.732 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:28 smithi152 ceph-mon[32701]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-25T11:22:28.850 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:28 smithi146 ceph-mon[32821]: from='client.24328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:22:28.851 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:28 smithi146 ceph-mon[32821]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-25T11:22:28.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:28 smithi069 ceph-mon[29188]: from='client.24328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:22:28.945 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:28 smithi069 ceph-mon[29188]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-25T11:22:30.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:30 smithi152 ceph-mon[32701]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T11:22:30.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:30 smithi152 ceph-mon[32701]: from='client.? 172.21.15.152:0/2833219242' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7788c2de-6930-4808-be0f-f1fd1ae0cb4b"}]: dispatch 2024-04-25T11:22:30.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:30 smithi152 ceph-mon[32701]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7788c2de-6930-4808-be0f-f1fd1ae0cb4b"}]: dispatch 2024-04-25T11:22:30.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:30 smithi152 ceph-mon[32701]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7788c2de-6930-4808-be0f-f1fd1ae0cb4b"}]': finished 2024-04-25T11:22:30.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:30 smithi152 ceph-mon[32701]: osdmap e67: 12 total, 11 up, 12 in 2024-04-25T11:22:30.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:30 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T11:22:31.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:30 smithi146 ceph-mon[32821]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T11:22:31.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:30 smithi146 ceph-mon[32821]: from='client.? 172.21.15.152:0/2833219242' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7788c2de-6930-4808-be0f-f1fd1ae0cb4b"}]: dispatch 2024-04-25T11:22:31.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:30 smithi146 ceph-mon[32821]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7788c2de-6930-4808-be0f-f1fd1ae0cb4b"}]: dispatch 2024-04-25T11:22:31.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:30 smithi146 ceph-mon[32821]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7788c2de-6930-4808-be0f-f1fd1ae0cb4b"}]': finished 2024-04-25T11:22:31.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:30 smithi146 ceph-mon[32821]: osdmap e67: 12 total, 11 up, 12 in 2024-04-25T11:22:31.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:30 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T11:22:31.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:30 smithi069 ceph-mon[29188]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T11:22:31.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:30 smithi069 ceph-mon[29188]: from='client.? 172.21.15.152:0/2833219242' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7788c2de-6930-4808-be0f-f1fd1ae0cb4b"}]: dispatch 2024-04-25T11:22:31.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:30 smithi069 ceph-mon[29188]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7788c2de-6930-4808-be0f-f1fd1ae0cb4b"}]: dispatch 2024-04-25T11:22:31.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:30 smithi069 ceph-mon[29188]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7788c2de-6930-4808-be0f-f1fd1ae0cb4b"}]': finished 2024-04-25T11:22:31.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:30 smithi069 ceph-mon[29188]: osdmap e67: 12 total, 11 up, 12 in 2024-04-25T11:22:31.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:30 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T11:22:32.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:31 smithi146 ceph-mon[32821]: from='client.? 172.21.15.152:0/3271879700' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:22:32.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:31 smithi069 ceph-mon[29188]: from='client.? 172.21.15.152:0/3271879700' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:22:32.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:31 smithi152 ceph-mon[32701]: from='client.? 172.21.15.152:0/3271879700' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:22:33.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:32 smithi146 ceph-mon[32821]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-04-25T11:22:33.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:32 smithi069 ceph-mon[29188]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-04-25T11:22:33.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:32 smithi152 ceph-mon[32701]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-04-25T11:22:35.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:34 smithi146 ceph-mon[32821]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-04-25T11:22:35.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:34 smithi069 ceph-mon[29188]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-04-25T11:22:35.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:34 smithi152 ceph-mon[32701]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-04-25T11:22:36.947 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:36 smithi152 ceph-mon[32701]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T11:22:37.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:36 smithi146 ceph-mon[32821]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T11:22:37.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:36 smithi069 ceph-mon[29188]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T11:22:38.923 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:38 smithi152 ceph-mon[32701]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T11:22:38.923 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:38 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T11:22:38.923 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:38 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:22:39.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:38 smithi146 ceph-mon[32821]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T11:22:39.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:38 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T11:22:39.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:38 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:22:39.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:38 smithi069 ceph-mon[29188]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T11:22:39.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:38 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T11:22:39.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:38 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:22:39.912 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:39 smithi152 ceph-mon[32701]: Deploying daemon osd.11 on smithi152 2024-04-25T11:22:40.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:39 smithi146 ceph-mon[32821]: Deploying daemon osd.11 on smithi152 2024-04-25T11:22:40.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:39 smithi069 ceph-mon[29188]: Deploying daemon osd.11 on smithi152 2024-04-25T11:22:40.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:40 smithi152 ceph-mon[32701]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T11:22:41.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:40 smithi146 ceph-mon[32821]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T11:22:41.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:40 smithi069 ceph-mon[29188]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T11:22:42.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:42 smithi152 ceph-mon[32701]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T11:22:43.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:42 smithi146 ceph-mon[32821]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T11:22:43.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:42 smithi069 ceph-mon[29188]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T11:22:44.894 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:44 smithi152 ceph-mon[32701]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T11:22:44.894 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:44 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:44.894 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:44 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:44.894 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:44 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:22:45.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:44 smithi146 ceph-mon[32821]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T11:22:45.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:44 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:45.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:44 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:45.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:44 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:22:45.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:44 smithi069 ceph-mon[29188]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T11:22:45.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:44 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:45.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:44 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:45.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:44 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:22:46.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:46 smithi152 ceph-mon[32701]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T11:22:47.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:46 smithi146 ceph-mon[32821]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T11:22:47.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:46 smithi069 ceph-mon[29188]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T11:22:47.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:47.708+0000 7fc1bc92e700 1 -- 172.21.15.152:0/1759750611 <== mgr.14152 v2:172.21.15.69:6800/4220065409 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fc1b8063ad0 con 0x7fc1a40afc00 2024-04-25T11:22:47.705 INFO:teuthology.orchestra.run.smithi152.stdout:Created osd(s) 11 on host 'smithi152' 2024-04-25T11:22:47.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:47.712+0000 7fc1bd930700 1 -- 172.21.15.152:0/1759750611 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc1a40afc00 msgr2=0x7fc1a40b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:47.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:47.712+0000 7fc1bd930700 1 --2- 172.21.15.152:0/1759750611 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc1a40afc00 0x7fc1a40b2110 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8104110 tx=0x7fc1a800a3b0 comp rx=0 tx=0).stop 2024-04-25T11:22:47.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:47.712+0000 7fc1bd930700 1 -- 172.21.15.152:0/1759750611 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1b8104bb0 msgr2=0x7fc1b81a0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:47.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:47.712+0000 7fc1bd930700 1 --2- 172.21.15.152:0/1759750611 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1b8104bb0 0x7fc1b81a0a40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0009030 tx=0x7fc1b00097f0 comp rx=0 tx=0).stop 2024-04-25T11:22:47.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:47.712+0000 7fc1bd930700 1 -- 172.21.15.152:0/1759750611 shutdown_connections 2024-04-25T11:22:47.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:47.712+0000 7fc1bd930700 1 --2- 172.21.15.152:0/1759750611 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc1a40afc00 0x7fc1a40b2110 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:47.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:47.712+0000 7fc1bd930700 1 --2- 172.21.15.152:0/1759750611 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1b8104bb0 0x7fc1b81a0a40 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:47.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:47.712+0000 7fc1bd930700 1 --2- 172.21.15.152:0/1759750611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc1b81030e0 0x7fc1b8199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:47.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:47.712+0000 7fc1bd930700 1 --2- 172.21.15.152:0/1759750611 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1b8104210 0x7fc1b8199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:47.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:47.712+0000 7fc1bd930700 1 -- 172.21.15.152:0/1759750611 >> 172.21.15.152:0/1759750611 conn(0x7fc1b80fe740 msgr2=0x7fc1b8107440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:47.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:47.712+0000 7fc1bd930700 1 -- 172.21.15.152:0/1759750611 shutdown_connections 2024-04-25T11:22:47.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:47.712+0000 7fc1bd930700 1 -- 172.21.15.152:0/1759750611 wait complete. 2024-04-25T11:22:48.538 DEBUG:teuthology.orchestra.run.smithi152:osd.11> sudo journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@osd.11.service 2024-04-25T11:22:48.540 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-25T11:22:48.540 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-25T11:22:48.792 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:22:48.915 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:48 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:48.915 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:48 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:48.915 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:48 smithi069 ceph-mon[29188]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T11:22:48.915 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:48 smithi069 ceph-mon[29188]: from='osd.11 [v2:172.21.15.152:6824/2934336833,v1:172.21.15.152:6825/2934336833]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T11:22:48.915 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:48 smithi069 ceph-mon[29188]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T11:22:48.915 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:48 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:48.915 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:48 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:48.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:48 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:48.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:48 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:48.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:48 smithi152 ceph-mon[32701]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T11:22:48.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:48 smithi152 ceph-mon[32701]: from='osd.11 [v2:172.21.15.152:6824/2934336833,v1:172.21.15.152:6825/2934336833]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T11:22:48.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:48 smithi152 ceph-mon[32701]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T11:22:48.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:48 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:48.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:48 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:49.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:48 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:49.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:48 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:49.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:48 smithi146 ceph-mon[32821]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T11:22:49.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:48 smithi146 ceph-mon[32821]: from='osd.11 [v2:172.21.15.152:6824/2934336833,v1:172.21.15.152:6825/2934336833]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T11:22:49.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:48 smithi146 ceph-mon[32821]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T11:22:49.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:48 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:49.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:48 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:49.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:49 smithi069 ceph-mon[29188]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T11:22:49.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:49 smithi069 ceph-mon[29188]: osdmap e68: 12 total, 11 up, 12 in 2024-04-25T11:22:49.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:49 smithi069 ceph-mon[29188]: from='osd.11 [v2:172.21.15.152:6824/2934336833,v1:172.21.15.152:6825/2934336833]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T11:22:49.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:49 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T11:22:49.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:49 smithi069 ceph-mon[29188]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T11:22:49.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:49 smithi152 ceph-mon[32701]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T11:22:49.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:49 smithi152 ceph-mon[32701]: osdmap e68: 12 total, 11 up, 12 in 2024-04-25T11:22:49.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:49 smithi152 ceph-mon[32701]: from='osd.11 [v2:172.21.15.152:6824/2934336833,v1:172.21.15.152:6825/2934336833]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T11:22:49.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:49 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T11:22:49.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:49 smithi152 ceph-mon[32701]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T11:22:49.982 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:22:49 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:22:49.694+0000 7f392afde700 -1 osd.11 0 waiting for initial osdmap 2024-04-25T11:22:49.982 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:22:49 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:22:49.708+0000 7f3926d93700 -1 osd.11 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:22:50.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:49 smithi146 ceph-mon[32821]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T11:22:50.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:49 smithi146 ceph-mon[32821]: osdmap e68: 12 total, 11 up, 12 in 2024-04-25T11:22:50.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:49 smithi146 ceph-mon[32821]: from='osd.11 [v2:172.21.15.152:6824/2934336833,v1:172.21.15.152:6825/2934336833]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T11:22:50.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:49 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T11:22:50.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:49 smithi146 ceph-mon[32821]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-04-25T11:22:50.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.726+0000 7fc3914fd700 1 -- 172.21.15.69:0/3310388477 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc38c1030e0 msgr2=0x7fc38c1034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:50.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.726+0000 7fc3914fd700 1 --2- 172.21.15.69:0/3310388477 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc38c1030e0 0x7fc38c1034f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc37c009990 tx=0x7fc37c009ca0 comp rx=0 tx=0).stop 2024-04-25T11:22:50.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.726+0000 7fc3914fd700 1 -- 172.21.15.69:0/3310388477 shutdown_connections 2024-04-25T11:22:50.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.726+0000 7fc3914fd700 1 --2- 172.21.15.69:0/3310388477 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc38c104bb0 0x7fc38c10b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:50.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.726+0000 7fc3914fd700 1 --2- 172.21.15.69:0/3310388477 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc38c1030e0 0x7fc38c1034f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:50.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.726+0000 7fc3914fd700 1 --2- 172.21.15.69:0/3310388477 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc38c104210 0x7fc38c104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:50.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.726+0000 7fc3914fd700 1 -- 172.21.15.69:0/3310388477 >> 172.21.15.69:0/3310388477 conn(0x7fc38c0fe740 msgr2=0x7fc38c100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:50.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.726+0000 7fc3914fd700 1 -- 172.21.15.69:0/3310388477 shutdown_connections 2024-04-25T11:22:50.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.726+0000 7fc3914fd700 1 -- 172.21.15.69:0/3310388477 wait complete. 2024-04-25T11:22:50.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.727+0000 7fc3914fd700 1 Processor -- start 2024-04-25T11:22:50.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.727+0000 7fc3914fd700 1 -- start start 2024-04-25T11:22:50.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.727+0000 7fc3914fd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc38c1030e0 0x7fc38c06bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:50.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.727+0000 7fc3914fd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc38c104210 0x7fc38c19f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:50.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.727+0000 7fc3914fd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc38c104bb0 0x7fc38c1a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:50.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.727+0000 7fc3914fd700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc38c10e9e0 con 0x7fc38c104210 2024-04-25T11:22:50.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.727+0000 7fc3914fd700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc38c10e860 con 0x7fc38c1030e0 2024-04-25T11:22:50.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.727+0000 7fc3914fd700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc38c10eb60 con 0x7fc38c104bb0 2024-04-25T11:22:50.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.727+0000 7fc383fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc38c104210 0x7fc38c19f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:50.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.727+0000 7fc383fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc38c104210 0x7fc38c19f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43482/0 (socket says 172.21.15.69:43482) 2024-04-25T11:22:50.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.727+0000 7fc383fff700 1 -- 172.21.15.69:0/1475421074 learned_addr learned my addr 172.21.15.69:0/1475421074 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:22:50.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.727+0000 7fc38b7fe700 1 --2- 172.21.15.69:0/1475421074 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc38c104bb0 0x7fc38c1a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:50.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.727+0000 7fc38affd700 1 --2- 172.21.15.69:0/1475421074 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc38c1030e0 0x7fc38c06bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:50.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.727+0000 7fc383fff700 1 -- 172.21.15.69:0/1475421074 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc38c104bb0 msgr2=0x7fc38c1a1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:50.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.727+0000 7fc383fff700 1 --2- 172.21.15.69:0/1475421074 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc38c104bb0 0x7fc38c1a1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:50.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.727+0000 7fc383fff700 1 -- 172.21.15.69:0/1475421074 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc38c1030e0 msgr2=0x7fc38c06bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:50.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.727+0000 7fc383fff700 1 --2- 172.21.15.69:0/1475421074 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc38c1030e0 0x7fc38c06bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:50.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.727+0000 7fc383fff700 1 -- 172.21.15.69:0/1475421074 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc38c1a2270 con 0x7fc38c104210 2024-04-25T11:22:50.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.727+0000 7fc383fff700 1 --2- 172.21.15.69:0/1475421074 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc38c104210 0x7fc38c19f560 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc37400c9b0 tx=0x7fc37400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:50.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.728+0000 7fc388ff9700 1 -- 172.21.15.69:0/1475421074 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc374007b00 con 0x7fc38c104210 2024-04-25T11:22:50.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.728+0000 7fc3914fd700 1 -- 172.21.15.69:0/1475421074 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc38c1a2560 con 0x7fc38c104210 2024-04-25T11:22:50.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.728+0000 7fc388ff9700 1 -- 172.21.15.69:0/1475421074 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc37400ce50 con 0x7fc38c104210 2024-04-25T11:22:50.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.728+0000 7fc388ff9700 1 -- 172.21.15.69:0/1475421074 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc374018660 con 0x7fc38c104210 2024-04-25T11:22:50.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.728+0000 7fc3914fd700 1 -- 172.21.15.69:0/1475421074 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc38c1a2aa0 con 0x7fc38c104210 2024-04-25T11:22:50.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.729+0000 7fc3914fd700 1 -- 172.21.15.69:0/1475421074 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc38c051190 con 0x7fc38c104210 2024-04-25T11:22:50.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.730+0000 7fc388ff9700 1 -- 172.21.15.69:0/1475421074 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fc37400f450 con 0x7fc38c104210 2024-04-25T11:22:50.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.731+0000 7fc388ff9700 1 --2- 172.21.15.69:0/1475421074 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc3780afb80 0x7fc3780b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:50.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.731+0000 7fc38affd700 1 --2- 172.21.15.69:0/1475421074 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc3780afb80 0x7fc3780b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:50.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.731+0000 7fc388ff9700 1 -- 172.21.15.69:0/1475421074 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7fc3740de940 con 0x7fc38c104210 2024-04-25T11:22:50.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.731+0000 7fc38affd700 1 --2- 172.21.15.69:0/1475421074 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc3780afb80 0x7fc3780b2090 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc37c0098e0 tx=0x7fc37c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:50.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.735+0000 7fc388ff9700 1 -- 172.21.15.69:0/1475421074 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc374014030 con 0x7fc38c104210 2024-04-25T11:22:50.822 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:50 smithi152 ceph-mon[32701]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-04-25T11:22:50.822 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:50 smithi152 ceph-mon[32701]: osdmap e69: 12 total, 11 up, 12 in 2024-04-25T11:22:50.822 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:50 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T11:22:50.822 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:50 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T11:22:50.823 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:50 smithi152 ceph-mon[32701]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T11:22:50.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.932+0000 7fc3914fd700 1 -- 172.21.15.69:0/1475421074 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fc38c02e510 con 0x7fc38c104210 2024-04-25T11:22:50.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.932+0000 7fc388ff9700 1 -- 172.21.15.69:0/1475421074 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v70) ==== 74+0+133 (secure 0 0 0) 0x7fc3740a7a50 con 0x7fc38c104210 2024-04-25T11:22:50.927 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:22:50.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.934+0000 7fc3914fd700 1 -- 172.21.15.69:0/1475421074 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc3780afb80 msgr2=0x7fc3780b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:50.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.934+0000 7fc3914fd700 1 --2- 172.21.15.69:0/1475421074 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc3780afb80 0x7fc3780b2090 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc37c0098e0 tx=0x7fc37c01a040 comp rx=0 tx=0).stop 2024-04-25T11:22:50.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.934+0000 7fc3914fd700 1 -- 172.21.15.69:0/1475421074 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc38c104210 msgr2=0x7fc38c19f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:50.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.934+0000 7fc3914fd700 1 --2- 172.21.15.69:0/1475421074 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc38c104210 0x7fc38c19f560 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc37400c9b0 tx=0x7fc37400ccc0 comp rx=0 tx=0).stop 2024-04-25T11:22:50.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.934+0000 7fc3914fd700 1 -- 172.21.15.69:0/1475421074 shutdown_connections 2024-04-25T11:22:50.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.934+0000 7fc3914fd700 1 --2- 172.21.15.69:0/1475421074 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc3780afb80 0x7fc3780b2090 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:50.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.934+0000 7fc3914fd700 1 --2- 172.21.15.69:0/1475421074 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc38c104bb0 0x7fc38c1a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:50.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.934+0000 7fc3914fd700 1 --2- 172.21.15.69:0/1475421074 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc38c1030e0 0x7fc38c06bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:50.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.934+0000 7fc3914fd700 1 --2- 172.21.15.69:0/1475421074 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc38c104210 0x7fc38c19f560 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:50.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.934+0000 7fc3914fd700 1 -- 172.21.15.69:0/1475421074 >> 172.21.15.69:0/1475421074 conn(0x7fc38c0fe740 msgr2=0x7fc38c107440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:50.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.934+0000 7fc3914fd700 1 -- 172.21.15.69:0/1475421074 shutdown_connections 2024-04-25T11:22:50.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:50.934+0000 7fc3914fd700 1 -- 172.21.15.69:0/1475421074 wait complete. 2024-04-25T11:22:50.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:50 smithi069 ceph-mon[29188]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-04-25T11:22:50.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:50 smithi069 ceph-mon[29188]: osdmap e69: 12 total, 11 up, 12 in 2024-04-25T11:22:50.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:50 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T11:22:50.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:50 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T11:22:50.945 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:50 smithi069 ceph-mon[29188]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T11:22:51.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:50 smithi146 ceph-mon[32821]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-04-25T11:22:51.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:50 smithi146 ceph-mon[32821]: osdmap e69: 12 total, 11 up, 12 in 2024-04-25T11:22:51.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:50 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T11:22:51.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:50 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T11:22:51.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:50 smithi146 ceph-mon[32821]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-04-25T11:22:51.620 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":70,"num_osds":12,"num_up_osds":12,"osd_up_since":1714044170,"num_in_osds":12,"osd_in_since":1714044150,"num_remapped_pgs":0} 2024-04-25T11:22:51.621 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T11:22:51.731 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:51 smithi152 ceph-mon[32701]: purged_snaps scrub starts 2024-04-25T11:22:51.732 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:51 smithi152 ceph-mon[32701]: purged_snaps scrub ok 2024-04-25T11:22:51.732 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:51 smithi152 ceph-mon[32701]: osd.11 [v2:172.21.15.152:6824/2934336833,v1:172.21.15.152:6825/2934336833] boot 2024-04-25T11:22:51.732 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:51 smithi152 ceph-mon[32701]: osdmap e70: 12 total, 12 up, 12 in 2024-04-25T11:22:51.732 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:51 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T11:22:51.732 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:51 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/1475421074' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T11:22:51.879 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:22:51.910 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:51 smithi069 ceph-mon[29188]: purged_snaps scrub starts 2024-04-25T11:22:51.911 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:51 smithi069 ceph-mon[29188]: purged_snaps scrub ok 2024-04-25T11:22:51.911 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:51 smithi069 ceph-mon[29188]: osd.11 [v2:172.21.15.152:6824/2934336833,v1:172.21.15.152:6825/2934336833] boot 2024-04-25T11:22:51.911 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:51 smithi069 ceph-mon[29188]: osdmap e70: 12 total, 12 up, 12 in 2024-04-25T11:22:51.911 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:51 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T11:22:51.911 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:51 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/1475421074' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T11:22:52.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:51 smithi146 ceph-mon[32821]: purged_snaps scrub starts 2024-04-25T11:22:52.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:51 smithi146 ceph-mon[32821]: purged_snaps scrub ok 2024-04-25T11:22:52.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:51 smithi146 ceph-mon[32821]: osd.11 [v2:172.21.15.152:6824/2934336833,v1:172.21.15.152:6825/2934336833] boot 2024-04-25T11:22:52.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:51 smithi146 ceph-mon[32821]: osdmap e70: 12 total, 12 up, 12 in 2024-04-25T11:22:52.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:51 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T11:22:52.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:51 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/1475421074' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T11:22:52.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:52 smithi152 ceph-mon[32701]: osdmap e71: 12 total, 12 up, 12 in 2024-04-25T11:22:52.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:52 smithi152 ceph-mon[32701]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:22:52.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:52 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:52.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:52 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:52.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:52 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:22:52.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:52 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:22:52.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:52 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:22:52.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:52 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:22:52.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:52 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:52.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:52 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:22:52.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:52 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:22:52.983 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:52 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:52.983 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:52 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:22:53.065 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:52 smithi069 ceph-mon[29188]: osdmap e71: 12 total, 12 up, 12 in 2024-04-25T11:22:53.065 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:52 smithi069 ceph-mon[29188]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:22:53.065 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:52 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:53.065 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:52 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:53.065 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:52 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:22:53.065 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:52 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:22:53.065 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:52 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:22:53.065 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:52 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:22:53.066 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:52 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:53.066 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:52 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:22:53.066 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:52 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:22:53.066 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:52 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:53.066 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:52 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:22:53.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:52 smithi146 ceph-mon[32821]: osdmap e71: 12 total, 12 up, 12 in 2024-04-25T11:22:53.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:52 smithi146 ceph-mon[32821]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:22:53.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:52 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:53.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:52 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:53.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:52 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:22:53.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:52 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:22:53.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:52 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:22:53.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:52 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:22:53.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:52 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:53.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:52 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:22:53.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:52 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:22:53.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:52 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:53.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:52 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:22:53.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.650+0000 7f553103a700 1 -- 172.21.15.69:0/242888945 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f552c107ce0 msgr2=0x7f552c1080f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:53.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.650+0000 7f553103a700 1 --2- 172.21.15.69:0/242888945 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f552c107ce0 0x7f552c1080f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f551c0099c0 tx=0x7f551c009cd0 comp rx=0 tx=0).stop 2024-04-25T11:22:53.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.650+0000 7f553103a700 1 -- 172.21.15.69:0/242888945 shutdown_connections 2024-04-25T11:22:53.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.650+0000 7f553103a700 1 --2- 172.21.15.69:0/242888945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f552c108630 0x7f552c10ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:53.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.650+0000 7f553103a700 1 --2- 172.21.15.69:0/242888945 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f552c1033b0 0x7f552c1037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:53.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.650+0000 7f553103a700 1 --2- 172.21.15.69:0/242888945 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f552c107ce0 0x7f552c1080f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:53.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.650+0000 7f553103a700 1 -- 172.21.15.69:0/242888945 >> 172.21.15.69:0/242888945 conn(0x7f552c078cb0 msgr2=0x7f552c0790f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:53.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.650+0000 7f553103a700 1 -- 172.21.15.69:0/242888945 shutdown_connections 2024-04-25T11:22:53.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.650+0000 7f553103a700 1 -- 172.21.15.69:0/242888945 wait complete. 2024-04-25T11:22:53.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.651+0000 7f553103a700 1 Processor -- start 2024-04-25T11:22:53.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.651+0000 7f553103a700 1 -- start start 2024-04-25T11:22:53.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.652+0000 7f553103a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f552c1033b0 0x7f552c19ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:53.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.652+0000 7f553103a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f552c107ce0 0x7f552c19bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:53.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.652+0000 7f553103a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f552c108630 0x7f552c1a6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:53.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.652+0000 7f553103a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f552c1130f0 con 0x7f552c107ce0 2024-04-25T11:22:53.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.652+0000 7f553103a700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f552c112f70 con 0x7f552c1033b0 2024-04-25T11:22:53.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.652+0000 7f553103a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f552c113270 con 0x7f552c108630 2024-04-25T11:22:53.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.652+0000 7f552a59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f552c107ce0 0x7f552c19bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:53.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.652+0000 7f552a59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f552c107ce0 0x7f552c19bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60242/0 (socket says 172.21.15.69:60242) 2024-04-25T11:22:53.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.652+0000 7f552a59c700 1 -- 172.21.15.69:0/3793122050 learned_addr learned my addr 172.21.15.69:0/3793122050 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:22:53.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.652+0000 7f552ad9d700 1 --2- 172.21.15.69:0/3793122050 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f552c1033b0 0x7f552c19ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:53.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.652+0000 7f552b59e700 1 --2- 172.21.15.69:0/3793122050 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f552c108630 0x7f552c1a6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:53.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.652+0000 7f552a59c700 1 -- 172.21.15.69:0/3793122050 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f552c108630 msgr2=0x7f552c1a6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:53.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.652+0000 7f552a59c700 1 --2- 172.21.15.69:0/3793122050 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f552c108630 0x7f552c1a6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:53.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.652+0000 7f552a59c700 1 -- 172.21.15.69:0/3793122050 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f552c1033b0 msgr2=0x7f552c19ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:53.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.652+0000 7f552a59c700 1 --2- 172.21.15.69:0/3793122050 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f552c1033b0 0x7f552c19ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:53.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.652+0000 7f552a59c700 1 -- 172.21.15.69:0/3793122050 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f552c19c860 con 0x7f552c107ce0 2024-04-25T11:22:53.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.652+0000 7f552a59c700 1 --2- 172.21.15.69:0/3793122050 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f552c107ce0 0x7f552c19bfb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f551c00b580 tx=0x7f551c004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:53.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.653+0000 7f5513fff700 1 -- 172.21.15.69:0/3793122050 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f551c027070 con 0x7f552c107ce0 2024-04-25T11:22:53.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.653+0000 7f553103a700 1 -- 172.21.15.69:0/3793122050 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f552c1a70f0 con 0x7f552c107ce0 2024-04-25T11:22:53.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.653+0000 7f5513fff700 1 -- 172.21.15.69:0/3793122050 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f551c00fc00 con 0x7f552c107ce0 2024-04-25T11:22:53.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.653+0000 7f5513fff700 1 -- 172.21.15.69:0/3793122050 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f551c022cb0 con 0x7f552c107ce0 2024-04-25T11:22:53.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.653+0000 7f553103a700 1 -- 172.21.15.69:0/3793122050 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f552c1a7540 con 0x7f552c107ce0 2024-04-25T11:22:53.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.654+0000 7f553103a700 1 -- 172.21.15.69:0/3793122050 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f552c051190 con 0x7f552c107ce0 2024-04-25T11:22:53.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.655+0000 7f5513fff700 1 -- 172.21.15.69:0/3793122050 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f551c02a070 con 0x7f552c107ce0 2024-04-25T11:22:53.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.656+0000 7f5513fff700 1 --2- 172.21.15.69:0/3793122050 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f55140afb80 0x7f55140b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:53.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.656+0000 7f552ad9d700 1 --2- 172.21.15.69:0/3793122050 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f55140afb80 0x7f55140b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:53.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.656+0000 7f5513fff700 1 -- 172.21.15.69:0/3793122050 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f551c0dfc40 con 0x7f552c107ce0 2024-04-25T11:22:53.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.656+0000 7f552ad9d700 1 --2- 172.21.15.69:0/3793122050 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f55140afb80 0x7f55140b2090 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5520005950 tx=0x7f552000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:53.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.659+0000 7f5513fff700 1 -- 172.21.15.69:0/3793122050 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f551c031020 con 0x7f552c107ce0 2024-04-25T11:22:53.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.860+0000 7f553103a700 1 -- 172.21.15.69:0/3793122050 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f552c02e510 con 0x7f552c107ce0 2024-04-25T11:22:53.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.861+0000 7f5513fff700 1 -- 172.21.15.69:0/3793122050 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+18458 (secure 0 0 0) 0x7f551c013070 con 0x7f552c107ce0 2024-04-25T11:22:53.857 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:22:53.857 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":71,"fsid":"f7ed73dc-02f4-11ef-bc93-c7b262605968","created":"2024-04-25T11:15:37.965281+0000","modified":"2024-04-25T11:22:51.698858+0000","last_up_change":"2024-04-25T11:22:50.691869+0000","last_in_change":"2024-04-25T11:22:30.271959+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:18:57.781263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a6a0d179-4998-4754-bf82-2ea3dac54e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6803","nonce":3756025235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6805","nonce":3756025235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6809","nonce":3756025235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6807","nonce":3756025235}]},"public_addr":"172.21.15.69:6803/3756025235","cluster_addr":"172.21.15.69:6805/3756025235","heartbeat_back_addr":"172.21.15.69:6809/3756025235","heartbeat_front_addr":"172.21.15.69:6807/3756025235","state":["exists","up"]},{"osd":1,"uuid":"c0c66930-4233-493f-b2bb-99c473994b59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6801","nonce":3166305213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6803","nonce":3166305213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6807","nonce":3166305213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6805","nonce":3166305213}]},"public_addr":"172.21.15.146:6801/3166305213","cluster_addr":"172.21.15.146:6803/3166305213","heartbeat_back_addr":"172.21.15.146:6807/3166305213","heartbeat_front_addr":"172.21.15.146:6805/3166305213","state":["exists","up"]},{"osd":2,"uuid":"d4ef735b-9786-4cb9-83ef-7c3f0691a729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6801","nonce":1474319175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6803","nonce":1474319175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6807","nonce":1474319175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6805","nonce":1474319175}]},"public_addr":"172.21.15.152:6801/1474319175","cluster_addr":"172.21.15.152:6803/1474319175","heartbeat_back_addr":"172.21.15.152:6807/1474319175","heartbeat_front_addr":"172.21.15.152:6805/1474319175","state":["exists","up"]},{"osd":3,"uuid":"0ab85fb8-5bbb-4ffc-bddf-cae7644a64ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6811","nonce":1809071296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6813","nonce":1809071296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6817","nonce":1809071296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6815","nonce":1809071296}]},"public_addr":"172.21.15.69:6811/1809071296","cluster_addr":"172.21.15.69:6813/1809071296","heartbeat_back_addr":"172.21.15.69:6817/1809071296","heartbeat_front_addr":"172.21.15.69:6815/1809071296","state":["exists","up"]},{"osd":4,"uuid":"273e0592-bbbc-4188-8eeb-46a3879cf632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6809","nonce":1898520972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6811","nonce":1898520972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6815","nonce":1898520972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6813","nonce":1898520972}]},"public_addr":"172.21.15.146:6809/1898520972","cluster_addr":"172.21.15.146:6811/1898520972","heartbeat_back_addr":"172.21.15.146:6815/1898520972","heartbeat_front_addr":"172.21.15.146:6813/1898520972","state":["exists","up"]},{"osd":5,"uuid":"bbeae945-9fa2-48f9-a1d6-8bf18dcaef19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6809","nonce":1996555108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6811","nonce":1996555108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6815","nonce":1996555108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6813","nonce":1996555108}]},"public_addr":"172.21.15.152:6809/1996555108","cluster_addr":"172.21.15.152:6811/1996555108","heartbeat_back_addr":"172.21.15.152:6815/1996555108","heartbeat_front_addr":"172.21.15.152:6813/1996555108","state":["exists","up"]},{"osd":6,"uuid":"dbb61253-30bc-418e-bd75-118c80bbf931","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6819","nonce":491215762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6821","nonce":491215762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6825","nonce":491215762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6823","nonce":491215762}]},"public_addr":"172.21.15.69:6819/491215762","cluster_addr":"172.21.15.69:6821/491215762","heartbeat_back_addr":"172.21.15.69:6825/491215762","heartbeat_front_addr":"172.21.15.69:6823/491215762","state":["exists","up"]},{"osd":7,"uuid":"75f23854-e283-4e3e-94cf-7b4079d83876","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6817","nonce":2330294690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6819","nonce":2330294690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6823","nonce":2330294690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6821","nonce":2330294690}]},"public_addr":"172.21.15.146:6817/2330294690","cluster_addr":"172.21.15.146:6819/2330294690","heartbeat_back_addr":"172.21.15.146:6823/2330294690","heartbeat_front_addr":"172.21.15.146:6821/2330294690","state":["exists","up"]},{"osd":8,"uuid":"1aa1060c-0c14-4abd-be56-5ebfeb78759c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6817","nonce":950669381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6819","nonce":950669381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6823","nonce":950669381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6821","nonce":950669381}]},"public_addr":"172.21.15.152:6817/950669381","cluster_addr":"172.21.15.152:6819/950669381","heartbeat_back_addr":"172.21.15.152:6823/950669381","heartbeat_front_addr":"172.21.15.152:6821/950669381","state":["exists","up"]},{"osd":9,"uuid":"3f4bf800-4090-4138-9bb3-d001f13f450c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6827","nonce":2414186758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6829","nonce":2414186758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6833","nonce":2414186758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6831","nonce":2414186758}]},"public_addr":"172.21.15.69:6827/2414186758","cluster_addr":"172.21.15.69:6829/2414186758","heartbeat_back_addr":"172.21.15.69:6833/2414186758","heartbeat_front_addr":"172.21.15.69:6831/2414186758","state":["exists","up"]},{"osd":10,"uuid":"19e967e5-51ce-4506-80c9-0a95441baf99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6825","nonce":707256400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6827","nonce":707256400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6831","nonce":707256400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6829","nonce":707256400}]},"public_addr":"172.21.15.146:6825/707256400","cluster_addr":"172.21.15.146:6827/707256400","heartbeat_back_addr":"172.21.15.146:6831/707256400","heartbeat_front_addr":"172.21.15.146:6829/707256400","state":["exists","up"]},{"osd":11,"uuid":"7788c2de-6930-4808-be0f-f1fd1ae0cb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6825","nonce":2934336833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6827","nonce":2934336833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6831","nonce":2934336833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6829","nonce":2934336833}]},"public_addr":"172.21.15.152:6825/2934336833","cluster_addr":"172.21.15.152:6827/2934336833","heartbeat_back_addr":"172.21.15.152:6831/2934336833","heartbeat_front_addr":"172.21.15.152:6829/2934336833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:07.497448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:31.227494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:55.027978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:20.713437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:46.214704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:20:11.199983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:20:37.177070+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:02.951240+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:28.963680+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:56.095807+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:22:22.221744+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:22:48.929334+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/1372727546":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6801/1097124842":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6800/1508198215":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3103465990":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3540132960":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3899912190":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6800/1097124842":"2024-04-26T11:16:31.688159+0000","172.21.15.69:0/1199907202":"2024-04-26T11:16:04.860792+0000","172.21.15.69:6801/1508198215":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/2879102529":"2024-04-26T11:16:31.688159+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:22:53.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.864+0000 7f553103a700 1 -- 172.21.15.69:0/3793122050 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f55140afb80 msgr2=0x7f55140b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:53.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.864+0000 7f553103a700 1 --2- 172.21.15.69:0/3793122050 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f55140afb80 0x7f55140b2090 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5520005950 tx=0x7f552000a300 comp rx=0 tx=0).stop 2024-04-25T11:22:53.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.864+0000 7f553103a700 1 -- 172.21.15.69:0/3793122050 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f552c107ce0 msgr2=0x7f552c19bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:53.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.864+0000 7f553103a700 1 --2- 172.21.15.69:0/3793122050 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f552c107ce0 0x7f552c19bfb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f551c00b580 tx=0x7f551c004790 comp rx=0 tx=0).stop 2024-04-25T11:22:53.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.864+0000 7f553103a700 1 -- 172.21.15.69:0/3793122050 shutdown_connections 2024-04-25T11:22:53.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.864+0000 7f553103a700 1 --2- 172.21.15.69:0/3793122050 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f55140afb80 0x7f55140b2090 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:53.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.864+0000 7f553103a700 1 --2- 172.21.15.69:0/3793122050 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f552c108630 0x7f552c1a6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:53.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.864+0000 7f553103a700 1 --2- 172.21.15.69:0/3793122050 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f552c1033b0 0x7f552c19ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:53.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.864+0000 7f553103a700 1 --2- 172.21.15.69:0/3793122050 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f552c107ce0 0x7f552c19bfb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:53.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.864+0000 7f553103a700 1 -- 172.21.15.69:0/3793122050 >> 172.21.15.69:0/3793122050 conn(0x7f552c078cb0 msgr2=0x7f552c100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:53.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.864+0000 7f553103a700 1 -- 172.21.15.69:0/3793122050 shutdown_connections 2024-04-25T11:22:53.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:53.864+0000 7f553103a700 1 -- 172.21.15.69:0/3793122050 wait complete. 2024-04-25T11:22:53.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:53 smithi152 ceph-mon[32701]: Detected new or changed devices on smithi152 2024-04-25T11:22:53.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:53 smithi152 ceph-mon[32701]: Adjusting osd_memory_target on smithi152 to 4272M 2024-04-25T11:22:53.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:53 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:22:53.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:53 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:22:53.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:53 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:54.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:53 smithi146 ceph-mon[32821]: Detected new or changed devices on smithi152 2024-04-25T11:22:54.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:53 smithi146 ceph-mon[32821]: Adjusting osd_memory_target on smithi152 to 4272M 2024-04-25T11:22:54.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:53 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:22:54.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:53 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:22:54.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:53 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:54.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:53 smithi069 ceph-mon[29188]: Detected new or changed devices on smithi152 2024-04-25T11:22:54.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:53 smithi069 ceph-mon[29188]: Adjusting osd_memory_target on smithi152 to 4272M 2024-04-25T11:22:54.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:53 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:22:54.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:53 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:22:54.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:53 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:22:54.679 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:18:57.781263+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:22:54.679 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-25T11:22:54.917 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:22:54.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:54 smithi069 ceph-mon[29188]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:22:54.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:54 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/3793122050' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:22:54.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:54 smithi152 ceph-mon[32701]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:22:54.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:54 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/3793122050' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:22:55.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:54 smithi146 ceph-mon[32821]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:22:55.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:54 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/3793122050' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:22:56.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.666+0000 7f04ed3ed700 1 -- 172.21.15.69:0/3654438809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04e81091d0 msgr2=0x7f04e81095e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:56.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.666+0000 7f04ed3ed700 1 --2- 172.21.15.69:0/3654438809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04e81091d0 0x7f04e81095e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f04d8009990 tx=0x7f04d8009ca0 comp rx=0 tx=0).stop 2024-04-25T11:22:56.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.666+0000 7f04ed3ed700 1 -- 172.21.15.69:0/3654438809 shutdown_connections 2024-04-25T11:22:56.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.666+0000 7f04ed3ed700 1 --2- 172.21.15.69:0/3654438809 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04e8102fb0 0x7f04e8111040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:56.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.666+0000 7f04ed3ed700 1 --2- 172.21.15.69:0/3654438809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04e81091d0 0x7f04e81095e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:56.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.666+0000 7f04ed3ed700 1 --2- 172.21.15.69:0/3654438809 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04e8102660 0x7f04e8102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:56.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.666+0000 7f04ed3ed700 1 -- 172.21.15.69:0/3654438809 >> 172.21.15.69:0/3654438809 conn(0x7f04e80fe3f0 msgr2=0x7f04e8100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:56.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.666+0000 7f04ed3ed700 1 -- 172.21.15.69:0/3654438809 shutdown_connections 2024-04-25T11:22:56.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.666+0000 7f04ed3ed700 1 -- 172.21.15.69:0/3654438809 wait complete. 2024-04-25T11:22:56.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.667+0000 7f04ed3ed700 1 Processor -- start 2024-04-25T11:22:56.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.667+0000 7f04ed3ed700 1 -- start start 2024-04-25T11:22:56.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.667+0000 7f04ed3ed700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04e8102660 0x7f04e819ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:56.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.667+0000 7f04ed3ed700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04e8102fb0 0x7f04e819b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:56.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.667+0000 7f04ed3ed700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04e81091d0 0x7f04e819f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:56.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.667+0000 7f04ed3ed700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f04e8114380 con 0x7f04e81091d0 2024-04-25T11:22:56.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.667+0000 7f04ed3ed700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f04e8114200 con 0x7f04e8102fb0 2024-04-25T11:22:56.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.667+0000 7f04ed3ed700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f04e8114500 con 0x7f04e8102660 2024-04-25T11:22:56.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.667+0000 7f04e77fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04e81091d0 0x7f04e819f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:56.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.667+0000 7f04e77fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04e81091d0 0x7f04e819f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60270/0 (socket says 172.21.15.69:60270) 2024-04-25T11:22:56.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.667+0000 7f04e77fe700 1 -- 172.21.15.69:0/2968678527 learned_addr learned my addr 172.21.15.69:0/2968678527 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:22:56.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.668+0000 7f04de7fc700 1 --2- 172.21.15.69:0/2968678527 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04e8102fb0 0x7f04e819b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:56.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.668+0000 7f04e6ffd700 1 --2- 172.21.15.69:0/2968678527 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04e8102660 0x7f04e819ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:56.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.668+0000 7f04e77fe700 1 -- 172.21.15.69:0/2968678527 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04e8102660 msgr2=0x7f04e819ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:56.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.668+0000 7f04e77fe700 1 --2- 172.21.15.69:0/2968678527 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04e8102660 0x7f04e819ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:56.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.668+0000 7f04e77fe700 1 -- 172.21.15.69:0/2968678527 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04e8102fb0 msgr2=0x7f04e819b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:56.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.668+0000 7f04e77fe700 1 --2- 172.21.15.69:0/2968678527 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04e8102fb0 0x7f04e819b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:56.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.668+0000 7f04e77fe700 1 -- 172.21.15.69:0/2968678527 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e819fc40 con 0x7f04e81091d0 2024-04-25T11:22:56.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.668+0000 7f04de7fc700 1 --2- 172.21.15.69:0/2968678527 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04e8102fb0 0x7f04e819b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:22:56.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.668+0000 7f04e77fe700 1 --2- 172.21.15.69:0/2968678527 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04e81091d0 0x7f04e819f5a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f04e000ada0 tx=0x7f04e000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:56.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.668+0000 7f04e4ff9700 1 -- 172.21.15.69:0/2968678527 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e0017020 con 0x7f04e81091d0 2024-04-25T11:22:56.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.668+0000 7f04ed3ed700 1 -- 172.21.15.69:0/2968678527 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04e819ff30 con 0x7f04e81091d0 2024-04-25T11:22:56.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.668+0000 7f04e4ff9700 1 -- 172.21.15.69:0/2968678527 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f04e000ce10 con 0x7f04e81091d0 2024-04-25T11:22:56.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.668+0000 7f04e4ff9700 1 -- 172.21.15.69:0/2968678527 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e0005440 con 0x7f04e81091d0 2024-04-25T11:22:56.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.668+0000 7f04ed3ed700 1 -- 172.21.15.69:0/2968678527 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04e81a7850 con 0x7f04e81091d0 2024-04-25T11:22:56.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.669+0000 7f04ed3ed700 1 -- 172.21.15.69:0/2968678527 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04e8051190 con 0x7f04e81091d0 2024-04-25T11:22:56.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.671+0000 7f04e4ff9700 1 -- 172.21.15.69:0/2968678527 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f04e0004ad0 con 0x7f04e81091d0 2024-04-25T11:22:56.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.672+0000 7f04e4ff9700 1 --2- 172.21.15.69:0/2968678527 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f04d40afb80 0x7f04d40b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:56.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.672+0000 7f04e6ffd700 1 --2- 172.21.15.69:0/2968678527 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f04d40afb80 0x7f04d40b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:56.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.672+0000 7f04e4ff9700 1 -- 172.21.15.69:0/2968678527 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f04e00de560 con 0x7f04e81091d0 2024-04-25T11:22:56.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.672+0000 7f04e6ffd700 1 --2- 172.21.15.69:0/2968678527 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f04d40afb80 0x7f04d40b2090 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f04d8006010 tx=0x7f04d800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:56.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.676+0000 7f04e4ff9700 1 -- 172.21.15.69:0/2968678527 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f04e0021020 con 0x7f04e81091d0 2024-04-25T11:22:56.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.873+0000 7f04ed3ed700 1 -- 172.21.15.69:0/2968678527 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f04e802e510 con 0x7f04e81091d0 2024-04-25T11:22:56.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.874+0000 7f04e4ff9700 1 -- 172.21.15.69:0/2968678527 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7f04e00a7670 con 0x7f04e81091d0 2024-04-25T11:22:56.869 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-04-25T11:22:56.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.876+0000 7f04ed3ed700 1 -- 172.21.15.69:0/2968678527 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f04d40afb80 msgr2=0x7f04d40b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:56.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.876+0000 7f04ed3ed700 1 --2- 172.21.15.69:0/2968678527 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f04d40afb80 0x7f04d40b2090 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f04d8006010 tx=0x7f04d800b540 comp rx=0 tx=0).stop 2024-04-25T11:22:56.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.876+0000 7f04ed3ed700 1 -- 172.21.15.69:0/2968678527 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04e81091d0 msgr2=0x7f04e819f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:56.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.876+0000 7f04ed3ed700 1 --2- 172.21.15.69:0/2968678527 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04e81091d0 0x7f04e819f5a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f04e000ada0 tx=0x7f04e000c330 comp rx=0 tx=0).stop 2024-04-25T11:22:56.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.876+0000 7f04ed3ed700 1 -- 172.21.15.69:0/2968678527 shutdown_connections 2024-04-25T11:22:56.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.876+0000 7f04ed3ed700 1 --2- 172.21.15.69:0/2968678527 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f04d40afb80 0x7f04d40b2090 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:56.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.876+0000 7f04ed3ed700 1 --2- 172.21.15.69:0/2968678527 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04e81091d0 0x7f04e819f5a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:56.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.876+0000 7f04ed3ed700 1 --2- 172.21.15.69:0/2968678527 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04e8102660 0x7f04e819ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:56.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.876+0000 7f04ed3ed700 1 --2- 172.21.15.69:0/2968678527 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f04e8102fb0 0x7f04e819b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:56.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.876+0000 7f04ed3ed700 1 -- 172.21.15.69:0/2968678527 >> 172.21.15.69:0/2968678527 conn(0x7f04e80fe3f0 msgr2=0x7f04e81071a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:56.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.876+0000 7f04ed3ed700 1 -- 172.21.15.69:0/2968678527 shutdown_connections 2024-04-25T11:22:56.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:22:56.876+0000 7f04ed3ed700 1 -- 172.21.15.69:0/2968678527 wait complete. 2024-04-25T11:22:57.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:56 smithi146 ceph-mon[32821]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:22:57.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:56 smithi069 ceph-mon[29188]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:22:57.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:56 smithi152 ceph-mon[32701]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:22:57.700 INFO:tasks.cephadm:Adding mds.a on smithi069 2024-04-25T11:22:57.700 INFO:tasks.cephadm:Adding mds.d on smithi069 2024-04-25T11:22:57.700 INFO:tasks.cephadm:Adding mds.g on smithi069 2024-04-25T11:22:57.700 INFO:tasks.cephadm:Adding mds.j on smithi069 2024-04-25T11:22:57.701 INFO:tasks.cephadm:Adding mds.b on smithi146 2024-04-25T11:22:57.701 INFO:tasks.cephadm:Adding mds.e on smithi146 2024-04-25T11:22:57.701 INFO:tasks.cephadm:Adding mds.h on smithi146 2024-04-25T11:22:57.701 INFO:tasks.cephadm:Adding mds.k on smithi146 2024-04-25T11:22:57.701 INFO:tasks.cephadm:Adding mds.c on smithi152 2024-04-25T11:22:57.701 INFO:tasks.cephadm:Adding mds.f on smithi152 2024-04-25T11:22:57.701 INFO:tasks.cephadm:Adding mds.i on smithi152 2024-04-25T11:22:57.701 INFO:tasks.cephadm:Adding mds.l on smithi152 2024-04-25T11:22:57.701 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi146=b;smithi146=e;smithi146=h;smithi146=k;smithi152=c;smithi152=f;smithi152=i;smithi152=l' 2024-04-25T11:22:57.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:57 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2968678527' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:22:57.954 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T11:22:57.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:57 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/2968678527' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:22:58.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:57 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/2968678527' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:22:59.078 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:22:58 smithi152 ceph-mon[32701]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:22:59.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:22:58 smithi146 ceph-mon[32821]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:22:59.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:22:58 smithi069 ceph-mon[29188]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:22:59.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.618+0000 7f7e88573700 1 -- 172.21.15.152:0/3097756525 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e80107640 msgr2=0x7f7e80109a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:59.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.618+0000 7f7e88573700 1 --2- 172.21.15.152:0/3097756525 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e80107640 0x7f7e80109a60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7e74009990 tx=0x7f7e74009ca0 comp rx=0 tx=0).stop 2024-04-25T11:22:59.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.618+0000 7f7e88573700 1 -- 172.21.15.152:0/3097756525 shutdown_connections 2024-04-25T11:22:59.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.618+0000 7f7e88573700 1 --2- 172.21.15.152:0/3097756525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e80109fa0 0x7f7e8010c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:59.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.618+0000 7f7e88573700 1 --2- 172.21.15.152:0/3097756525 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e80104ce0 0x7f7e80107100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:59.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.618+0000 7f7e88573700 1 --2- 172.21.15.152:0/3097756525 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e80107640 0x7f7e80109a60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:59.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.618+0000 7f7e88573700 1 -- 172.21.15.152:0/3097756525 >> 172.21.15.152:0/3097756525 conn(0x7f7e800fe740 msgr2=0x7f7e80100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:59.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.618+0000 7f7e88573700 1 -- 172.21.15.152:0/3097756525 shutdown_connections 2024-04-25T11:22:59.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.618+0000 7f7e88573700 1 -- 172.21.15.152:0/3097756525 wait complete. 2024-04-25T11:22:59.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.619+0000 7f7e88573700 1 Processor -- start 2024-04-25T11:22:59.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.619+0000 7f7e88573700 1 -- start start 2024-04-25T11:22:59.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.619+0000 7f7e88573700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e80104ce0 0x7f7e8019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:59.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.619+0000 7f7e88573700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e80107640 0x7f7e8019abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:59.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.619+0000 7f7e88573700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e80109fa0 0x7f7e801a1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:59.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.619+0000 7f7e88573700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8010fb00 con 0x7f7e80104ce0 2024-04-25T11:22:59.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.619+0000 7f7e88573700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8010f980 con 0x7f7e80107640 2024-04-25T11:22:59.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.619+0000 7f7e88573700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8010fc80 con 0x7f7e80109fa0 2024-04-25T11:22:59.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.619+0000 7f7e86b10700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e80109fa0 0x7f7e801a1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:59.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.619+0000 7f7e86b10700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e80109fa0 0x7f7e801a1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41202/0 (socket says 172.21.15.152:41202) 2024-04-25T11:22:59.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.619+0000 7f7e86b10700 1 -- 172.21.15.152:0/1857904611 learned_addr learned my addr 172.21.15.152:0/1857904611 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-04-25T11:22:59.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.619+0000 7f7e8630f700 1 --2- 172.21.15.152:0/1857904611 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e80104ce0 0x7f7e8019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:59.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.620+0000 7f7e85b0e700 1 --2- 172.21.15.152:0/1857904611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e80107640 0x7f7e8019abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:59.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.620+0000 7f7e86b10700 1 -- 172.21.15.152:0/1857904611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e80107640 msgr2=0x7f7e8019abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:59.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.620+0000 7f7e86b10700 1 --2- 172.21.15.152:0/1857904611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e80107640 0x7f7e8019abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:59.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.620+0000 7f7e86b10700 1 -- 172.21.15.152:0/1857904611 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e80104ce0 msgr2=0x7f7e8019a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:59.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.620+0000 7f7e86b10700 1 --2- 172.21.15.152:0/1857904611 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e80104ce0 0x7f7e8019a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:59.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.620+0000 7f7e86b10700 1 -- 172.21.15.152:0/1857904611 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e801a2210 con 0x7f7e80109fa0 2024-04-25T11:22:59.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.620+0000 7f7e86b10700 1 --2- 172.21.15.152:0/1857904611 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e80109fa0 0x7f7e801a1b90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7e7800c970 tx=0x7f7e7800cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:59.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.620+0000 7f7e7f7fe700 1 -- 172.21.15.152:0/1857904611 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7800ce60 con 0x7f7e80109fa0 2024-04-25T11:22:59.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.620+0000 7f7e7f7fe700 1 -- 172.21.15.152:0/1857904611 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e78004830 con 0x7f7e80109fa0 2024-04-25T11:22:59.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.620+0000 7f7e7f7fe700 1 -- 172.21.15.152:0/1857904611 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e780053e0 con 0x7f7e80109fa0 2024-04-25T11:22:59.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.620+0000 7f7e88573700 1 -- 172.21.15.152:0/1857904611 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e801a2500 con 0x7f7e80109fa0 2024-04-25T11:22:59.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.620+0000 7f7e88573700 1 -- 172.21.15.152:0/1857904611 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e801a2a40 con 0x7f7e80109fa0 2024-04-25T11:22:59.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.621+0000 7f7e88573700 1 -- 172.21.15.152:0/1857904611 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e80051190 con 0x7f7e80109fa0 2024-04-25T11:22:59.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.623+0000 7f7e7f7fe700 1 -- 172.21.15.152:0/1857904611 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f7e78007b00 con 0x7f7e80109fa0 2024-04-25T11:22:59.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.624+0000 7f7e7f7fe700 1 --2- 172.21.15.152:0/1857904611 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f7e680afc00 0x7f7e680b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:59.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.624+0000 7f7e7f7fe700 1 -- 172.21.15.152:0/1857904611 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f7e780de6e0 con 0x7f7e80109fa0 2024-04-25T11:22:59.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.624+0000 7f7e8630f700 1 --2- 172.21.15.152:0/1857904611 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f7e680afc00 0x7f7e680b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:59.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.624+0000 7f7e8630f700 1 --2- 172.21.15.152:0/1857904611 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f7e680afc00 0x7f7e680b2110 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7e70009550 tx=0x7f7e700091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:59.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.628+0000 7f7e7f7fe700 1 -- 172.21.15.152:0/1857904611 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7e78014030 con 0x7f7e80109fa0 2024-04-25T11:22:59.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.835+0000 7f7e88573700 1 -- 172.21.15.152:0/1857904611 --> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi146=b;smithi146=e;smithi146=h;smithi146=k;smithi152=c;smithi152=f;smithi152=i;smithi152=l", "target": ["mon-mgr", ""]}) -- 0x7f7e80063bd0 con 0x7f7e680afc00 2024-04-25T11:22:59.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.843+0000 7f7e7f7fe700 1 -- 172.21.15.152:0/1857904611 <== mgr.14152 v2:172.21.15.69:6800/4220065409 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f7e80063bd0 con 0x7f7e680afc00 2024-04-25T11:22:59.838 INFO:teuthology.orchestra.run.smithi152.stdout:Scheduled mds.all update... 2024-04-25T11:22:59.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.845+0000 7f7e88573700 1 -- 172.21.15.152:0/1857904611 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f7e680afc00 msgr2=0x7f7e680b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:59.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.845+0000 7f7e88573700 1 --2- 172.21.15.152:0/1857904611 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f7e680afc00 0x7f7e680b2110 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7e70009550 tx=0x7f7e700091b0 comp rx=0 tx=0).stop 2024-04-25T11:22:59.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.845+0000 7f7e88573700 1 -- 172.21.15.152:0/1857904611 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e80109fa0 msgr2=0x7f7e801a1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:59.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.845+0000 7f7e88573700 1 --2- 172.21.15.152:0/1857904611 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e80109fa0 0x7f7e801a1b90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7e7800c970 tx=0x7f7e7800cc80 comp rx=0 tx=0).stop 2024-04-25T11:22:59.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.845+0000 7f7e88573700 1 -- 172.21.15.152:0/1857904611 shutdown_connections 2024-04-25T11:22:59.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.845+0000 7f7e88573700 1 --2- 172.21.15.152:0/1857904611 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f7e680afc00 0x7f7e680b2110 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:59.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.845+0000 7f7e88573700 1 --2- 172.21.15.152:0/1857904611 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e80109fa0 0x7f7e801a1b90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:59.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.845+0000 7f7e88573700 1 --2- 172.21.15.152:0/1857904611 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e80104ce0 0x7f7e8019a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:59.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.845+0000 7f7e88573700 1 --2- 172.21.15.152:0/1857904611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e80107640 0x7f7e8019abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:59.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.845+0000 7f7e88573700 1 -- 172.21.15.152:0/1857904611 >> 172.21.15.152:0/1857904611 conn(0x7f7e800fe740 msgr2=0x7f7e801057b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:59.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.845+0000 7f7e88573700 1 -- 172.21.15.152:0/1857904611 shutdown_connections 2024-04-25T11:22:59.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-04-25T11:22:59.845+0000 7f7e88573700 1 -- 172.21.15.152:0/1857904611 wait complete. 2024-04-25T11:23:00.531 DEBUG:teuthology.orchestra.run.smithi069:mds.a> sudo journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mds.a.service 2024-04-25T11:23:00.534 DEBUG:teuthology.orchestra.run.smithi069:mds.d> sudo journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mds.d.service 2024-04-25T11:23:00.576 DEBUG:teuthology.orchestra.run.smithi069:mds.g> sudo journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mds.g.service 2024-04-25T11:23:00.619 DEBUG:teuthology.orchestra.run.smithi069:mds.j> sudo journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mds.j.service 2024-04-25T11:23:00.662 DEBUG:teuthology.orchestra.run.smithi146:mds.b> sudo journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mds.b.service 2024-04-25T11:23:00.665 DEBUG:teuthology.orchestra.run.smithi146:mds.e> sudo journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mds.e.service 2024-04-25T11:23:00.709 DEBUG:teuthology.orchestra.run.smithi146:mds.h> sudo journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mds.h.service 2024-04-25T11:23:00.751 DEBUG:teuthology.orchestra.run.smithi146:mds.k> sudo journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mds.k.service 2024-04-25T11:23:00.794 DEBUG:teuthology.orchestra.run.smithi152:mds.c> sudo journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mds.c.service 2024-04-25T11:23:00.797 DEBUG:teuthology.orchestra.run.smithi152:mds.f> sudo journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mds.f.service 2024-04-25T11:23:00.839 DEBUG:teuthology.orchestra.run.smithi152:mds.i> sudo journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mds.i.service 2024-04-25T11:23:00.882 DEBUG:teuthology.orchestra.run.smithi152:mds.l> sudo journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mds.l.service 2024-04-25T11:23:00.925 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-25T11:23:00.926 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:23:01.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.056+0000 7eff50e92640 1 -- 172.21.15.69:0/1371082319 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff4c103c40 msgr2=0x7eff4c104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:01.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.056+0000 7eff50e92640 1 --2- 172.21.15.69:0/1371082319 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff4c103c40 0x7eff4c104050 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7eff34009900 tx=0x7eff3402a410 comp rx=0 tx=0).stop 2024-04-25T11:23:01.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.056+0000 7eff50e92640 1 -- 172.21.15.69:0/1371082319 shutdown_connections 2024-04-25T11:23:01.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.056+0000 7eff50e92640 1 --2- 172.21.15.69:0/1371082319 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff4c108940 0x7eff4c10ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:01.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.056+0000 7eff50e92640 1 --2- 172.21.15.69:0/1371082319 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff4c103c40 0x7eff4c104050 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:01.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.056+0000 7eff50e92640 1 --2- 172.21.15.69:0/1371082319 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff4c102a00 0x7eff4c103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:01.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.056+0000 7eff50e92640 1 -- 172.21.15.69:0/1371082319 >> 172.21.15.69:0/1371082319 conn(0x7eff4c0fdbf0 msgr2=0x7eff4c1000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:01.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.056+0000 7eff50e92640 1 -- 172.21.15.69:0/1371082319 shutdown_connections 2024-04-25T11:23:01.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.056+0000 7eff50e92640 1 -- 172.21.15.69:0/1371082319 wait complete. 2024-04-25T11:23:01.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.056+0000 7eff50e92640 1 Processor -- start 2024-04-25T11:23:01.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.056+0000 7eff50e92640 1 -- start start 2024-04-25T11:23:01.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.057+0000 7eff50e92640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff4c102a00 0x7eff4c19a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:01.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.057+0000 7eff50e92640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff4c103c40 0x7eff4c19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:01.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.057+0000 7eff50e92640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff4c108940 0x7eff4c1a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:01.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.057+0000 7eff50e92640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7eff4c10f4e0 con 0x7eff4c103c40 2024-04-25T11:23:01.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.057+0000 7eff50e92640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7eff4c10f360 con 0x7eff4c108940 2024-04-25T11:23:01.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.057+0000 7eff50e92640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7eff4c10f660 con 0x7eff4c102a00 2024-04-25T11:23:01.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.057+0000 7eff43fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff4c103c40 0x7eff4c19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:01.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.057+0000 7eff43fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff4c103c40 0x7eff4c19a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60290/0 (socket says 172.21.15.69:60290) 2024-04-25T11:23:01.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.057+0000 7eff43fff640 1 -- 172.21.15.69:0/1931788695 learned_addr learned my addr 172.21.15.69:0/1931788695 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:01.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.057+0000 7eff4a59c640 1 --2- 172.21.15.69:0/1931788695 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff4c102a00 0x7eff4c19a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:01.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.057+0000 7eff4ad9d640 1 --2- 172.21.15.69:0/1931788695 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff4c108940 0x7eff4c1a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:01.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.057+0000 7eff43fff640 1 -- 172.21.15.69:0/1931788695 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff4c102a00 msgr2=0x7eff4c19a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:01.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.057+0000 7eff43fff640 1 --2- 172.21.15.69:0/1931788695 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff4c102a00 0x7eff4c19a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:01.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.057+0000 7eff43fff640 1 -- 172.21.15.69:0/1931788695 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff4c108940 msgr2=0x7eff4c1a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:01.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.057+0000 7eff43fff640 1 --2- 172.21.15.69:0/1931788695 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff4c108940 0x7eff4c1a15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:01.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.057+0000 7eff43fff640 1 -- 172.21.15.69:0/1931788695 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff4c1a1c20 con 0x7eff4c103c40 2024-04-25T11:23:01.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.057+0000 7eff43fff640 1 --2- 172.21.15.69:0/1931788695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff4c103c40 0x7eff4c19a620 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7eff34009230 tx=0x7eff340040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:01.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.057+0000 7eff437fe640 1 -- 172.21.15.69:0/1931788695 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3403f030 con 0x7eff4c103c40 2024-04-25T11:23:01.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.057+0000 7eff437fe640 1 -- 172.21.15.69:0/1931788695 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff340044c0 con 0x7eff4c103c40 2024-04-25T11:23:01.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.057+0000 7eff437fe640 1 -- 172.21.15.69:0/1931788695 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff34035070 con 0x7eff4c103c40 2024-04-25T11:23:01.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.057+0000 7eff50e92640 1 -- 172.21.15.69:0/1931788695 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff4c1a1eb0 con 0x7eff4c103c40 2024-04-25T11:23:01.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.058+0000 7eff50e92640 1 -- 172.21.15.69:0/1931788695 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff4c1a2310 con 0x7eff4c103c40 2024-04-25T11:23:01.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.058+0000 7eff50e92640 1 -- 172.21.15.69:0/1931788695 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff4c107750 con 0x7eff4c103c40 2024-04-25T11:23:01.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.061+0000 7eff437fe640 1 -- 172.21.15.69:0/1931788695 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7eff34042030 con 0x7eff4c103c40 2024-04-25T11:23:01.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.061+0000 7eff437fe640 1 --2- 172.21.15.69:0/1931788695 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7eff380afb70 0x7eff380b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:01.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.062+0000 7eff4a59c640 1 --2- 172.21.15.69:0/1931788695 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7eff380afb70 0x7eff380b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:01.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.062+0000 7eff437fe640 1 -- 172.21.15.69:0/1931788695 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7eff340fefe0 con 0x7eff4c103c40 2024-04-25T11:23:01.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.062+0000 7eff4a59c640 1 --2- 172.21.15.69:0/1931788695 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7eff380afb70 0x7eff380b2080 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7eff3c005fd0 tx=0x7eff3c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:01.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.065+0000 7eff437fe640 1 -- 172.21.15.69:0/1931788695 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7eff340c7a10 con 0x7eff4c103c40 2024-04-25T11:23:01.075 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:00 smithi152 ceph-mon[32701]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:01.075 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:00 smithi152 ceph-mon[32701]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi146=b;smithi146=e;smithi146=h;smithi146=k;smithi152=c;smithi152=f;smithi152=i;smithi152=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:23:01.075 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:00 smithi152 ceph-mon[32701]: Saving service mds.all spec with placement smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi146=b;smithi146=e;smithi146=h;smithi146=k;smithi152=c;smithi152=f;smithi152=i;smithi152=l;count:12 2024-04-25T11:23:01.075 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:00 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:01.075 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:00 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:23:01.076 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:00 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:23:01.076 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:00 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:23:01.076 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:00 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:01.076 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:00 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:23:01.076 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:00 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:23:01.076 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:00 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:23:01.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:00 smithi146 ceph-mon[32821]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:01.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:00 smithi146 ceph-mon[32821]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi146=b;smithi146=e;smithi146=h;smithi146=k;smithi152=c;smithi152=f;smithi152=i;smithi152=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:23:01.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:00 smithi146 ceph-mon[32821]: Saving service mds.all spec with placement smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi146=b;smithi146=e;smithi146=h;smithi146=k;smithi152=c;smithi152=f;smithi152=i;smithi152=l;count:12 2024-04-25T11:23:01.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:00 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:01.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:00 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:23:01.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:00 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:23:01.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:00 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:23:01.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:00 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:01.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:00 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:23:01.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:00 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:23:01.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:00 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:23:01.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:00 smithi069 ceph-mon[29188]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:01.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:00 smithi069 ceph-mon[29188]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi146=b;smithi146=e;smithi146=h;smithi146=k;smithi152=c;smithi152=f;smithi152=i;smithi152=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:23:01.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:00 smithi069 ceph-mon[29188]: Saving service mds.all spec with placement smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi146=b;smithi146=e;smithi146=h;smithi146=k;smithi152=c;smithi152=f;smithi152=i;smithi152=l;count:12 2024-04-25T11:23:01.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:00 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:01.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:00 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:23:01.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:00 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:23:01.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:00 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:23:01.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:00 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:01.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:00 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:23:01.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:00 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:23:01.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:00 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:23:01.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.246+0000 7eff50e92640 1 -- 172.21.15.69:0/1931788695 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7eff4c0637d0 con 0x7eff4c103c40 2024-04-25T11:23:01.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.247+0000 7eff437fe640 1 -- 172.21.15.69:0/1931788695 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+18458 (secure 0 0 0) 0x7eff340d00e0 con 0x7eff4c103c40 2024-04-25T11:23:01.242 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:23:01.242 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":71,"fsid":"f7ed73dc-02f4-11ef-bc93-c7b262605968","created":"2024-04-25T11:15:37.965281+0000","modified":"2024-04-25T11:22:51.698858+0000","last_up_change":"2024-04-25T11:22:50.691869+0000","last_in_change":"2024-04-25T11:22:30.271959+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:18:57.781263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a6a0d179-4998-4754-bf82-2ea3dac54e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6803","nonce":3756025235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6805","nonce":3756025235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6809","nonce":3756025235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6807","nonce":3756025235}]},"public_addr":"172.21.15.69:6803/3756025235","cluster_addr":"172.21.15.69:6805/3756025235","heartbeat_back_addr":"172.21.15.69:6809/3756025235","heartbeat_front_addr":"172.21.15.69:6807/3756025235","state":["exists","up"]},{"osd":1,"uuid":"c0c66930-4233-493f-b2bb-99c473994b59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6801","nonce":3166305213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6803","nonce":3166305213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6807","nonce":3166305213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6805","nonce":3166305213}]},"public_addr":"172.21.15.146:6801/3166305213","cluster_addr":"172.21.15.146:6803/3166305213","heartbeat_back_addr":"172.21.15.146:6807/3166305213","heartbeat_front_addr":"172.21.15.146:6805/3166305213","state":["exists","up"]},{"osd":2,"uuid":"d4ef735b-9786-4cb9-83ef-7c3f0691a729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6801","nonce":1474319175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6803","nonce":1474319175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6807","nonce":1474319175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6805","nonce":1474319175}]},"public_addr":"172.21.15.152:6801/1474319175","cluster_addr":"172.21.15.152:6803/1474319175","heartbeat_back_addr":"172.21.15.152:6807/1474319175","heartbeat_front_addr":"172.21.15.152:6805/1474319175","state":["exists","up"]},{"osd":3,"uuid":"0ab85fb8-5bbb-4ffc-bddf-cae7644a64ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6811","nonce":1809071296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6813","nonce":1809071296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6817","nonce":1809071296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6815","nonce":1809071296}]},"public_addr":"172.21.15.69:6811/1809071296","cluster_addr":"172.21.15.69:6813/1809071296","heartbeat_back_addr":"172.21.15.69:6817/1809071296","heartbeat_front_addr":"172.21.15.69:6815/1809071296","state":["exists","up"]},{"osd":4,"uuid":"273e0592-bbbc-4188-8eeb-46a3879cf632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6809","nonce":1898520972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6811","nonce":1898520972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6815","nonce":1898520972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6813","nonce":1898520972}]},"public_addr":"172.21.15.146:6809/1898520972","cluster_addr":"172.21.15.146:6811/1898520972","heartbeat_back_addr":"172.21.15.146:6815/1898520972","heartbeat_front_addr":"172.21.15.146:6813/1898520972","state":["exists","up"]},{"osd":5,"uuid":"bbeae945-9fa2-48f9-a1d6-8bf18dcaef19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6809","nonce":1996555108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6811","nonce":1996555108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6815","nonce":1996555108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6813","nonce":1996555108}]},"public_addr":"172.21.15.152:6809/1996555108","cluster_addr":"172.21.15.152:6811/1996555108","heartbeat_back_addr":"172.21.15.152:6815/1996555108","heartbeat_front_addr":"172.21.15.152:6813/1996555108","state":["exists","up"]},{"osd":6,"uuid":"dbb61253-30bc-418e-bd75-118c80bbf931","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6819","nonce":491215762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6821","nonce":491215762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6825","nonce":491215762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6823","nonce":491215762}]},"public_addr":"172.21.15.69:6819/491215762","cluster_addr":"172.21.15.69:6821/491215762","heartbeat_back_addr":"172.21.15.69:6825/491215762","heartbeat_front_addr":"172.21.15.69:6823/491215762","state":["exists","up"]},{"osd":7,"uuid":"75f23854-e283-4e3e-94cf-7b4079d83876","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6817","nonce":2330294690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6819","nonce":2330294690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6823","nonce":2330294690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6821","nonce":2330294690}]},"public_addr":"172.21.15.146:6817/2330294690","cluster_addr":"172.21.15.146:6819/2330294690","heartbeat_back_addr":"172.21.15.146:6823/2330294690","heartbeat_front_addr":"172.21.15.146:6821/2330294690","state":["exists","up"]},{"osd":8,"uuid":"1aa1060c-0c14-4abd-be56-5ebfeb78759c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6817","nonce":950669381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6819","nonce":950669381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6823","nonce":950669381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6821","nonce":950669381}]},"public_addr":"172.21.15.152:6817/950669381","cluster_addr":"172.21.15.152:6819/950669381","heartbeat_back_addr":"172.21.15.152:6823/950669381","heartbeat_front_addr":"172.21.15.152:6821/950669381","state":["exists","up"]},{"osd":9,"uuid":"3f4bf800-4090-4138-9bb3-d001f13f450c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6827","nonce":2414186758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6829","nonce":2414186758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6833","nonce":2414186758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6831","nonce":2414186758}]},"public_addr":"172.21.15.69:6827/2414186758","cluster_addr":"172.21.15.69:6829/2414186758","heartbeat_back_addr":"172.21.15.69:6833/2414186758","heartbeat_front_addr":"172.21.15.69:6831/2414186758","state":["exists","up"]},{"osd":10,"uuid":"19e967e5-51ce-4506-80c9-0a95441baf99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6825","nonce":707256400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6827","nonce":707256400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6831","nonce":707256400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6829","nonce":707256400}]},"public_addr":"172.21.15.146:6825/707256400","cluster_addr":"172.21.15.146:6827/707256400","heartbeat_back_addr":"172.21.15.146:6831/707256400","heartbeat_front_addr":"172.21.15.146:6829/707256400","state":["exists","up"]},{"osd":11,"uuid":"7788c2de-6930-4808-be0f-f1fd1ae0cb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6825","nonce":2934336833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6827","nonce":2934336833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6831","nonce":2934336833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6829","nonce":2934336833}]},"public_addr":"172.21.15.152:6825/2934336833","cluster_addr":"172.21.15.152:6827/2934336833","heartbeat_back_addr":"172.21.15.152:6831/2934336833","heartbeat_front_addr":"172.21.15.152:6829/2934336833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:07.497448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:31.227494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:55.027978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:20.713437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:46.214704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:20:11.199983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:20:37.177070+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:02.951240+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:28.963680+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:56.095807+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:22:22.221744+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:22:48.929334+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/1372727546":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6801/1097124842":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6800/1508198215":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3103465990":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3540132960":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3899912190":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6800/1097124842":"2024-04-26T11:16:31.688159+0000","172.21.15.69:0/1199907202":"2024-04-26T11:16:04.860792+0000","172.21.15.69:6801/1508198215":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/2879102529":"2024-04-26T11:16:31.688159+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:23:01.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.249+0000 7eff50e92640 1 -- 172.21.15.69:0/1931788695 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7eff380afb70 msgr2=0x7eff380b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:01.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.249+0000 7eff50e92640 1 --2- 172.21.15.69:0/1931788695 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7eff380afb70 0x7eff380b2080 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7eff3c005fd0 tx=0x7eff3c00b410 comp rx=0 tx=0).stop 2024-04-25T11:23:01.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.249+0000 7eff50e92640 1 -- 172.21.15.69:0/1931788695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff4c103c40 msgr2=0x7eff4c19a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:01.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.249+0000 7eff50e92640 1 --2- 172.21.15.69:0/1931788695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff4c103c40 0x7eff4c19a620 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7eff34009230 tx=0x7eff340040c0 comp rx=0 tx=0).stop 2024-04-25T11:23:01.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.249+0000 7eff50e92640 1 -- 172.21.15.69:0/1931788695 shutdown_connections 2024-04-25T11:23:01.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.249+0000 7eff50e92640 1 --2- 172.21.15.69:0/1931788695 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7eff380afb70 0x7eff380b2080 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:01.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.249+0000 7eff50e92640 1 --2- 172.21.15.69:0/1931788695 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff4c108940 0x7eff4c1a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:01.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.249+0000 7eff50e92640 1 --2- 172.21.15.69:0/1931788695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff4c103c40 0x7eff4c19a620 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:01.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.249+0000 7eff50e92640 1 --2- 172.21.15.69:0/1931788695 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff4c102a00 0x7eff4c19a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:01.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.249+0000 7eff50e92640 1 -- 172.21.15.69:0/1931788695 >> 172.21.15.69:0/1931788695 conn(0x7eff4c0fdbf0 msgr2=0x7eff4c104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:01.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.249+0000 7eff50e92640 1 -- 172.21.15.69:0/1931788695 shutdown_connections 2024-04-25T11:23:01.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.249+0000 7eff50e92640 1 -- 172.21.15.69:0/1931788695 wait complete. 2024-04-25T11:23:01.255 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:18:57.781263+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:23:01.255 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:23:01.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.371+0000 7f8d283b7640 1 -- 172.21.15.69:0/3219810622 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d20103af0 msgr2=0x7f8d201047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:01.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.371+0000 7f8d283b7640 1 --2- 172.21.15.69:0/3219810622 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d20103af0 0x7f8d201047f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c009ca0 tx=0x7f8d1c02a8c0 comp rx=0 tx=0).stop 2024-04-25T11:23:01.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.371+0000 7f8d283b7640 1 -- 172.21.15.69:0/3219810622 shutdown_connections 2024-04-25T11:23:01.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.371+0000 7f8d283b7640 1 --2- 172.21.15.69:0/3219810622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d20109a30 0x7f8d2010be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:01.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.371+0000 7f8d283b7640 1 --2- 172.21.15.69:0/3219810622 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d20104d30 0x7f8d20105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:01.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.371+0000 7f8d283b7640 1 --2- 172.21.15.69:0/3219810622 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d20103af0 0x7f8d201047f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:01.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.371+0000 7f8d283b7640 1 -- 172.21.15.69:0/3219810622 >> 172.21.15.69:0/3219810622 conn(0x7f8d200fece0 msgr2=0x7f8d20101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:01.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.371+0000 7f8d283b7640 1 -- 172.21.15.69:0/3219810622 shutdown_connections 2024-04-25T11:23:01.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.371+0000 7f8d283b7640 1 -- 172.21.15.69:0/3219810622 wait complete. 2024-04-25T11:23:01.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.372+0000 7f8d283b7640 1 Processor -- start 2024-04-25T11:23:01.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.372+0000 7f8d283b7640 1 -- start start 2024-04-25T11:23:01.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.372+0000 7f8d283b7640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d20103af0 0x7f8d2019a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:01.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.372+0000 7f8d283b7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d20104d30 0x7f8d2019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:01.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.372+0000 7f8d283b7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d20109a30 0x7f8d201a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:01.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.372+0000 7f8d283b7640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2010f4f0 con 0x7f8d20104d30 2024-04-25T11:23:01.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.372+0000 7f8d283b7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2010f370 con 0x7f8d20109a30 2024-04-25T11:23:01.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.372+0000 7f8d283b7640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2010f670 con 0x7f8d20103af0 2024-04-25T11:23:01.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.372+0000 7f8d25952640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d20104d30 0x7f8d2019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:01.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.372+0000 7f8d25952640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d20104d30 0x7f8d2019a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60312/0 (socket says 172.21.15.69:60312) 2024-04-25T11:23:01.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.372+0000 7f8d25952640 1 -- 172.21.15.69:0/234549263 learned_addr learned my addr 172.21.15.69:0/234549263 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:01.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.372+0000 7f8d26153640 1 --2- 172.21.15.69:0/234549263 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d20103af0 0x7f8d2019a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:01.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.372+0000 7f8d26954640 1 --2- 172.21.15.69:0/234549263 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d20109a30 0x7f8d201a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:01.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.372+0000 7f8d25952640 1 -- 172.21.15.69:0/234549263 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d20103af0 msgr2=0x7f8d2019a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:01.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.372+0000 7f8d25952640 1 --2- 172.21.15.69:0/234549263 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d20103af0 0x7f8d2019a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:01.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.372+0000 7f8d25952640 1 -- 172.21.15.69:0/234549263 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d20109a30 msgr2=0x7f8d201a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:01.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.372+0000 7f8d25952640 1 --2- 172.21.15.69:0/234549263 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d20109a30 0x7f8d201a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:01.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.372+0000 7f8d25952640 1 -- 172.21.15.69:0/234549263 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d201a1cc0 con 0x7f8d20104d30 2024-04-25T11:23:01.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.372+0000 7f8d26954640 1 --2- 172.21.15.69:0/234549263 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d20109a30 0x7f8d201a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:23:01.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.372+0000 7f8d25952640 1 --2- 172.21.15.69:0/234549263 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d20104d30 0x7f8d2019a6c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8d1000e9f0 tx=0x7f8d1000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:01.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.372+0000 7f8d177fe640 1 -- 172.21.15.69:0/234549263 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1000cd80 con 0x7f8d20104d30 2024-04-25T11:23:01.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.373+0000 7f8d177fe640 1 -- 172.21.15.69:0/234549263 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8d100054b0 con 0x7f8d20104d30 2024-04-25T11:23:01.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.373+0000 7f8d283b7640 1 -- 172.21.15.69:0/234549263 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d201a1fb0 con 0x7f8d20104d30 2024-04-25T11:23:01.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.373+0000 7f8d177fe640 1 -- 172.21.15.69:0/234549263 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d10010430 con 0x7f8d20104d30 2024-04-25T11:23:01.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.373+0000 7f8d283b7640 1 -- 172.21.15.69:0/234549263 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d201a23c0 con 0x7f8d20104d30 2024-04-25T11:23:01.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.374+0000 7f8d283b7640 1 -- 172.21.15.69:0/234549263 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d20108840 con 0x7f8d20104d30 2024-04-25T11:23:01.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.375+0000 7f8d177fe640 1 -- 172.21.15.69:0/234549263 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f8d100046c0 con 0x7f8d20104d30 2024-04-25T11:23:01.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.375+0000 7f8d177fe640 1 --2- 172.21.15.69:0/234549263 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f8d0c0afaf0 0x7f8d0c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:01.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.376+0000 7f8d26153640 1 --2- 172.21.15.69:0/234549263 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f8d0c0afaf0 0x7f8d0c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:01.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.376+0000 7f8d177fe640 1 -- 172.21.15.69:0/234549263 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f8d100ded60 con 0x7f8d20104d30 2024-04-25T11:23:01.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.376+0000 7f8d26153640 1 --2- 172.21.15.69:0/234549263 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f8d0c0afaf0 0x7f8d0c0b2000 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c006010 tx=0x7f8d1c0318e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:01.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.379+0000 7f8d177fe640 1 -- 172.21.15.69:0/234549263 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8d10053ff0 con 0x7f8d20104d30 2024-04-25T11:23:01.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.554+0000 7f8d283b7640 1 -- 172.21.15.69:0/234549263 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8d200637d0 con 0x7f8d20104d30 2024-04-25T11:23:01.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.555+0000 7f8d177fe640 1 -- 172.21.15.69:0/234549263 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7f8d100a7dc0 con 0x7f8d20104d30 2024-04-25T11:23:01.550 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-04-25T11:23:01.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.557+0000 7f8d283b7640 1 -- 172.21.15.69:0/234549263 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f8d0c0afaf0 msgr2=0x7f8d0c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:01.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.557+0000 7f8d283b7640 1 --2- 172.21.15.69:0/234549263 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f8d0c0afaf0 0x7f8d0c0b2000 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c006010 tx=0x7f8d1c0318e0 comp rx=0 tx=0).stop 2024-04-25T11:23:01.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.557+0000 7f8d283b7640 1 -- 172.21.15.69:0/234549263 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d20104d30 msgr2=0x7f8d2019a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:01.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.557+0000 7f8d283b7640 1 --2- 172.21.15.69:0/234549263 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d20104d30 0x7f8d2019a6c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8d1000e9f0 tx=0x7f8d1000edf0 comp rx=0 tx=0).stop 2024-04-25T11:23:01.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.557+0000 7f8d283b7640 1 -- 172.21.15.69:0/234549263 shutdown_connections 2024-04-25T11:23:01.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.557+0000 7f8d283b7640 1 --2- 172.21.15.69:0/234549263 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f8d0c0afaf0 0x7f8d0c0b2000 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:01.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.557+0000 7f8d283b7640 1 --2- 172.21.15.69:0/234549263 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d20109a30 0x7f8d201a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:01.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.557+0000 7f8d283b7640 1 --2- 172.21.15.69:0/234549263 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d20104d30 0x7f8d2019a6c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:01.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.557+0000 7f8d283b7640 1 --2- 172.21.15.69:0/234549263 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d20103af0 0x7f8d2019a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:01.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.557+0000 7f8d283b7640 1 -- 172.21.15.69:0/234549263 >> 172.21.15.69:0/234549263 conn(0x7f8d200fece0 msgr2=0x7f8d20105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:01.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.557+0000 7f8d283b7640 1 -- 172.21.15.69:0/234549263 shutdown_connections 2024-04-25T11:23:01.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.557+0000 7f8d283b7640 1 -- 172.21.15.69:0/234549263 wait complete. 2024-04-25T11:23:01.563 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:23:01.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.675+0000 7f59b017a640 1 -- 172.21.15.69:0/459915383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59a8108900 msgr2=0x7f59a810ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:01.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.675+0000 7f59b017a640 1 --2- 172.21.15.69:0/459915383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59a8108900 0x7f59a810ad20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f59a0009ca0 tx=0x7f59a002a8c0 comp rx=0 tx=0).stop 2024-04-25T11:23:01.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.675+0000 7f59b017a640 1 -- 172.21.15.69:0/459915383 shutdown_connections 2024-04-25T11:23:01.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.675+0000 7f59b017a640 1 --2- 172.21.15.69:0/459915383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59a8108900 0x7f59a810ad20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:01.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.675+0000 7f59b017a640 1 --2- 172.21.15.69:0/459915383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59a8103c00 0x7f59a8104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:01.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.675+0000 7f59b017a640 1 --2- 172.21.15.69:0/459915383 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59a81029c0 0x7f59a81036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:01.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.675+0000 7f59b017a640 1 -- 172.21.15.69:0/459915383 >> 172.21.15.69:0/459915383 conn(0x7f59a80fdbd0 msgr2=0x7f59a8100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:01.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.675+0000 7f59b017a640 1 -- 172.21.15.69:0/459915383 shutdown_connections 2024-04-25T11:23:01.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.675+0000 7f59b017a640 1 -- 172.21.15.69:0/459915383 wait complete. 2024-04-25T11:23:01.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.675+0000 7f59b017a640 1 Processor -- start 2024-04-25T11:23:01.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.675+0000 7f59b017a640 1 -- start start 2024-04-25T11:23:01.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.676+0000 7f59b017a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59a81029c0 0x7f59a8198ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:01.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.676+0000 7f59b017a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59a8103c00 0x7f59a8199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:01.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.676+0000 7f59b017a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59a8108900 0x7f59a81a04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:01.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.676+0000 7f59b017a640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f59a810e3f0 con 0x7f59a81029c0 2024-04-25T11:23:01.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.676+0000 7f59b017a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f59a810e270 con 0x7f59a8103c00 2024-04-25T11:23:01.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.676+0000 7f59b017a640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f59a810e570 con 0x7f59a8108900 2024-04-25T11:23:01.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.676+0000 7f59adf16640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59a81029c0 0x7f59a8198ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:01.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.676+0000 7f59adf16640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59a81029c0 0x7f59a8198ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60320/0 (socket says 172.21.15.69:60320) 2024-04-25T11:23:01.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.676+0000 7f59adf16640 1 -- 172.21.15.69:0/2907089864 learned_addr learned my addr 172.21.15.69:0/2907089864 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:01.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.676+0000 7f59ae717640 1 --2- 172.21.15.69:0/2907089864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59a8108900 0x7f59a81a04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:01.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.676+0000 7f59ad715640 1 --2- 172.21.15.69:0/2907089864 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59a8103c00 0x7f59a8199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:01.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.676+0000 7f59adf16640 1 -- 172.21.15.69:0/2907089864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59a8108900 msgr2=0x7f59a81a04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:01.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.676+0000 7f59adf16640 1 --2- 172.21.15.69:0/2907089864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59a8108900 0x7f59a81a04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:01.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.676+0000 7f59adf16640 1 -- 172.21.15.69:0/2907089864 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59a8103c00 msgr2=0x7f59a8199530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:01.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.676+0000 7f59adf16640 1 --2- 172.21.15.69:0/2907089864 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59a8103c00 0x7f59a8199530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:01.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.676+0000 7f59adf16640 1 -- 172.21.15.69:0/2907089864 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59a81a0b30 con 0x7f59a81029c0 2024-04-25T11:23:01.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.676+0000 7f59ae717640 1 --2- 172.21.15.69:0/2907089864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59a8108900 0x7f59a81a04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:23:01.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.676+0000 7f59ad715640 1 --2- 172.21.15.69:0/2907089864 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59a8103c00 0x7f59a8199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:23:01.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.676+0000 7f59adf16640 1 --2- 172.21.15.69:0/2907089864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59a81029c0 0x7f59a8198ff0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f59a4005fd0 tx=0x7f59a400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:01.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.676+0000 7f599effd640 1 -- 172.21.15.69:0/2907089864 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a4017070 con 0x7f59a81029c0 2024-04-25T11:23:01.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.676+0000 7f59b017a640 1 -- 172.21.15.69:0/2907089864 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59a81a0e20 con 0x7f59a81029c0 2024-04-25T11:23:01.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.676+0000 7f599effd640 1 -- 172.21.15.69:0/2907089864 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59a4009c30 con 0x7f59a81029c0 2024-04-25T11:23:01.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.676+0000 7f599effd640 1 -- 172.21.15.69:0/2907089864 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a4003c90 con 0x7f59a81029c0 2024-04-25T11:23:01.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.676+0000 7f59b017a640 1 -- 172.21.15.69:0/2907089864 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59a81a1290 con 0x7f59a81029c0 2024-04-25T11:23:01.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.678+0000 7f59b017a640 1 -- 172.21.15.69:0/2907089864 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59a8107710 con 0x7f59a81029c0 2024-04-25T11:23:01.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.679+0000 7f599effd640 1 -- 172.21.15.69:0/2907089864 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f59a4003e30 con 0x7f59a81029c0 2024-04-25T11:23:01.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.679+0000 7f599effd640 1 --2- 172.21.15.69:0/2907089864 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f59940afaf0 0x7f59940b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:01.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.679+0000 7f59ad715640 1 --2- 172.21.15.69:0/2907089864 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f59940afaf0 0x7f59940b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:01.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.679+0000 7f599effd640 1 -- 172.21.15.69:0/2907089864 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f59a40de500 con 0x7f59a81029c0 2024-04-25T11:23:01.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.680+0000 7f59ad715640 1 --2- 172.21.15.69:0/2907089864 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f59940afaf0 0x7f59940b2000 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f59a819a090 tx=0x7f5998005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:01.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.684+0000 7f599effd640 1 -- 172.21.15.69:0/2907089864 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f59a4020030 con 0x7f59a81029c0 2024-04-25T11:23:01.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.901+0000 7f59b017a640 1 -- 172.21.15.69:0/2907089864 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f59a802dda0 con 0x7f59a81029c0 2024-04-25T11:23:01.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.901+0000 7f599effd640 1 -- 172.21.15.69:0/2907089864 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f59a40a7580 con 0x7f59a81029c0 2024-04-25T11:23:01.896 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:23:01.896 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-25T11:23:01.897 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 1 2024-04-25T11:23:01.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.903+0000 7f59b017a640 1 -- 172.21.15.69:0/2907089864 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f59940afaf0 msgr2=0x7f59940b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:01.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.903+0000 7f59b017a640 1 --2- 172.21.15.69:0/2907089864 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f59940afaf0 0x7f59940b2000 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f59a819a090 tx=0x7f5998005f90 comp rx=0 tx=0).stop 2024-04-25T11:23:01.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.903+0000 7f59b017a640 1 -- 172.21.15.69:0/2907089864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59a81029c0 msgr2=0x7f59a8198ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:01.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.903+0000 7f59b017a640 1 --2- 172.21.15.69:0/2907089864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59a81029c0 0x7f59a8198ff0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f59a4005fd0 tx=0x7f59a400dde0 comp rx=0 tx=0).stop 2024-04-25T11:23:01.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.903+0000 7f59b017a640 1 -- 172.21.15.69:0/2907089864 shutdown_connections 2024-04-25T11:23:01.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.903+0000 7f59b017a640 1 --2- 172.21.15.69:0/2907089864 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f59940afaf0 0x7f59940b2000 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:01.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.903+0000 7f59b017a640 1 --2- 172.21.15.69:0/2907089864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59a8108900 0x7f59a81a04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:01.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.903+0000 7f59b017a640 1 --2- 172.21.15.69:0/2907089864 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59a8103c00 0x7f59a8199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:01.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.903+0000 7f59b017a640 1 --2- 172.21.15.69:0/2907089864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59a81029c0 0x7f59a8198ff0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:01.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.903+0000 7f59b017a640 1 -- 172.21.15.69:0/2907089864 >> 172.21.15.69:0/2907089864 conn(0x7f59a80fdbd0 msgr2=0x7f59a81048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:01.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.903+0000 7f59b017a640 1 -- 172.21.15.69:0/2907089864 shutdown_connections 2024-04-25T11:23:01.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:01.903+0000 7f59b017a640 1 -- 172.21.15.69:0/2907089864 wait complete. 2024-04-25T11:23:02.049 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:01 smithi152 ceph-mon[32701]: Deploying daemon mds.l on smithi152 2024-04-25T11:23:02.049 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:01 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/1931788695' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:23:02.050 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:01 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/234549263' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:23:02.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:01 smithi146 ceph-mon[32821]: Deploying daemon mds.l on smithi152 2024-04-25T11:23:02.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:01 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/1931788695' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:23:02.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:01 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/234549263' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:23:02.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:01 smithi069 ceph-mon[29188]: Deploying daemon mds.l on smithi152 2024-04-25T11:23:02.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:01 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/1931788695' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:23:02.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:01 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/234549263' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:23:02.440 INFO:journalctl@ceph.mds.l.smithi152.stdout:Apr 25 11:23:02 smithi152 systemd[1]: Starting Ceph mds.l for f7ed73dc-02f4-11ef-bc93-c7b262605968... 2024-04-25T11:23:02.703 INFO:journalctl@ceph.mds.l.smithi152.stdout:Apr 25 11:23:02 smithi152 podman[62959]: 2024-04-25 11:23:02.349297266 +0000 UTC m=+0.018435970 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T11:23:02.703 INFO:journalctl@ceph.mds.l.smithi152.stdout:Apr 25 11:23:02 smithi152 podman[62959]: 2024-04-25 11:23:02.46456559 +0000 UTC m=+0.133704262 container create 98e10171bbd1d7e0e9d7f360e1e39d5265c7556f2eece45bd7cb5b2f3a31b2a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-l, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424) 2024-04-25T11:23:02.703 INFO:journalctl@ceph.mds.l.smithi152.stdout:Apr 25 11:23:02 smithi152 podman[62959]: 2024-04-25 11:23:02.709458171 +0000 UTC m=+0.378596855 container init 98e10171bbd1d7e0e9d7f360e1e39d5265c7556f2eece45bd7cb5b2f3a31b2a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, ceph=True) 2024-04-25T11:23:02.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:02 smithi152 ceph-mon[32701]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:02.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:02 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/2907089864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:02.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:02 smithi152 ceph-mon[32701]: mds.? [v2:172.21.15.152:6832/4197482473,v1:172.21.15.152:6833/4197482473] up:boot 2024-04-25T11:23:02.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:02 smithi152 ceph-mon[32701]: fsmap 1 up:standby 2024-04-25T11:23:02.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:02 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T11:23:02.982 INFO:journalctl@ceph.mds.l.smithi152.stdout:Apr 25 11:23:02 smithi152 podman[62959]: 2024-04-25 11:23:02.712150135 +0000 UTC m=+0.381288812 container start 98e10171bbd1d7e0e9d7f360e1e39d5265c7556f2eece45bd7cb5b2f3a31b2a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-l, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD) 2024-04-25T11:23:02.982 INFO:journalctl@ceph.mds.l.smithi152.stdout:Apr 25 11:23:02 smithi152 ceph-mds[62978]: -- 172.21.15.152:0/342458165 <== mon.1 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c833485a40 con 0x55c8334cd600 2024-04-25T11:23:02.982 INFO:journalctl@ceph.mds.l.smithi152.stdout:Apr 25 11:23:02 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-l[62974]: starting mds.l at 2024-04-25T11:23:02.982 INFO:journalctl@ceph.mds.l.smithi152.stdout:Apr 25 11:23:02 smithi152 bash[62959]: 98e10171bbd1d7e0e9d7f360e1e39d5265c7556f2eece45bd7cb5b2f3a31b2a5 2024-04-25T11:23:02.982 INFO:journalctl@ceph.mds.l.smithi152.stdout:Apr 25 11:23:02 smithi152 systemd[1]: Started Ceph mds.l for f7ed73dc-02f4-11ef-bc93-c7b262605968. 2024-04-25T11:23:03.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:02 smithi146 ceph-mon[32821]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:03.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:02 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/2907089864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:03.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:02 smithi146 ceph-mon[32821]: mds.? [v2:172.21.15.152:6832/4197482473,v1:172.21.15.152:6833/4197482473] up:boot 2024-04-25T11:23:03.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:02 smithi146 ceph-mon[32821]: fsmap 1 up:standby 2024-04-25T11:23:03.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:02 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T11:23:03.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:02 smithi069 ceph-mon[29188]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:03.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:02 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2907089864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:03.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:02 smithi069 ceph-mon[29188]: mds.? [v2:172.21.15.152:6832/4197482473,v1:172.21.15.152:6833/4197482473] up:boot 2024-04-25T11:23:03.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:02 smithi069 ceph-mon[29188]: fsmap 1 up:standby 2024-04-25T11:23:03.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:02 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T11:23:03.910 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:23:04.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.037+0000 7f8ca8c9f640 1 -- 172.21.15.69:0/4130420676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ca4108940 msgr2=0x7f8ca410ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:04.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.037+0000 7f8ca8c9f640 1 --2- 172.21.15.69:0/4130420676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ca4108940 0x7f8ca410ad60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c009ca0 tx=0x7f8c9c02a8c0 comp rx=0 tx=0).stop 2024-04-25T11:23:04.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.037+0000 7f8ca8c9f640 1 -- 172.21.15.69:0/4130420676 shutdown_connections 2024-04-25T11:23:04.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.037+0000 7f8ca8c9f640 1 --2- 172.21.15.69:0/4130420676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ca4108940 0x7f8ca410ad60 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:04.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.037+0000 7f8ca8c9f640 1 --2- 172.21.15.69:0/4130420676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ca4103c40 0x7f8ca4104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:04.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.037+0000 7f8ca8c9f640 1 --2- 172.21.15.69:0/4130420676 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ca4102a00 0x7f8ca4103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:04.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.037+0000 7f8ca8c9f640 1 -- 172.21.15.69:0/4130420676 >> 172.21.15.69:0/4130420676 conn(0x7f8ca40fdbf0 msgr2=0x7f8ca41000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:04.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.037+0000 7f8ca8c9f640 1 -- 172.21.15.69:0/4130420676 shutdown_connections 2024-04-25T11:23:04.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.037+0000 7f8ca8c9f640 1 -- 172.21.15.69:0/4130420676 wait complete. 2024-04-25T11:23:04.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.038+0000 7f8ca8c9f640 1 Processor -- start 2024-04-25T11:23:04.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.038+0000 7f8ca8c9f640 1 -- start start 2024-04-25T11:23:04.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.038+0000 7f8ca8c9f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ca4102a00 0x7f8ca40757f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:04.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.038+0000 7f8ca8c9f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ca4103c40 0x7f8ca419ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:04.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.038+0000 7f8ca8c9f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ca4108940 0x7f8ca41a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:04.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.038+0000 7f8ca8c9f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca410e430 con 0x7f8ca4108940 2024-04-25T11:23:04.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.038+0000 7f8ca8c9f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca410e2b0 con 0x7f8ca4103c40 2024-04-25T11:23:04.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.038+0000 7f8ca8c9f640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca410e5b0 con 0x7f8ca4102a00 2024-04-25T11:23:04.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.038+0000 7f8ca2d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ca4108940 0x7f8ca41a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:04.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.038+0000 7f8ca2d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ca4108940 0x7f8ca41a13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58736/0 (socket says 172.21.15.69:58736) 2024-04-25T11:23:04.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.038+0000 7f8ca2d9d640 1 -- 172.21.15.69:0/860646360 learned_addr learned my addr 172.21.15.69:0/860646360 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:04.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.039+0000 7f8ca259c640 1 --2- 172.21.15.69:0/860646360 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ca4102a00 0x7f8ca40757f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:04.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.039+0000 7f8ca1d9b640 1 --2- 172.21.15.69:0/860646360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ca4103c40 0x7f8ca419ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:04.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.039+0000 7f8ca2d9d640 1 -- 172.21.15.69:0/860646360 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ca4102a00 msgr2=0x7f8ca40757f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:04.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.039+0000 7f8ca2d9d640 1 --2- 172.21.15.69:0/860646360 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ca4102a00 0x7f8ca40757f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:04.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.039+0000 7f8ca2d9d640 1 -- 172.21.15.69:0/860646360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ca4103c40 msgr2=0x7f8ca419ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:04.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.039+0000 7f8ca2d9d640 1 --2- 172.21.15.69:0/860646360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ca4103c40 0x7f8ca419ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:04.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.039+0000 7f8ca2d9d640 1 -- 172.21.15.69:0/860646360 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca41a1bb0 con 0x7f8ca4108940 2024-04-25T11:23:04.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.039+0000 7f8ca2d9d640 1 --2- 172.21.15.69:0/860646360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ca4108940 0x7f8ca41a13e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c031040 tx=0x7f8c9c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:04.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.039+0000 7f8c937fe640 1 -- 172.21.15.69:0/860646360 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c003d60 con 0x7f8ca4108940 2024-04-25T11:23:04.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.039+0000 7f8c937fe640 1 -- 172.21.15.69:0/860646360 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8c9c0044d0 con 0x7f8ca4108940 2024-04-25T11:23:04.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.039+0000 7f8ca8c9f640 1 -- 172.21.15.69:0/860646360 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ca41a1e40 con 0x7f8ca4108940 2024-04-25T11:23:04.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.039+0000 7f8c937fe640 1 -- 172.21.15.69:0/860646360 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c030c50 con 0x7f8ca4108940 2024-04-25T11:23:04.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.039+0000 7f8ca8c9f640 1 -- 172.21.15.69:0/860646360 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ca41a22a0 con 0x7f8ca4108940 2024-04-25T11:23:04.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.040+0000 7f8ca8c9f640 1 -- 172.21.15.69:0/860646360 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ca406a190 con 0x7f8ca4108940 2024-04-25T11:23:04.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.042+0000 7f8c937fe640 1 -- 172.21.15.69:0/860646360 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f8c9c004040 con 0x7f8ca4108940 2024-04-25T11:23:04.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.042+0000 7f8c937fe640 1 --2- 172.21.15.69:0/860646360 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f8c8c0afaf0 0x7f8c8c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:04.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.042+0000 7f8ca259c640 1 --2- 172.21.15.69:0/860646360 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f8c8c0afaf0 0x7f8c8c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:04.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.043+0000 7f8c937fe640 1 -- 172.21.15.69:0/860646360 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f8c9c0ffde0 con 0x7f8ca4108940 2024-04-25T11:23:04.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.043+0000 7f8ca259c640 1 --2- 172.21.15.69:0/860646360 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f8c8c0afaf0 0x7f8c8c0b2000 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4103600 tx=0x7f8c98005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:04.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.046+0000 7f8c937fe640 1 -- 172.21.15.69:0/860646360 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8c9c0c8e60 con 0x7f8ca4108940 2024-04-25T11:23:04.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:03 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:04.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:03 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:04.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:03 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:04.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:03 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:23:04.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:03 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:23:04.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:03 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:23:04.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:03 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:04.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:03 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:04.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:03 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:04.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:03 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:23:04.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:03 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:23:04.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:03 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:23:04.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.263+0000 7f8ca8c9f640 1 -- 172.21.15.69:0/860646360 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ca4107750 con 0x7f8ca4108940 2024-04-25T11:23:04.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.264+0000 7f8c937fe640 1 -- 172.21.15.69:0/860646360 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f8c9c0c8760 con 0x7f8ca4108940 2024-04-25T11:23:04.258 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:23:04.259 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/4197482473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":4197482473},{"type":"v1","addr":"172.21.15.152:6833","nonce":4197482473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T11:23:04.259 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 2 2024-04-25T11:23:04.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.266+0000 7f8ca8c9f640 1 -- 172.21.15.69:0/860646360 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f8c8c0afaf0 msgr2=0x7f8c8c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:04.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.266+0000 7f8ca8c9f640 1 --2- 172.21.15.69:0/860646360 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f8c8c0afaf0 0x7f8c8c0b2000 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4103600 tx=0x7f8c98005f90 comp rx=0 tx=0).stop 2024-04-25T11:23:04.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.266+0000 7f8ca8c9f640 1 -- 172.21.15.69:0/860646360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ca4108940 msgr2=0x7f8ca41a13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:04.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.266+0000 7f8ca8c9f640 1 --2- 172.21.15.69:0/860646360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ca4108940 0x7f8ca41a13e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c031040 tx=0x7f8c9c003970 comp rx=0 tx=0).stop 2024-04-25T11:23:04.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.266+0000 7f8ca8c9f640 1 -- 172.21.15.69:0/860646360 shutdown_connections 2024-04-25T11:23:04.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.266+0000 7f8ca8c9f640 1 --2- 172.21.15.69:0/860646360 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f8c8c0afaf0 0x7f8c8c0b2000 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:04.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.266+0000 7f8ca8c9f640 1 --2- 172.21.15.69:0/860646360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8ca4108940 0x7f8ca41a13e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:04.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.266+0000 7f8ca8c9f640 1 --2- 172.21.15.69:0/860646360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ca4103c40 0x7f8ca419ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:04.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.266+0000 7f8ca8c9f640 1 --2- 172.21.15.69:0/860646360 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ca4102a00 0x7f8ca40757f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:04.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.266+0000 7f8ca8c9f640 1 -- 172.21.15.69:0/860646360 >> 172.21.15.69:0/860646360 conn(0x7f8ca40fdbf0 msgr2=0x7f8ca4104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:04.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.266+0000 7f8ca8c9f640 1 -- 172.21.15.69:0/860646360 shutdown_connections 2024-04-25T11:23:04.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:04.266+0000 7f8ca8c9f640 1 -- 172.21.15.69:0/860646360 wait complete. 2024-04-25T11:23:04.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:03 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:04.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:03 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:04.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:03 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:04.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:03 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:23:04.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:03 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:23:04.351 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:03 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:23:05.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:04 smithi069 ceph-mon[29188]: Deploying daemon mds.k on smithi146 2024-04-25T11:23:05.207 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:04 smithi069 ceph-mon[29188]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:05.207 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:04 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/860646360' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:05.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:04 smithi152 ceph-mon[32701]: Deploying daemon mds.k on smithi146 2024-04-25T11:23:05.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:04 smithi152 ceph-mon[32701]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:05.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:04 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/860646360' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:05.308 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:04 smithi146 ceph-mon[32821]: Deploying daemon mds.k on smithi146 2024-04-25T11:23:05.308 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:04 smithi146 ceph-mon[32821]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:05.308 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:04 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/860646360' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:05.308 INFO:journalctl@ceph.mds.k.smithi146.stdout:Apr 25 11:23:05 smithi146 systemd[1]: Starting Ceph mds.k for f7ed73dc-02f4-11ef-bc93-c7b262605968... 2024-04-25T11:23:05.595 INFO:journalctl@ceph.mds.k.smithi146.stdout:Apr 25 11:23:05 smithi146 podman[61937]: 2024-04-25 11:23:05.216163201 +0000 UTC m=+0.017965677 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T11:23:05.595 INFO:journalctl@ceph.mds.k.smithi146.stdout:Apr 25 11:23:05 smithi146 podman[61937]: 2024-04-25 11:23:05.340494589 +0000 UTC m=+0.142297065 container create 115cfd316cf7007d5d8afe65351a5983e51320ad640ca5f1760be725594a23ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-k, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T11:23:05.595 INFO:journalctl@ceph.mds.k.smithi146.stdout:Apr 25 11:23:05 smithi146 podman[61937]: 2024-04-25 11:23:05.601495145 +0000 UTC m=+0.403297619 container init 115cfd316cf7007d5d8afe65351a5983e51320ad640ca5f1760be725594a23ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-k, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T11:23:05.850 INFO:journalctl@ceph.mds.k.smithi146.stdout:Apr 25 11:23:05 smithi146 podman[61937]: 2024-04-25 11:23:05.60418854 +0000 UTC m=+0.405991024 container start 115cfd316cf7007d5d8afe65351a5983e51320ad640ca5f1760be725594a23ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-k, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T11:23:05.851 INFO:journalctl@ceph.mds.k.smithi146.stdout:Apr 25 11:23:05 smithi146 ceph-mds[61956]: -- 172.21.15.146:0/4172372202 <== mon.2 v2:172.21.15.146:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561af1b2da40 con 0x561af1b75180 2024-04-25T11:23:05.851 INFO:journalctl@ceph.mds.k.smithi146.stdout:Apr 25 11:23:05 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-k[61952]: starting mds.k at 2024-04-25T11:23:05.851 INFO:journalctl@ceph.mds.k.smithi146.stdout:Apr 25 11:23:05 smithi146 bash[61937]: 115cfd316cf7007d5d8afe65351a5983e51320ad640ca5f1760be725594a23ef 2024-04-25T11:23:05.851 INFO:journalctl@ceph.mds.k.smithi146.stdout:Apr 25 11:23:05 smithi146 systemd[1]: Started Ceph mds.k for f7ed73dc-02f4-11ef-bc93-c7b262605968. 2024-04-25T11:23:06.273 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:23:06.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.388+0000 7f3a4fa4c640 1 -- 172.21.15.69:0/222463222 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a48106f10 msgr2=0x7f3a48109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:06.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.388+0000 7f3a4fa4c640 1 --2- 172.21.15.69:0/222463222 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a48106f10 0x7f3a48109330 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3a38009900 tx=0x7f3a3802a410 comp rx=0 tx=0).stop 2024-04-25T11:23:06.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.388+0000 7f3a4fa4c640 1 -- 172.21.15.69:0/222463222 shutdown_connections 2024-04-25T11:23:06.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.388+0000 7f3a4fa4c640 1 --2- 172.21.15.69:0/222463222 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a48109870 0x7f3a4810bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:06.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.388+0000 7f3a4fa4c640 1 --2- 172.21.15.69:0/222463222 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a48106f10 0x7f3a48109330 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:06.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.388+0000 7f3a4fa4c640 1 --2- 172.21.15.69:0/222463222 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a481045b0 0x7f3a481069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:06.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.388+0000 7f3a4fa4c640 1 -- 172.21.15.69:0/222463222 >> 172.21.15.69:0/222463222 conn(0x7f3a480fdbd0 msgr2=0x7f3a48100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:06.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.388+0000 7f3a4fa4c640 1 -- 172.21.15.69:0/222463222 shutdown_connections 2024-04-25T11:23:06.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.388+0000 7f3a4fa4c640 1 -- 172.21.15.69:0/222463222 wait complete. 2024-04-25T11:23:06.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.389+0000 7f3a4fa4c640 1 Processor -- start 2024-04-25T11:23:06.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.389+0000 7f3a4fa4c640 1 -- start start 2024-04-25T11:23:06.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.389+0000 7f3a4fa4c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a481045b0 0x7f3a4819a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:06.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.389+0000 7f3a4fa4c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a48106f10 0x7f3a4819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:06.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.389+0000 7f3a4fa4c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a48109870 0x7f3a481a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:06.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.389+0000 7f3a4fa4c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4810f480 con 0x7f3a48106f10 2024-04-25T11:23:06.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.389+0000 7f3a4fa4c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4810f300 con 0x7f3a48109870 2024-04-25T11:23:06.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.389+0000 7f3a4fa4c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4810f600 con 0x7f3a481045b0 2024-04-25T11:23:06.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.389+0000 7f3a4cfe7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a48106f10 0x7f3a4819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:06.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.389+0000 7f3a4cfe7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a48106f10 0x7f3a4819a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58762/0 (socket says 172.21.15.69:58762) 2024-04-25T11:23:06.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.389+0000 7f3a4cfe7640 1 -- 172.21.15.69:0/1531125819 learned_addr learned my addr 172.21.15.69:0/1531125819 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:06.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.389+0000 7f3a4d7e8640 1 --2- 172.21.15.69:0/1531125819 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a481045b0 0x7f3a4819a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:06.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.389+0000 7f3a4dfe9640 1 --2- 172.21.15.69:0/1531125819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a48109870 0x7f3a481a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:06.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.390+0000 7f3a4cfe7640 1 -- 172.21.15.69:0/1531125819 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a481045b0 msgr2=0x7f3a4819a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:06.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.390+0000 7f3a4cfe7640 1 --2- 172.21.15.69:0/1531125819 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a481045b0 0x7f3a4819a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:06.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.390+0000 7f3a4cfe7640 1 -- 172.21.15.69:0/1531125819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a48109870 msgr2=0x7f3a481a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:06.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.390+0000 7f3a4cfe7640 1 --2- 172.21.15.69:0/1531125819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a48109870 0x7f3a481a1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:06.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.390+0000 7f3a4cfe7640 1 -- 172.21.15.69:0/1531125819 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a481a1c90 con 0x7f3a48106f10 2024-04-25T11:23:06.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.390+0000 7f3a4cfe7640 1 --2- 172.21.15.69:0/1531125819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a48106f10 0x7f3a4819a690 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3a38009230 tx=0x7f3a380040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:06.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.390+0000 7f3a3e7fc640 1 -- 172.21.15.69:0/1531125819 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3803f030 con 0x7f3a48106f10 2024-04-25T11:23:06.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.390+0000 7f3a3e7fc640 1 -- 172.21.15.69:0/1531125819 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a380044c0 con 0x7f3a48106f10 2024-04-25T11:23:06.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.390+0000 7f3a4fa4c640 1 -- 172.21.15.69:0/1531125819 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a481a1f20 con 0x7f3a48106f10 2024-04-25T11:23:06.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.390+0000 7f3a3e7fc640 1 -- 172.21.15.69:0/1531125819 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a38035070 con 0x7f3a48106f10 2024-04-25T11:23:06.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.390+0000 7f3a4fa4c640 1 -- 172.21.15.69:0/1531125819 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a481a2330 con 0x7f3a48106f10 2024-04-25T11:23:06.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.391+0000 7f3a4fa4c640 1 -- 172.21.15.69:0/1531125819 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a480ff6f0 con 0x7f3a48106f10 2024-04-25T11:23:06.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.393+0000 7f3a3e7fc640 1 -- 172.21.15.69:0/1531125819 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f3a38042030 con 0x7f3a48106f10 2024-04-25T11:23:06.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.393+0000 7f3a3e7fc640 1 --2- 172.21.15.69:0/1531125819 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f3a340afaf0 0x7f3a340b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:06.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.393+0000 7f3a4d7e8640 1 --2- 172.21.15.69:0/1531125819 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f3a340afaf0 0x7f3a340b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:06.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.393+0000 7f3a3e7fc640 1 -- 172.21.15.69:0/1531125819 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f3a380ff060 con 0x7f3a48106f10 2024-04-25T11:23:06.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.394+0000 7f3a4d7e8640 1 --2- 172.21.15.69:0/1531125819 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f3a340afaf0 0x7f3a340b2000 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3a4806a8b0 tx=0x7f3a44005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:06.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.401+0000 7f3a3e7fc640 1 -- 172.21.15.69:0/1531125819 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3a380c7a90 con 0x7f3a48106f10 2024-04-25T11:23:06.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.626+0000 7f3a4fa4c640 1 -- 172.21.15.69:0/1531125819 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a4802db50 con 0x7f3a48106f10 2024-04-25T11:23:06.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.627+0000 7f3a3e7fc640 1 -- 172.21.15.69:0/1531125819 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7f3a380cbd90 con 0x7f3a48106f10 2024-04-25T11:23:06.623 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:23:06.623 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/4197482473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":4197482473},{"type":"v1","addr":"172.21.15.152:6833","nonce":4197482473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/4267885323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4267885323},{"type":"v1","addr":"172.21.15.146:6833","nonce":4267885323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T11:23:06.624 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 3 2024-04-25T11:23:06.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.629+0000 7f3a4fa4c640 1 -- 172.21.15.69:0/1531125819 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f3a340afaf0 msgr2=0x7f3a340b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:06.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.629+0000 7f3a4fa4c640 1 --2- 172.21.15.69:0/1531125819 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f3a340afaf0 0x7f3a340b2000 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3a4806a8b0 tx=0x7f3a44005f90 comp rx=0 tx=0).stop 2024-04-25T11:23:06.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.629+0000 7f3a4fa4c640 1 -- 172.21.15.69:0/1531125819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a48106f10 msgr2=0x7f3a4819a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:06.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.629+0000 7f3a4fa4c640 1 --2- 172.21.15.69:0/1531125819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a48106f10 0x7f3a4819a690 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3a38009230 tx=0x7f3a380040c0 comp rx=0 tx=0).stop 2024-04-25T11:23:06.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.629+0000 7f3a4fa4c640 1 -- 172.21.15.69:0/1531125819 shutdown_connections 2024-04-25T11:23:06.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.629+0000 7f3a4fa4c640 1 --2- 172.21.15.69:0/1531125819 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f3a340afaf0 0x7f3a340b2000 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:06.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.629+0000 7f3a4fa4c640 1 --2- 172.21.15.69:0/1531125819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a48109870 0x7f3a481a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:06.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.629+0000 7f3a4fa4c640 1 --2- 172.21.15.69:0/1531125819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a48106f10 0x7f3a4819a690 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:06.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.629+0000 7f3a4fa4c640 1 --2- 172.21.15.69:0/1531125819 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a481045b0 0x7f3a4819a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:06.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.629+0000 7f3a4fa4c640 1 -- 172.21.15.69:0/1531125819 >> 172.21.15.69:0/1531125819 conn(0x7f3a480fdbd0 msgr2=0x7f3a481065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:06.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.629+0000 7f3a4fa4c640 1 -- 172.21.15.69:0/1531125819 shutdown_connections 2024-04-25T11:23:06.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:06.629+0000 7f3a4fa4c640 1 -- 172.21.15.69:0/1531125819 wait complete. 2024-04-25T11:23:06.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:06 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:06.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:06 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:06.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:06 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:06.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:06 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:23:06.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:06 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:23:06.945 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:06 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:23:06.945 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:06 smithi069 ceph-mon[29188]: Deploying daemon mds.j on smithi069 2024-04-25T11:23:06.945 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:06 smithi069 ceph-mon[29188]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:06.945 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:06 smithi069 ceph-mon[29188]: mds.? [v2:172.21.15.146:6832/4267885323,v1:172.21.15.146:6833/4267885323] up:boot 2024-04-25T11:23:06.945 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:06 smithi069 ceph-mon[29188]: fsmap 2 up:standby 2024-04-25T11:23:06.945 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:06 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T11:23:06.945 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:06 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/1531125819' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:06.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:06 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:06.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:06 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:06.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:06 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:06.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:06 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:23:06.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:06 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:23:06.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:06 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:23:06.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:06 smithi152 ceph-mon[32701]: Deploying daemon mds.j on smithi069 2024-04-25T11:23:06.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:06 smithi152 ceph-mon[32701]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:06.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:06 smithi152 ceph-mon[32701]: mds.? [v2:172.21.15.146:6832/4267885323,v1:172.21.15.146:6833/4267885323] up:boot 2024-04-25T11:23:06.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:06 smithi152 ceph-mon[32701]: fsmap 2 up:standby 2024-04-25T11:23:06.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:06 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T11:23:06.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:06 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/1531125819' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:07.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:06 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:07.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:06 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:07.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:06 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:07.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:06 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:23:07.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:06 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:23:07.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:06 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:23:07.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:06 smithi146 ceph-mon[32821]: Deploying daemon mds.j on smithi069 2024-04-25T11:23:07.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:06 smithi146 ceph-mon[32821]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:07.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:06 smithi146 ceph-mon[32821]: mds.? [v2:172.21.15.146:6832/4267885323,v1:172.21.15.146:6833/4267885323] up:boot 2024-04-25T11:23:07.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:06 smithi146 ceph-mon[32821]: fsmap 2 up:standby 2024-04-25T11:23:07.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:06 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T11:23:07.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:06 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/1531125819' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:08.194 INFO:journalctl@ceph.mds.j.smithi069.stdout:Apr 25 11:23:07 smithi069 systemd[1]: Starting Ceph mds.j for f7ed73dc-02f4-11ef-bc93-c7b262605968... 2024-04-25T11:23:08.479 INFO:journalctl@ceph.mds.j.smithi069.stdout:Apr 25 11:23:08 smithi069 podman[69523]: 2024-04-25 11:23:08.112318367 +0000 UTC m=+0.018492253 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T11:23:08.479 INFO:journalctl@ceph.mds.j.smithi069.stdout:Apr 25 11:23:08 smithi069 podman[69523]: 2024-04-25 11:23:08.240980821 +0000 UTC m=+0.147154740 container create 755177b434f5e856109567d6b2acb9dd02a337c13fb212098ec14d5445476324 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-j, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T11:23:08.479 INFO:journalctl@ceph.mds.j.smithi069.stdout:Apr 25 11:23:08 smithi069 podman[69523]: 2024-04-25 11:23:08.485248486 +0000 UTC m=+0.391422359 container init 755177b434f5e856109567d6b2acb9dd02a337c13fb212098ec14d5445476324 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0) 2024-04-25T11:23:08.636 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:23:08.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:08.756+0000 7f8577f84640 1 -- 172.21.15.69:0/4007539576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8570106f30 msgr2=0x7f8570109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:08.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:08.756+0000 7f8577f84640 1 --2- 172.21.15.69:0/4007539576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8570106f30 0x7f8570109350 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8560009900 tx=0x7f856002a410 comp rx=0 tx=0).stop 2024-04-25T11:23:08.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:08.756+0000 7f8577f84640 1 -- 172.21.15.69:0/4007539576 shutdown_connections 2024-04-25T11:23:08.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:08.756+0000 7f8577f84640 1 --2- 172.21.15.69:0/4007539576 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8570109890 0x7f857010bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:08.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:08.756+0000 7f8577f84640 1 --2- 172.21.15.69:0/4007539576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8570106f30 0x7f8570109350 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:08.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:08.756+0000 7f8577f84640 1 --2- 172.21.15.69:0/4007539576 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85701045d0 0x7f85701069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:08.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:08.756+0000 7f8577f84640 1 -- 172.21.15.69:0/4007539576 >> 172.21.15.69:0/4007539576 conn(0x7f85700fdbf0 msgr2=0x7f8570100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:08.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:08.757+0000 7f8577f84640 1 -- 172.21.15.69:0/4007539576 shutdown_connections 2024-04-25T11:23:08.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:08.757+0000 7f8577f84640 1 -- 172.21.15.69:0/4007539576 wait complete. 2024-04-25T11:23:08.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:08.757+0000 7f8577f84640 1 Processor -- start 2024-04-25T11:23:08.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:08.757+0000 7f8577f84640 1 -- start start 2024-04-25T11:23:08.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:08.758+0000 7f8577f84640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85701045d0 0x7f857019a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:08.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:08.758+0000 7f8577f84640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8570106f30 0x7f857019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:08.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:08.758+0000 7f8577f84640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8570109890 0x7f85701a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:08.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:08.758+0000 7f8577f84640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f857010f4a0 con 0x7f8570109890 2024-04-25T11:23:08.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:08.758+0000 7f8577f84640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f857010f320 con 0x7f85701045d0 2024-04-25T11:23:08.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:08.758+0000 7f8577f84640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f857010f620 con 0x7f8570106f30 2024-04-25T11:23:08.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:08.758+0000 7f8576521640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8570109890 0x7f85701a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:08.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:08.758+0000 7f8576521640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8570109890 0x7f85701a1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58812/0 (socket says 172.21.15.69:58812) 2024-04-25T11:23:08.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:08.758+0000 7f8576521640 1 -- 172.21.15.69:0/951639543 learned_addr learned my addr 172.21.15.69:0/951639543 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:08.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:08.758+0000 7f8575d20640 1 --2- 172.21.15.69:0/951639543 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85701045d0 0x7f857019a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:08.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:08.758+0000 7f857551f640 1 --2- 172.21.15.69:0/951639543 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8570106f30 0x7f857019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:08.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:08.758+0000 7f8576521640 1 -- 172.21.15.69:0/951639543 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8570106f30 msgr2=0x7f857019a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:08.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:08.758+0000 7f8576521640 1 --2- 172.21.15.69:0/951639543 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8570106f30 0x7f857019a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:08.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:08.758+0000 7f8576521640 1 -- 172.21.15.69:0/951639543 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85701045d0 msgr2=0x7f857019a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:08.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:08.758+0000 7f8576521640 1 --2- 172.21.15.69:0/951639543 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85701045d0 0x7f857019a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:08.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:08.758+0000 7f8576521640 1 -- 172.21.15.69:0/951639543 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85701a1d00 con 0x7f8570109890 2024-04-25T11:23:08.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:08.758+0000 7f857551f640 1 --2- 172.21.15.69:0/951639543 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8570106f30 0x7f857019a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:23:08.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:08.758+0000 7f8576521640 1 --2- 172.21.15.69:0/951639543 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8570109890 0x7f85701a1690 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f856800c970 tx=0x7f856800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:08.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:08.758+0000 7f8566ffd640 1 -- 172.21.15.69:0/951639543 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8568004230 con 0x7f8570109890 2024-04-25T11:23:08.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:08.759+0000 7f8566ffd640 1 -- 172.21.15.69:0/951639543 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8568007e40 con 0x7f8570109890 2024-04-25T11:23:08.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:08.759+0000 7f8577f84640 1 -- 172.21.15.69:0/951639543 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85701a1ff0 con 0x7f8570109890 2024-04-25T11:23:08.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:08.759+0000 7f8566ffd640 1 -- 172.21.15.69:0/951639543 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8568003da0 con 0x7f8570109890 2024-04-25T11:23:08.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:08.759+0000 7f8577f84640 1 -- 172.21.15.69:0/951639543 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85701a24b0 con 0x7f8570109890 2024-04-25T11:23:08.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:08.760+0000 7f8577f84640 1 -- 172.21.15.69:0/951639543 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85700ff710 con 0x7f8570109890 2024-04-25T11:23:08.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:08.761+0000 7f8566ffd640 1 -- 172.21.15.69:0/951639543 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f85680079b0 con 0x7f8570109890 2024-04-25T11:23:08.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:08.762+0000 7f8566ffd640 1 --2- 172.21.15.69:0/951639543 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f855c0afb70 0x7f855c0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:08.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:08.762+0000 7f8575d20640 1 --2- 172.21.15.69:0/951639543 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f855c0afb70 0x7f855c0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:08.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:08.762+0000 7f8566ffd640 1 -- 172.21.15.69:0/951639543 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f85680de440 con 0x7f8570109890 2024-04-25T11:23:08.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:08.762+0000 7f8575d20640 1 --2- 172.21.15.69:0/951639543 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f855c0afb70 0x7f855c0b2080 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f85700757f0 tx=0x7f856c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:08.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:08.766+0000 7f8566ffd640 1 -- 172.21.15.69:0/951639543 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8568014070 con 0x7f8570109890 2024-04-25T11:23:08.825 INFO:journalctl@ceph.mds.j.smithi069.stdout:Apr 25 11:23:08 smithi069 podman[69523]: 2024-04-25 11:23:08.487719429 +0000 UTC m=+0.393893303 container start 755177b434f5e856109567d6b2acb9dd02a337c13fb212098ec14d5445476324 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-j, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-25T11:23:08.825 INFO:journalctl@ceph.mds.j.smithi069.stdout:Apr 25 11:23:08 smithi069 ceph-mds[69542]: -- 172.21.15.69:0/3115925006 <== mon.0 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ab2efb7a40 con 0x55ab2efff600 2024-04-25T11:23:08.825 INFO:journalctl@ceph.mds.j.smithi069.stdout:Apr 25 11:23:08 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-j[69538]: starting mds.j at 2024-04-25T11:23:08.825 INFO:journalctl@ceph.mds.j.smithi069.stdout:Apr 25 11:23:08 smithi069 bash[69523]: 755177b434f5e856109567d6b2acb9dd02a337c13fb212098ec14d5445476324 2024-04-25T11:23:08.825 INFO:journalctl@ceph.mds.j.smithi069.stdout:Apr 25 11:23:08 smithi069 systemd[1]: Started Ceph mds.j for f7ed73dc-02f4-11ef-bc93-c7b262605968. 2024-04-25T11:23:08.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:09.005+0000 7f8577f84640 1 -- 172.21.15.69:0/951639543 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f85700637a0 con 0x7f8570109890 2024-04-25T11:23:09.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:09.005+0000 7f8566ffd640 1 -- 172.21.15.69:0/951639543 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3049 (secure 0 0 0) 0x7f85680a7420 con 0x7f8570109890 2024-04-25T11:23:09.000 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:23:09.000 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3399424094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3399424094},{"type":"v1","addr":"172.21.15.69:6835","nonce":3399424094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/4197482473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":4197482473},{"type":"v1","addr":"172.21.15.152:6833","nonce":4197482473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/4267885323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4267885323},{"type":"v1","addr":"172.21.15.146:6833","nonce":4267885323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T11:23:09.001 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 4 2024-04-25T11:23:09.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:09.008+0000 7f8577f84640 1 -- 172.21.15.69:0/951639543 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f855c0afb70 msgr2=0x7f855c0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:09.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:09.008+0000 7f8577f84640 1 --2- 172.21.15.69:0/951639543 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f855c0afb70 0x7f855c0b2080 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f85700757f0 tx=0x7f856c009500 comp rx=0 tx=0).stop 2024-04-25T11:23:09.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:09.008+0000 7f8577f84640 1 -- 172.21.15.69:0/951639543 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8570109890 msgr2=0x7f85701a1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:09.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:09.008+0000 7f8577f84640 1 --2- 172.21.15.69:0/951639543 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8570109890 0x7f85701a1690 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f856800c970 tx=0x7f856800ce30 comp rx=0 tx=0).stop 2024-04-25T11:23:09.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:09.008+0000 7f8577f84640 1 -- 172.21.15.69:0/951639543 shutdown_connections 2024-04-25T11:23:09.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:09.008+0000 7f8577f84640 1 --2- 172.21.15.69:0/951639543 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f855c0afb70 0x7f855c0b2080 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:09.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:09.008+0000 7f8577f84640 1 --2- 172.21.15.69:0/951639543 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8570109890 0x7f85701a1690 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:09.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:09.008+0000 7f8577f84640 1 --2- 172.21.15.69:0/951639543 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8570106f30 0x7f857019a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:09.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:09.008+0000 7f8577f84640 1 --2- 172.21.15.69:0/951639543 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85701045d0 0x7f857019a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:09.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:09.008+0000 7f8577f84640 1 -- 172.21.15.69:0/951639543 >> 172.21.15.69:0/951639543 conn(0x7f85700fdbf0 msgr2=0x7f8570106670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:09.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:09.008+0000 7f8577f84640 1 -- 172.21.15.69:0/951639543 shutdown_connections 2024-04-25T11:23:09.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:09.008+0000 7f8577f84640 1 -- 172.21.15.69:0/951639543 wait complete. 2024-04-25T11:23:09.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:08 smithi146 ceph-mon[32821]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:09.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:08 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:09.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:08 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:09.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:08 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:09.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:08 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:23:09.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:08 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:23:09.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:08 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:23:09.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:08 smithi069 ceph-mon[29188]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:09.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:08 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:09.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:08 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:09.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:08 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:09.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:08 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:23:09.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:08 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:23:09.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:08 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:23:09.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:08 smithi152 ceph-mon[32701]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:09.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:08 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:09.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:08 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:09.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:08 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:09.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:08 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:23:09.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:08 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:23:09.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:08 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:23:09.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:09 smithi069 ceph-mon[29188]: Deploying daemon mds.d on smithi069 2024-04-25T11:23:09.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:09 smithi069 ceph-mon[29188]: mds.? [v2:172.21.15.69:6834/3399424094,v1:172.21.15.69:6835/3399424094] up:boot 2024-04-25T11:23:09.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:09 smithi069 ceph-mon[29188]: fsmap 3 up:standby 2024-04-25T11:23:09.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:09 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T11:23:09.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:09 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/951639543' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:10.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:09 smithi146 ceph-mon[32821]: Deploying daemon mds.d on smithi069 2024-04-25T11:23:10.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:09 smithi146 ceph-mon[32821]: mds.? [v2:172.21.15.69:6834/3399424094,v1:172.21.15.69:6835/3399424094] up:boot 2024-04-25T11:23:10.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:09 smithi146 ceph-mon[32821]: fsmap 3 up:standby 2024-04-25T11:23:10.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:09 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T11:23:10.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:09 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/951639543' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:10.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:09 smithi152 ceph-mon[32701]: Deploying daemon mds.d on smithi069 2024-04-25T11:23:10.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:09 smithi152 ceph-mon[32701]: mds.? [v2:172.21.15.69:6834/3399424094,v1:172.21.15.69:6835/3399424094] up:boot 2024-04-25T11:23:10.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:09 smithi152 ceph-mon[32701]: fsmap 3 up:standby 2024-04-25T11:23:10.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:09 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T11:23:10.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:09 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/951639543' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:11.014 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:23:11.038 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:10 smithi069 ceph-mon[29188]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:11.039 INFO:journalctl@ceph.mds.d.smithi069.stdout:Apr 25 11:23:10 smithi069 systemd[1]: Starting Ceph mds.d for f7ed73dc-02f4-11ef-bc93-c7b262605968... 2024-04-25T11:23:11.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:10 smithi146 ceph-mon[32821]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:11.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.134+0000 7f9c4223c640 1 -- 172.21.15.69:0/1873813759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c3c103ae0 msgr2=0x7f9c3c1047e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:11.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.134+0000 7f9c4223c640 1 --2- 172.21.15.69:0/1873813759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c3c103ae0 0x7f9c3c1047e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c009900 tx=0x7f9c2c02a410 comp rx=0 tx=0).stop 2024-04-25T11:23:11.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.134+0000 7f9c4223c640 1 -- 172.21.15.69:0/1873813759 shutdown_connections 2024-04-25T11:23:11.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.134+0000 7f9c4223c640 1 --2- 172.21.15.69:0/1873813759 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9c3c109a20 0x7f9c3c10be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:11.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.134+0000 7f9c4223c640 1 --2- 172.21.15.69:0/1873813759 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c3c104d20 0x7f9c3c105130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:11.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.134+0000 7f9c4223c640 1 --2- 172.21.15.69:0/1873813759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c3c103ae0 0x7f9c3c1047e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:11.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.134+0000 7f9c4223c640 1 -- 172.21.15.69:0/1873813759 >> 172.21.15.69:0/1873813759 conn(0x7f9c3c0fecd0 msgr2=0x7f9c3c101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:11.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.134+0000 7f9c4223c640 1 -- 172.21.15.69:0/1873813759 shutdown_connections 2024-04-25T11:23:11.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.134+0000 7f9c4223c640 1 -- 172.21.15.69:0/1873813759 wait complete. 2024-04-25T11:23:11.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.135+0000 7f9c4223c640 1 Processor -- start 2024-04-25T11:23:11.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.135+0000 7f9c4223c640 1 -- start start 2024-04-25T11:23:11.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.135+0000 7f9c4223c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c3c103ae0 0x7f9c3c19a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:11.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.135+0000 7f9c4223c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9c3c104d20 0x7f9c3c19a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:11.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.135+0000 7f9c4223c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c3c109a20 0x7f9c3c1a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:11.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.135+0000 7f9c4223c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9c3c10f510 con 0x7f9c3c103ae0 2024-04-25T11:23:11.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.135+0000 7f9c4223c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9c3c10f390 con 0x7f9c3c104d20 2024-04-25T11:23:11.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.135+0000 7f9c4223c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9c3c10f690 con 0x7f9c3c109a20 2024-04-25T11:23:11.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.135+0000 7f9c3b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c3c103ae0 0x7f9c3c19a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:11.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.135+0000 7f9c3b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c3c103ae0 0x7f9c3c19a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58836/0 (socket says 172.21.15.69:58836) 2024-04-25T11:23:11.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.135+0000 7f9c3b7fe640 1 -- 172.21.15.69:0/3942490880 learned_addr learned my addr 172.21.15.69:0/3942490880 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:11.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.136+0000 7f9c3bfff640 1 --2- 172.21.15.69:0/3942490880 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c3c109a20 0x7f9c3c1a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:11.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.136+0000 7f9c3affd640 1 --2- 172.21.15.69:0/3942490880 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9c3c104d20 0x7f9c3c19a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:11.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.136+0000 7f9c3b7fe640 1 -- 172.21.15.69:0/3942490880 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c3c109a20 msgr2=0x7f9c3c1a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:11.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.136+0000 7f9c3b7fe640 1 --2- 172.21.15.69:0/3942490880 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c3c109a20 0x7f9c3c1a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:11.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.136+0000 7f9c3b7fe640 1 -- 172.21.15.69:0/3942490880 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9c3c104d20 msgr2=0x7f9c3c19a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:11.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.136+0000 7f9c3b7fe640 1 --2- 172.21.15.69:0/3942490880 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9c3c104d20 0x7f9c3c19a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:11.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.136+0000 7f9c3b7fe640 1 -- 172.21.15.69:0/3942490880 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c3c1a1cf0 con 0x7f9c3c103ae0 2024-04-25T11:23:11.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.136+0000 7f9c3b7fe640 1 --2- 172.21.15.69:0/3942490880 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c3c103ae0 0x7f9c3c19a1b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c005760 tx=0x7f9c2c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:11.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.136+0000 7f9c38ff9640 1 -- 172.21.15.69:0/3942490880 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c03f030 con 0x7f9c3c103ae0 2024-04-25T11:23:11.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.136+0000 7f9c38ff9640 1 -- 172.21.15.69:0/3942490880 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c2c003a80 con 0x7f9c3c103ae0 2024-04-25T11:23:11.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.136+0000 7f9c4223c640 1 -- 172.21.15.69:0/3942490880 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c3c1a1f80 con 0x7f9c3c103ae0 2024-04-25T11:23:11.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.136+0000 7f9c38ff9640 1 -- 172.21.15.69:0/3942490880 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c035070 con 0x7f9c3c103ae0 2024-04-25T11:23:11.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.136+0000 7f9c4223c640 1 -- 172.21.15.69:0/3942490880 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c3c1a2460 con 0x7f9c3c103ae0 2024-04-25T11:23:11.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.137+0000 7f9c4223c640 1 -- 172.21.15.69:0/3942490880 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c3c108830 con 0x7f9c3c103ae0 2024-04-25T11:23:11.135 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:10 smithi152 ceph-mon[32701]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:11.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.139+0000 7f9c38ff9640 1 -- 172.21.15.69:0/3942490880 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f9c2c042030 con 0x7f9c3c103ae0 2024-04-25T11:23:11.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.139+0000 7f9c38ff9640 1 --2- 172.21.15.69:0/3942490880 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f9c240afb70 0x7f9c240b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:11.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.139+0000 7f9c3affd640 1 --2- 172.21.15.69:0/3942490880 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f9c240afb70 0x7f9c240b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:11.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.139+0000 7f9c38ff9640 1 -- 172.21.15.69:0/3942490880 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f9c2c0ff250 con 0x7f9c3c103ae0 2024-04-25T11:23:11.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.140+0000 7f9c3affd640 1 --2- 172.21.15.69:0/3942490880 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f9c240afb70 0x7f9c240b2080 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9c30005fd0 tx=0x7f9c30005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:11.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.143+0000 7f9c38ff9640 1 -- 172.21.15.69:0/3942490880 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9c2c0c82d0 con 0x7f9c3c103ae0 2024-04-25T11:23:11.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.364+0000 7f9c4223c640 1 -- 172.21.15.69:0/3942490880 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c3c02dc70 con 0x7f9c3c103ae0 2024-04-25T11:23:11.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.365+0000 7f9c38ff9640 1 -- 172.21.15.69:0/3942490880 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3049 (secure 0 0 0) 0x7f9c2c0d0350 con 0x7f9c3c103ae0 2024-04-25T11:23:11.360 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:23:11.360 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3399424094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3399424094},{"type":"v1","addr":"172.21.15.69:6835","nonce":3399424094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/4197482473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":4197482473},{"type":"v1","addr":"172.21.15.152:6833","nonce":4197482473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/4267885323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4267885323},{"type":"v1","addr":"172.21.15.146:6833","nonce":4267885323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T11:23:11.360 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 4 2024-04-25T11:23:11.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.367+0000 7f9c4223c640 1 -- 172.21.15.69:0/3942490880 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f9c240afb70 msgr2=0x7f9c240b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:11.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.367+0000 7f9c4223c640 1 --2- 172.21.15.69:0/3942490880 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f9c240afb70 0x7f9c240b2080 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9c30005fd0 tx=0x7f9c30005850 comp rx=0 tx=0).stop 2024-04-25T11:23:11.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.367+0000 7f9c4223c640 1 -- 172.21.15.69:0/3942490880 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c3c103ae0 msgr2=0x7f9c3c19a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:11.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.367+0000 7f9c4223c640 1 --2- 172.21.15.69:0/3942490880 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c3c103ae0 0x7f9c3c19a1b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c005760 tx=0x7f9c2c004270 comp rx=0 tx=0).stop 2024-04-25T11:23:11.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.367+0000 7f9c4223c640 1 -- 172.21.15.69:0/3942490880 shutdown_connections 2024-04-25T11:23:11.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.367+0000 7f9c4223c640 1 --2- 172.21.15.69:0/3942490880 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f9c240afb70 0x7f9c240b2080 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:11.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.367+0000 7f9c4223c640 1 --2- 172.21.15.69:0/3942490880 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c3c109a20 0x7f9c3c1a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:11.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.367+0000 7f9c4223c640 1 --2- 172.21.15.69:0/3942490880 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9c3c104d20 0x7f9c3c19a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:11.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.367+0000 7f9c4223c640 1 --2- 172.21.15.69:0/3942490880 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c3c103ae0 0x7f9c3c19a1b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:11.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.367+0000 7f9c4223c640 1 -- 172.21.15.69:0/3942490880 >> 172.21.15.69:0/3942490880 conn(0x7f9c3c0fecd0 msgr2=0x7f9c3c105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:11.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.367+0000 7f9c4223c640 1 -- 172.21.15.69:0/3942490880 shutdown_connections 2024-04-25T11:23:11.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:11.367+0000 7f9c4223c640 1 -- 172.21.15.69:0/3942490880 wait complete. 2024-04-25T11:23:11.371 INFO:journalctl@ceph.mds.d.smithi069.stdout:Apr 25 11:23:11 smithi069 podman[69869]: 2024-04-25 11:23:11.044477657 +0000 UTC m=+0.019886826 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T11:23:11.371 INFO:journalctl@ceph.mds.d.smithi069.stdout:Apr 25 11:23:11 smithi069 podman[69869]: 2024-04-25 11:23:11.17301477 +0000 UTC m=+0.148423926 container create 178baf59a50be0381351dea5b29d4fd5ffd281d77e467f6e8973b9a3ee6ea017 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-d, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-25T11:23:11.694 INFO:journalctl@ceph.mds.d.smithi069.stdout:Apr 25 11:23:11 smithi069 podman[69869]: 2024-04-25 11:23:11.39713628 +0000 UTC m=+0.372545447 container init 178baf59a50be0381351dea5b29d4fd5ffd281d77e467f6e8973b9a3ee6ea017 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-d, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-25T11:23:11.694 INFO:journalctl@ceph.mds.d.smithi069.stdout:Apr 25 11:23:11 smithi069 podman[69869]: 2024-04-25 11:23:11.399749928 +0000 UTC m=+0.375159087 container start 178baf59a50be0381351dea5b29d4fd5ffd281d77e467f6e8973b9a3ee6ea017 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-d, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-25T11:23:11.694 INFO:journalctl@ceph.mds.d.smithi069.stdout:Apr 25 11:23:11 smithi069 ceph-mds[69929]: -- 172.21.15.69:0/1875064956 <== mon.0 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561c5aca7a40 con 0x561c5acef180 2024-04-25T11:23:11.694 INFO:journalctl@ceph.mds.d.smithi069.stdout:Apr 25 11:23:11 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-d[69923]: starting mds.d at 2024-04-25T11:23:11.694 INFO:journalctl@ceph.mds.d.smithi069.stdout:Apr 25 11:23:11 smithi069 bash[69869]: 178baf59a50be0381351dea5b29d4fd5ffd281d77e467f6e8973b9a3ee6ea017 2024-04-25T11:23:11.694 INFO:journalctl@ceph.mds.d.smithi069.stdout:Apr 25 11:23:11 smithi069 systemd[1]: Started Ceph mds.d for f7ed73dc-02f4-11ef-bc93-c7b262605968. 2024-04-25T11:23:12.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:11 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/3942490880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:12.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:11 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:12.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:11 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:12.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:11 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:12.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:11 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:23:12.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:11 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:23:12.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:11 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:23:12.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:11 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/3942490880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:12.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:11 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:12.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:11 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:12.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:11 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:12.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:11 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:23:12.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:11 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:23:12.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:11 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:23:12.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:11 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/3942490880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:12.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:11 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:12.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:11 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:12.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:11 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:12.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:11 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:23:12.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:11 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:23:12.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:11 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:23:13.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:12 smithi146 ceph-mon[32821]: Deploying daemon mds.e on smithi146 2024-04-25T11:23:13.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:12 smithi146 ceph-mon[32821]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:13.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:12 smithi146 ceph-mon[32821]: mds.? [v2:172.21.15.69:6836/1463217771,v1:172.21.15.69:6837/1463217771] up:boot 2024-04-25T11:23:13.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:12 smithi146 ceph-mon[32821]: fsmap 4 up:standby 2024-04-25T11:23:13.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:12 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T11:23:13.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:12 smithi069 ceph-mon[29188]: Deploying daemon mds.e on smithi146 2024-04-25T11:23:13.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:12 smithi069 ceph-mon[29188]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:13.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:12 smithi069 ceph-mon[29188]: mds.? [v2:172.21.15.69:6836/1463217771,v1:172.21.15.69:6837/1463217771] up:boot 2024-04-25T11:23:13.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:12 smithi069 ceph-mon[29188]: fsmap 4 up:standby 2024-04-25T11:23:13.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:12 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T11:23:13.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:12 smithi152 ceph-mon[32701]: Deploying daemon mds.e on smithi146 2024-04-25T11:23:13.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:12 smithi152 ceph-mon[32701]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:13.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:12 smithi152 ceph-mon[32701]: mds.? [v2:172.21.15.69:6836/1463217771,v1:172.21.15.69:6837/1463217771] up:boot 2024-04-25T11:23:13.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:12 smithi152 ceph-mon[32701]: fsmap 4 up:standby 2024-04-25T11:23:13.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:12 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T11:23:13.373 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:23:13.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.502+0000 7f2cb4684640 1 -- 172.21.15.69:0/4214653817 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2cac1050a0 msgr2=0x7f2cac1054b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:13.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.502+0000 7f2cb4684640 1 --2- 172.21.15.69:0/4214653817 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2cac1050a0 0x7f2cac1054b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8009860 tx=0x7f2ca802a340 comp rx=0 tx=0).stop 2024-04-25T11:23:13.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.502+0000 7f2cb4684640 1 -- 172.21.15.69:0/4214653817 shutdown_connections 2024-04-25T11:23:13.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.502+0000 7f2cb4684640 1 --2- 172.21.15.69:0/4214653817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2cac102250 0x7f2cac10bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:13.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.502+0000 7f2cb4684640 1 --2- 172.21.15.69:0/4214653817 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2cac101750 0x7f2cac101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:13.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.502+0000 7f2cb4684640 1 --2- 172.21.15.69:0/4214653817 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2cac1050a0 0x7f2cac1054b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:13.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.502+0000 7f2cb4684640 1 -- 172.21.15.69:0/4214653817 >> 172.21.15.69:0/4214653817 conn(0x7f2cac0fd370 msgr2=0x7f2cac0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:13.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.502+0000 7f2cb4684640 1 -- 172.21.15.69:0/4214653817 shutdown_connections 2024-04-25T11:23:13.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.502+0000 7f2cb4684640 1 -- 172.21.15.69:0/4214653817 wait complete. 2024-04-25T11:23:13.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.502+0000 7f2cb4684640 1 Processor -- start 2024-04-25T11:23:13.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.502+0000 7f2cb4684640 1 -- start start 2024-04-25T11:23:13.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.502+0000 7f2cb4684640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2cac101750 0x7f2cac19a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:13.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.503+0000 7f2cb4684640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2cac102250 0x7f2cac19a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:13.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.503+0000 7f2cb4684640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2cac1050a0 0x7f2cac1a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:13.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.503+0000 7f2cb4684640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac10f4c0 con 0x7f2cac101750 2024-04-25T11:23:13.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.503+0000 7f2cb4684640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac10f340 con 0x7f2cac1050a0 2024-04-25T11:23:13.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.503+0000 7f2cb4684640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac10f640 con 0x7f2cac102250 2024-04-25T11:23:13.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.503+0000 7f2cb2420640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2cac101750 0x7f2cac19a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:13.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.503+0000 7f2cb2420640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2cac101750 0x7f2cac19a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57206/0 (socket says 172.21.15.69:57206) 2024-04-25T11:23:13.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.503+0000 7f2cb2420640 1 -- 172.21.15.69:0/670853115 learned_addr learned my addr 172.21.15.69:0/670853115 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:13.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.503+0000 7f2cb2c21640 1 --2- 172.21.15.69:0/670853115 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2cac1050a0 0x7f2cac1a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:13.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.503+0000 7f2cb1c1f640 1 --2- 172.21.15.69:0/670853115 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2cac102250 0x7f2cac19a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:13.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.503+0000 7f2cb2420640 1 -- 172.21.15.69:0/670853115 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2cac102250 msgr2=0x7f2cac19a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:13.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.503+0000 7f2cb2420640 1 --2- 172.21.15.69:0/670853115 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2cac102250 0x7f2cac19a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:13.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.503+0000 7f2cb2420640 1 -- 172.21.15.69:0/670853115 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2cac1050a0 msgr2=0x7f2cac1a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:13.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.503+0000 7f2cb2420640 1 --2- 172.21.15.69:0/670853115 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2cac1050a0 0x7f2cac1a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:13.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.503+0000 7f2cb2420640 1 -- 172.21.15.69:0/670853115 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cac1a1cc0 con 0x7f2cac101750 2024-04-25T11:23:13.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.503+0000 7f2cb1c1f640 1 --2- 172.21.15.69:0/670853115 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2cac102250 0x7f2cac19a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:23:13.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.503+0000 7f2cb2c21640 1 --2- 172.21.15.69:0/670853115 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2cac1050a0 0x7f2cac1a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:23:13.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.503+0000 7f2cb2420640 1 --2- 172.21.15.69:0/670853115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2cac101750 0x7f2cac19a180 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2ca802a930 tx=0x7f2ca802ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:13.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.503+0000 7f2ca37fe640 1 -- 172.21.15.69:0/670853115 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8040030 con 0x7f2cac101750 2024-04-25T11:23:13.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.503+0000 7f2ca37fe640 1 -- 172.21.15.69:0/670853115 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2ca80305f0 con 0x7f2cac101750 2024-04-25T11:23:13.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.503+0000 7f2cb4684640 1 -- 172.21.15.69:0/670853115 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cac1a1f50 con 0x7f2cac101750 2024-04-25T11:23:13.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.503+0000 7f2ca37fe640 1 -- 172.21.15.69:0/670853115 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8037040 con 0x7f2cac101750 2024-04-25T11:23:13.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.503+0000 7f2cb4684640 1 -- 172.21.15.69:0/670853115 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cac1a23b0 con 0x7f2cac101750 2024-04-25T11:23:13.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.505+0000 7f2cb4684640 1 -- 172.21.15.69:0/670853115 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cac108a30 con 0x7f2cac101750 2024-04-25T11:23:13.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.506+0000 7f2ca37fe640 1 -- 172.21.15.69:0/670853115 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f2ca8043070 con 0x7f2cac101750 2024-04-25T11:23:13.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.506+0000 7f2ca37fe640 1 --2- 172.21.15.69:0/670853115 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f2c980afb70 0x7f2c980b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:13.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.507+0000 7f2cb1c1f640 1 --2- 172.21.15.69:0/670853115 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f2c980afb70 0x7f2c980b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:13.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.507+0000 7f2ca37fe640 1 -- 172.21.15.69:0/670853115 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f2ca80fe4c0 con 0x7f2cac101750 2024-04-25T11:23:13.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.507+0000 7f2cb1c1f640 1 --2- 172.21.15.69:0/670853115 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f2c980afb70 0x7f2c980b2080 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2cac19b220 tx=0x7f2c9c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:13.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.511+0000 7f2ca37fe640 1 -- 172.21.15.69:0/670853115 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2ca80c75d0 con 0x7f2cac101750 2024-04-25T11:23:13.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.723+0000 7f2cb4684640 1 -- 172.21.15.69:0/670853115 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2cac02dd00 con 0x7f2cac101750 2024-04-25T11:23:13.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.724+0000 7f2ca37fe640 1 -- 172.21.15.69:0/670853115 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3874 (secure 0 0 0) 0x7f2ca80c6ed0 con 0x7f2cac101750 2024-04-25T11:23:13.719 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:23:13.719 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3399424094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3399424094},{"type":"v1","addr":"172.21.15.69:6835","nonce":3399424094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1463217771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1463217771},{"type":"v1","addr":"172.21.15.69:6837","nonce":1463217771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/4197482473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":4197482473},{"type":"v1","addr":"172.21.15.152:6833","nonce":4197482473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/4267885323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4267885323},{"type":"v1","addr":"172.21.15.146:6833","nonce":4267885323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T11:23:13.719 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 5 2024-04-25T11:23:13.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.726+0000 7f2cb4684640 1 -- 172.21.15.69:0/670853115 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f2c980afb70 msgr2=0x7f2c980b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:13.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.726+0000 7f2cb4684640 1 --2- 172.21.15.69:0/670853115 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f2c980afb70 0x7f2c980b2080 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2cac19b220 tx=0x7f2c9c005f90 comp rx=0 tx=0).stop 2024-04-25T11:23:13.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.726+0000 7f2cb4684640 1 -- 172.21.15.69:0/670853115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2cac101750 msgr2=0x7f2cac19a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:13.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.726+0000 7f2cb4684640 1 --2- 172.21.15.69:0/670853115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2cac101750 0x7f2cac19a180 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2ca802a930 tx=0x7f2ca802ad70 comp rx=0 tx=0).stop 2024-04-25T11:23:13.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.726+0000 7f2cb4684640 1 -- 172.21.15.69:0/670853115 shutdown_connections 2024-04-25T11:23:13.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.726+0000 7f2cb4684640 1 --2- 172.21.15.69:0/670853115 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f2c980afb70 0x7f2c980b2080 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:13.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.726+0000 7f2cb4684640 1 --2- 172.21.15.69:0/670853115 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2cac1050a0 0x7f2cac1a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:13.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.726+0000 7f2cb4684640 1 --2- 172.21.15.69:0/670853115 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2cac102250 0x7f2cac19a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:13.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.726+0000 7f2cb4684640 1 --2- 172.21.15.69:0/670853115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2cac101750 0x7f2cac19a180 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:13.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.726+0000 7f2cb4684640 1 -- 172.21.15.69:0/670853115 >> 172.21.15.69:0/670853115 conn(0x7f2cac0fd370 msgr2=0x7f2cac105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:13.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.726+0000 7f2cb4684640 1 -- 172.21.15.69:0/670853115 shutdown_connections 2024-04-25T11:23:13.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:13.726+0000 7f2cb4684640 1 -- 172.21.15.69:0/670853115 wait complete. 2024-04-25T11:23:14.025 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:13 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/670853115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:14.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:13 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/670853115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:14.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:13 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/670853115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:14.300 INFO:journalctl@ceph.mds.e.smithi146.stdout:Apr 25 11:23:14 smithi146 podman[62257]: 2024-04-25 11:23:14.273142093 +0000 UTC m=+0.358520893 container start 6f4c14e69976442102f92ca019018e7da4a42aaa0126c0f4c0f1993ea64233d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-e, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-25T11:23:14.300 INFO:journalctl@ceph.mds.e.smithi146.stdout:Apr 25 11:23:14 smithi146 ceph-mds[62277]: -- 172.21.15.146:0/1304332633 <== mon.2 v2:172.21.15.146:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bde3017a40 con 0x55bde305f180 2024-04-25T11:23:14.600 INFO:journalctl@ceph.mds.e.smithi146.stdout:Apr 25 11:23:14 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-e[62273]: starting mds.e at 2024-04-25T11:23:14.600 INFO:journalctl@ceph.mds.e.smithi146.stdout:Apr 25 11:23:14 smithi146 bash[62257]: 6f4c14e69976442102f92ca019018e7da4a42aaa0126c0f4c0f1993ea64233d0 2024-04-25T11:23:14.600 INFO:journalctl@ceph.mds.e.smithi146.stdout:Apr 25 11:23:14 smithi146 systemd[1]: Started Ceph mds.e for f7ed73dc-02f4-11ef-bc93-c7b262605968. 2024-04-25T11:23:15.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:14 smithi069 ceph-mon[29188]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:15.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:14 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:15.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:14 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:15.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:14 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:15.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:14 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:23:15.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:14 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:23:15.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:14 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:23:15.216 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:14 smithi152 ceph-mon[32701]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:15.216 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:14 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:15.216 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:14 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:15.217 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:14 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:15.217 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:14 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:23:15.217 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:14 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:23:15.217 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:14 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:23:15.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:14 smithi146 ceph-mon[32821]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:15.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:14 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:15.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:14 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:15.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:14 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:15.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:14 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:23:15.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:14 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:23:15.351 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:14 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:23:15.732 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:23:15.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:15.858+0000 7fb9bce27640 1 -- 172.21.15.69:0/271387388 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9b8109a40 msgr2=0x7fb9b810be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:15.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:15.858+0000 7fb9bce27640 1 --2- 172.21.15.69:0/271387388 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9b8109a40 0x7fb9b810be60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0009ca0 tx=0x7fb9b002a8c0 comp rx=0 tx=0).stop 2024-04-25T11:23:15.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:15.858+0000 7fb9bce27640 1 -- 172.21.15.69:0/271387388 shutdown_connections 2024-04-25T11:23:15.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:15.858+0000 7fb9bce27640 1 --2- 172.21.15.69:0/271387388 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9b8109a40 0x7fb9b810be60 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:15.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:15.858+0000 7fb9bce27640 1 --2- 172.21.15.69:0/271387388 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9b8104d40 0x7fb9b8105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:15.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:15.858+0000 7fb9bce27640 1 --2- 172.21.15.69:0/271387388 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9b8103b00 0x7fb9b8104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:15.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:15.858+0000 7fb9bce27640 1 -- 172.21.15.69:0/271387388 >> 172.21.15.69:0/271387388 conn(0x7fb9b80fed10 msgr2=0x7fb9b81011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:15.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:15.859+0000 7fb9bce27640 1 -- 172.21.15.69:0/271387388 shutdown_connections 2024-04-25T11:23:15.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:15.859+0000 7fb9bce27640 1 -- 172.21.15.69:0/271387388 wait complete. 2024-04-25T11:23:15.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:15.859+0000 7fb9bce27640 1 Processor -- start 2024-04-25T11:23:15.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:15.859+0000 7fb9bce27640 1 -- start start 2024-04-25T11:23:15.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:15.860+0000 7fb9bce27640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9b8103b00 0x7fb9b819a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:15.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:15.860+0000 7fb9bce27640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9b8104d40 0x7fb9b819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:15.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:15.860+0000 7fb9bce27640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9b8109a40 0x7fb9b81a1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:15.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:15.860+0000 7fb9bce27640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b810f510 con 0x7fb9b8103b00 2024-04-25T11:23:15.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:15.860+0000 7fb9bce27640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b810f390 con 0x7fb9b8109a40 2024-04-25T11:23:15.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:15.860+0000 7fb9bce27640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b810f690 con 0x7fb9b8104d40 2024-04-25T11:23:15.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:15.860+0000 7fb9b659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9b8103b00 0x7fb9b819a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:15.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:15.860+0000 7fb9b659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9b8103b00 0x7fb9b819a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57218/0 (socket says 172.21.15.69:57218) 2024-04-25T11:23:15.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:15.860+0000 7fb9b659c640 1 -- 172.21.15.69:0/2562710938 learned_addr learned my addr 172.21.15.69:0/2562710938 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:15.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:15.860+0000 7fb9affff640 1 --2- 172.21.15.69:0/2562710938 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9b8104d40 0x7fb9b819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:15.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:15.860+0000 7fb9b6d9d640 1 --2- 172.21.15.69:0/2562710938 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9b8109a40 0x7fb9b81a1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:15.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:15.860+0000 7fb9b659c640 1 -- 172.21.15.69:0/2562710938 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9b8104d40 msgr2=0x7fb9b819a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:15.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:15.860+0000 7fb9b659c640 1 --2- 172.21.15.69:0/2562710938 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9b8104d40 0x7fb9b819a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:15.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:15.860+0000 7fb9b659c640 1 -- 172.21.15.69:0/2562710938 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9b8109a40 msgr2=0x7fb9b81a1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:15.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:15.860+0000 7fb9b659c640 1 --2- 172.21.15.69:0/2562710938 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9b8109a40 0x7fb9b81a1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:15.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:15.860+0000 7fb9b659c640 1 -- 172.21.15.69:0/2562710938 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9b81a1bc0 con 0x7fb9b8103b00 2024-04-25T11:23:15.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:15.860+0000 7fb9b6d9d640 1 --2- 172.21.15.69:0/2562710938 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9b8109a40 0x7fb9b81a1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:23:15.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:15.860+0000 7fb9b659c640 1 --2- 172.21.15.69:0/2562710938 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9b8103b00 0x7fb9b819a130 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb9a800d740 tx=0x7fb9a800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:15.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:15.860+0000 7fb9af7fe640 1 -- 172.21.15.69:0/2562710938 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a8017070 con 0x7fb9b8103b00 2024-04-25T11:23:15.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:15.860+0000 7fb9af7fe640 1 -- 172.21.15.69:0/2562710938 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb9a800de50 con 0x7fb9b8103b00 2024-04-25T11:23:15.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:15.860+0000 7fb9bce27640 1 -- 172.21.15.69:0/2562710938 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9b81a1eb0 con 0x7fb9b8103b00 2024-04-25T11:23:15.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:15.860+0000 7fb9af7fe640 1 -- 172.21.15.69:0/2562710938 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a8003bb0 con 0x7fb9b8103b00 2024-04-25T11:23:15.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:15.860+0000 7fb9bce27640 1 -- 172.21.15.69:0/2562710938 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9b81a2370 con 0x7fb9b8103b00 2024-04-25T11:23:15.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:15.862+0000 7fb9bce27640 1 -- 172.21.15.69:0/2562710938 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9b8108850 con 0x7fb9b8103b00 2024-04-25T11:23:15.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:15.863+0000 7fb9af7fe640 1 -- 172.21.15.69:0/2562710938 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fb9a8003d50 con 0x7fb9b8103b00 2024-04-25T11:23:15.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:15.863+0000 7fb9af7fe640 1 --2- 172.21.15.69:0/2562710938 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fb9a40afb70 0x7fb9a40b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:15.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:15.863+0000 7fb9affff640 1 --2- 172.21.15.69:0/2562710938 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fb9a40afb70 0x7fb9a40b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:15.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:15.863+0000 7fb9af7fe640 1 -- 172.21.15.69:0/2562710938 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fb9a80de170 con 0x7fb9b8103b00 2024-04-25T11:23:15.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:15.864+0000 7fb9affff640 1 --2- 172.21.15.69:0/2562710938 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fb9a40afb70 0x7fb9a40b2080 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0005e10 tx=0x7fb9a0005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:15.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:15.871+0000 7fb9af7fe640 1 -- 172.21.15.69:0/2562710938 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb9a80a71f0 con 0x7fb9b8103b00 2024-04-25T11:23:16.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:16.089+0000 7fb9bce27640 1 -- 172.21.15.69:0/2562710938 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb9b802dd40 con 0x7fb9b8103b00 2024-04-25T11:23:16.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:16.090+0000 7fb9af7fe640 1 -- 172.21.15.69:0/2562710938 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4702 (secure 0 0 0) 0x7fb9a80a6af0 con 0x7fb9b8103b00 2024-04-25T11:23:16.085 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:23:16.085 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3399424094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3399424094},{"type":"v1","addr":"172.21.15.69:6835","nonce":3399424094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1463217771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1463217771},{"type":"v1","addr":"172.21.15.69:6837","nonce":1463217771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/4197482473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":4197482473},{"type":"v1","addr":"172.21.15.152:6833","nonce":4197482473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/4267885323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4267885323},{"type":"v1","addr":"172.21.15.146:6833","nonce":4267885323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1876533699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1876533699},{"type":"v1","addr":"172.21.15.146:6835","nonce":1876533699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-25T11:23:16.085 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 6 2024-04-25T11:23:16.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:16.092+0000 7fb9bce27640 1 -- 172.21.15.69:0/2562710938 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fb9a40afb70 msgr2=0x7fb9a40b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:16.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:16.092+0000 7fb9bce27640 1 --2- 172.21.15.69:0/2562710938 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fb9a40afb70 0x7fb9a40b2080 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0005e10 tx=0x7fb9a0005da0 comp rx=0 tx=0).stop 2024-04-25T11:23:16.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:16.092+0000 7fb9bce27640 1 -- 172.21.15.69:0/2562710938 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9b8103b00 msgr2=0x7fb9b819a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:16.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:16.092+0000 7fb9bce27640 1 --2- 172.21.15.69:0/2562710938 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9b8103b00 0x7fb9b819a130 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb9a800d740 tx=0x7fb9a800dc00 comp rx=0 tx=0).stop 2024-04-25T11:23:16.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:16.092+0000 7fb9bce27640 1 -- 172.21.15.69:0/2562710938 shutdown_connections 2024-04-25T11:23:16.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:16.092+0000 7fb9bce27640 1 --2- 172.21.15.69:0/2562710938 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fb9a40afb70 0x7fb9a40b2080 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:16.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:16.092+0000 7fb9bce27640 1 --2- 172.21.15.69:0/2562710938 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9b8109a40 0x7fb9b81a1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:16.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:16.092+0000 7fb9bce27640 1 --2- 172.21.15.69:0/2562710938 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9b8104d40 0x7fb9b819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:16.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:16.092+0000 7fb9bce27640 1 --2- 172.21.15.69:0/2562710938 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9b8103b00 0x7fb9b819a130 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:16.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:16.092+0000 7fb9bce27640 1 -- 172.21.15.69:0/2562710938 >> 172.21.15.69:0/2562710938 conn(0x7fb9b80fed10 msgr2=0x7fb9b8105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:16.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:16.093+0000 7fb9bce27640 1 -- 172.21.15.69:0/2562710938 shutdown_connections 2024-04-25T11:23:16.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:16.093+0000 7fb9bce27640 1 -- 172.21.15.69:0/2562710938 wait complete. 2024-04-25T11:23:16.097 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:15 smithi069 ceph-mon[29188]: Deploying daemon mds.f on smithi152 2024-04-25T11:23:16.097 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:15 smithi069 ceph-mon[29188]: mds.? [v2:172.21.15.146:6834/1876533699,v1:172.21.15.146:6835/1876533699] up:boot 2024-04-25T11:23:16.097 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:15 smithi069 ceph-mon[29188]: fsmap 5 up:standby 2024-04-25T11:23:16.097 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:15 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T11:23:16.178 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:15 smithi152 ceph-mon[32701]: Deploying daemon mds.f on smithi152 2024-04-25T11:23:16.178 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:15 smithi152 ceph-mon[32701]: mds.? [v2:172.21.15.146:6834/1876533699,v1:172.21.15.146:6835/1876533699] up:boot 2024-04-25T11:23:16.178 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:15 smithi152 ceph-mon[32701]: fsmap 5 up:standby 2024-04-25T11:23:16.178 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:15 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T11:23:16.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:15 smithi146 ceph-mon[32821]: Deploying daemon mds.f on smithi152 2024-04-25T11:23:16.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:15 smithi146 ceph-mon[32821]: mds.? [v2:172.21.15.146:6834/1876533699,v1:172.21.15.146:6835/1876533699] up:boot 2024-04-25T11:23:16.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:15 smithi146 ceph-mon[32821]: fsmap 5 up:standby 2024-04-25T11:23:16.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:15 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T11:23:16.815 INFO:journalctl@ceph.mds.f.smithi152.stdout:Apr 25 11:23:16 smithi152 podman[63269]: 2024-04-25 11:23:16.674616132 +0000 UTC m=+0.017909287 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T11:23:16.815 INFO:journalctl@ceph.mds.f.smithi152.stdout:Apr 25 11:23:16 smithi152 podman[63269]: 2024-04-25 11:23:16.82135916 +0000 UTC m=+0.164652315 container create 840d6a5b4e77670b00b477263121551be0ad311e6013a88363b5ea71c6be2deb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-f, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T11:23:17.068 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:16 smithi152 ceph-mon[32701]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:17.068 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:16 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/2562710938' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:17.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:16 smithi069 ceph-mon[29188]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:17.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:16 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2562710938' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:17.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:16 smithi146 ceph-mon[32821]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:17.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:16 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/2562710938' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:17.481 INFO:journalctl@ceph.mds.f.smithi152.stdout:Apr 25 11:23:17 smithi152 podman[63269]: 2024-04-25 11:23:17.074577024 +0000 UTC m=+0.417870179 container init 840d6a5b4e77670b00b477263121551be0ad311e6013a88363b5ea71c6be2deb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-f, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, io.buildah.version=1.33.5) 2024-04-25T11:23:17.481 INFO:journalctl@ceph.mds.f.smithi152.stdout:Apr 25 11:23:17 smithi152 podman[63269]: 2024-04-25 11:23:17.077490401 +0000 UTC m=+0.420783559 container start 840d6a5b4e77670b00b477263121551be0ad311e6013a88363b5ea71c6be2deb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, ceph=True) 2024-04-25T11:23:17.481 INFO:journalctl@ceph.mds.f.smithi152.stdout:Apr 25 11:23:17 smithi152 ceph-mds[63289]: -- 172.21.15.152:0/2973601021 <== mon.1 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d572325a40 con 0x55d57236da80 2024-04-25T11:23:17.481 INFO:journalctl@ceph.mds.f.smithi152.stdout:Apr 25 11:23:17 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-f[63285]: starting mds.f at 2024-04-25T11:23:17.481 INFO:journalctl@ceph.mds.f.smithi152.stdout:Apr 25 11:23:17 smithi152 bash[63269]: 840d6a5b4e77670b00b477263121551be0ad311e6013a88363b5ea71c6be2deb 2024-04-25T11:23:17.482 INFO:journalctl@ceph.mds.f.smithi152.stdout:Apr 25 11:23:17 smithi152 systemd[1]: Started Ceph mds.f for f7ed73dc-02f4-11ef-bc93-c7b262605968. 2024-04-25T11:23:18.099 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:23:18.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.225+0000 7fa41490d640 1 -- 172.21.15.69:0/3316746792 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa410103c00 msgr2=0x7fa410104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:18.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.225+0000 7fa41490d640 1 --2- 172.21.15.69:0/3316746792 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa410103c00 0x7fa410104010 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8009900 tx=0x7fa3f802a410 comp rx=0 tx=0).stop 2024-04-25T11:23:18.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.225+0000 7fa41490d640 1 -- 172.21.15.69:0/3316746792 shutdown_connections 2024-04-25T11:23:18.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.225+0000 7fa41490d640 1 --2- 172.21.15.69:0/3316746792 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa410108900 0x7fa41010ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:18.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.225+0000 7fa41490d640 1 --2- 172.21.15.69:0/3316746792 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa410103c00 0x7fa410104010 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:18.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.225+0000 7fa41490d640 1 --2- 172.21.15.69:0/3316746792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4101029c0 0x7fa4101036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:18.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.225+0000 7fa41490d640 1 -- 172.21.15.69:0/3316746792 >> 172.21.15.69:0/3316746792 conn(0x7fa4100fdbd0 msgr2=0x7fa410100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:18.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.225+0000 7fa41490d640 1 -- 172.21.15.69:0/3316746792 shutdown_connections 2024-04-25T11:23:18.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.225+0000 7fa41490d640 1 -- 172.21.15.69:0/3316746792 wait complete. 2024-04-25T11:23:18.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.226+0000 7fa41490d640 1 Processor -- start 2024-04-25T11:23:18.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.226+0000 7fa41490d640 1 -- start start 2024-04-25T11:23:18.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.226+0000 7fa41490d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4101029c0 0x7fa41019a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:18.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.226+0000 7fa41490d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa410103c00 0x7fa41019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:18.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.226+0000 7fa41490d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa410108900 0x7fa4101a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:18.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.226+0000 7fa41490d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa41010e3a0 con 0x7fa4101029c0 2024-04-25T11:23:18.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.226+0000 7fa41490d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa41010e220 con 0x7fa410103c00 2024-04-25T11:23:18.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.226+0000 7fa41490d640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa41010e520 con 0x7fa410108900 2024-04-25T11:23:18.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.226+0000 7fa40e59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4101029c0 0x7fa41019a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:18.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.226+0000 7fa40e59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4101029c0 0x7fa41019a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57242/0 (socket says 172.21.15.69:57242) 2024-04-25T11:23:18.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.226+0000 7fa40e59c640 1 -- 172.21.15.69:0/207971582 learned_addr learned my addr 172.21.15.69:0/207971582 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:18.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.226+0000 7fa40ed9d640 1 --2- 172.21.15.69:0/207971582 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa410108900 0x7fa4101a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:18.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.226+0000 7fa407fff640 1 --2- 172.21.15.69:0/207971582 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa410103c00 0x7fa41019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:18.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.227+0000 7fa40e59c640 1 -- 172.21.15.69:0/207971582 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa410108900 msgr2=0x7fa4101a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:18.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.227+0000 7fa40e59c640 1 --2- 172.21.15.69:0/207971582 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa410108900 0x7fa4101a1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:18.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.227+0000 7fa40e59c640 1 -- 172.21.15.69:0/207971582 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa410103c00 msgr2=0x7fa41019a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:18.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.227+0000 7fa40e59c640 1 --2- 172.21.15.69:0/207971582 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa410103c00 0x7fa41019a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:18.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.227+0000 7fa40e59c640 1 -- 172.21.15.69:0/207971582 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4101a1c90 con 0x7fa4101029c0 2024-04-25T11:23:18.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.227+0000 7fa40e59c640 1 --2- 172.21.15.69:0/207971582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4101029c0 0x7fa41019a150 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa40000e9f0 tx=0x7fa40000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:18.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.230+0000 7fa4077fe640 1 -- 172.21.15.69:0/207971582 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40000ccc0 con 0x7fa4101029c0 2024-04-25T11:23:18.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.230+0000 7fa4077fe640 1 -- 172.21.15.69:0/207971582 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa4000044b0 con 0x7fa4101029c0 2024-04-25T11:23:18.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.230+0000 7fa41490d640 1 -- 172.21.15.69:0/207971582 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4101a1f80 con 0x7fa4101029c0 2024-04-25T11:23:18.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.230+0000 7fa4077fe640 1 -- 172.21.15.69:0/207971582 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa400010430 con 0x7fa4101029c0 2024-04-25T11:23:18.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.231+0000 7fa41490d640 1 -- 172.21.15.69:0/207971582 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4101a2440 con 0x7fa4101029c0 2024-04-25T11:23:18.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.231+0000 7fa41490d640 1 -- 172.21.15.69:0/207971582 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4101a2700 con 0x7fa4101029c0 2024-04-25T11:23:18.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.234+0000 7fa4077fe640 1 -- 172.21.15.69:0/207971582 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fa400003f90 con 0x7fa4101029c0 2024-04-25T11:23:18.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.234+0000 7fa4077fe640 1 --2- 172.21.15.69:0/207971582 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fa3fc0afb70 0x7fa3fc0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:18.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.234+0000 7fa407fff640 1 --2- 172.21.15.69:0/207971582 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fa3fc0afb70 0x7fa3fc0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:18.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.234+0000 7fa4077fe640 1 -- 172.21.15.69:0/207971582 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fa4000dea90 con 0x7fa4101029c0 2024-04-25T11:23:18.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.235+0000 7fa407fff640 1 --2- 172.21.15.69:0/207971582 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fa3fc0afb70 0x7fa3fc0b2080 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa41019b1f0 tx=0x7fa3f8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:18.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.238+0000 7fa4077fe640 1 -- 172.21.15.69:0/207971582 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa4000a7ba0 con 0x7fa4101029c0 2024-04-25T11:23:18.444 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:18 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:18.444 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:18 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:18.444 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:18 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:18.444 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:18 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:23:18.444 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:18 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:23:18.444 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:18 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:23:18.444 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:18 smithi069 ceph-mon[29188]: Deploying daemon mds.c on smithi152 2024-04-25T11:23:18.444 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:18 smithi069 ceph-mon[29188]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:18.444 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:18 smithi069 ceph-mon[29188]: mds.? [v2:172.21.15.152:6834/3188843362,v1:172.21.15.152:6835/3188843362] up:boot 2024-04-25T11:23:18.445 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:18 smithi069 ceph-mon[29188]: fsmap 6 up:standby 2024-04-25T11:23:18.445 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:18 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T11:23:18.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.455+0000 7fa41490d640 1 -- 172.21.15.69:0/207971582 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4101a28d0 con 0x7fa4101029c0 2024-04-25T11:23:18.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.456+0000 7fa4077fe640 1 -- 172.21.15.69:0/207971582 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5530 (secure 0 0 0) 0x7fa4000a74a0 con 0x7fa4101029c0 2024-04-25T11:23:18.450 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:23:18.450 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3399424094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3399424094},{"type":"v1","addr":"172.21.15.69:6835","nonce":3399424094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1463217771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1463217771},{"type":"v1","addr":"172.21.15.69:6837","nonce":1463217771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/4197482473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":4197482473},{"type":"v1","addr":"172.21.15.152:6833","nonce":4197482473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/4267885323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4267885323},{"type":"v1","addr":"172.21.15.146:6833","nonce":4267885323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24439,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/3188843362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3188843362},{"type":"v1","addr":"172.21.15.152:6835","nonce":3188843362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1876533699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1876533699},{"type":"v1","addr":"172.21.15.146:6835","nonce":1876533699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-25T11:23:18.451 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 7 2024-04-25T11:23:18.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.458+0000 7fa41490d640 1 -- 172.21.15.69:0/207971582 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fa3fc0afb70 msgr2=0x7fa3fc0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:18.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.458+0000 7fa41490d640 1 --2- 172.21.15.69:0/207971582 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fa3fc0afb70 0x7fa3fc0b2080 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa41019b1f0 tx=0x7fa3f8000bc0 comp rx=0 tx=0).stop 2024-04-25T11:23:18.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.458+0000 7fa41490d640 1 -- 172.21.15.69:0/207971582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4101029c0 msgr2=0x7fa41019a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:18.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.458+0000 7fa41490d640 1 --2- 172.21.15.69:0/207971582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4101029c0 0x7fa41019a150 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa40000e9f0 tx=0x7fa40000eeb0 comp rx=0 tx=0).stop 2024-04-25T11:23:18.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.458+0000 7fa41490d640 1 -- 172.21.15.69:0/207971582 shutdown_connections 2024-04-25T11:23:18.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.458+0000 7fa41490d640 1 --2- 172.21.15.69:0/207971582 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fa3fc0afb70 0x7fa3fc0b2080 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:18.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.458+0000 7fa41490d640 1 --2- 172.21.15.69:0/207971582 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa410108900 0x7fa4101a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:18.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.458+0000 7fa41490d640 1 --2- 172.21.15.69:0/207971582 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa410103c00 0x7fa41019a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:18.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.458+0000 7fa41490d640 1 --2- 172.21.15.69:0/207971582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4101029c0 0x7fa41019a150 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:18.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.458+0000 7fa41490d640 1 -- 172.21.15.69:0/207971582 >> 172.21.15.69:0/207971582 conn(0x7fa4100fdbd0 msgr2=0x7fa410109eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:18.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.458+0000 7fa41490d640 1 -- 172.21.15.69:0/207971582 shutdown_connections 2024-04-25T11:23:18.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:18.458+0000 7fa41490d640 1 -- 172.21.15.69:0/207971582 wait complete. 2024-04-25T11:23:18.481 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:18 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:18.481 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:18 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:18.481 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:18 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:18.481 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:18 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:23:18.481 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:18 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:23:18.481 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:18 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:23:18.481 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:18 smithi152 ceph-mon[32701]: Deploying daemon mds.c on smithi152 2024-04-25T11:23:18.481 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:18 smithi152 ceph-mon[32701]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:18.482 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:18 smithi152 ceph-mon[32701]: mds.? [v2:172.21.15.152:6834/3188843362,v1:172.21.15.152:6835/3188843362] up:boot 2024-04-25T11:23:18.482 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:18 smithi152 ceph-mon[32701]: fsmap 6 up:standby 2024-04-25T11:23:18.482 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:18 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T11:23:18.600 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:18 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:18.600 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:18 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:18.600 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:18 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:18.600 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:18 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:23:18.600 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:18 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:23:18.600 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:18 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:23:18.600 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:18 smithi146 ceph-mon[32821]: Deploying daemon mds.c on smithi152 2024-04-25T11:23:18.600 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:18 smithi146 ceph-mon[32821]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:18.600 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:18 smithi146 ceph-mon[32821]: mds.? [v2:172.21.15.152:6834/3188843362,v1:172.21.15.152:6835/3188843362] up:boot 2024-04-25T11:23:18.600 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:18 smithi146 ceph-mon[32821]: fsmap 6 up:standby 2024-04-25T11:23:18.601 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:18 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T11:23:19.600 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:19 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/207971582' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:19.693 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:19 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/207971582' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:19.720 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:19 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/207971582' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:19.720 INFO:journalctl@ceph.mds.c.smithi152.stdout:Apr 25 11:23:19 smithi152 systemd[1]: Starting Ceph mds.c for f7ed73dc-02f4-11ef-bc93-c7b262605968... 2024-04-25T11:23:19.981 INFO:journalctl@ceph.mds.c.smithi152.stdout:Apr 25 11:23:19 smithi152 podman[63580]: 2024-04-25 11:23:19.628231278 +0000 UTC m=+0.018554641 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T11:23:19.981 INFO:journalctl@ceph.mds.c.smithi152.stdout:Apr 25 11:23:19 smithi152 podman[63580]: 2024-04-25 11:23:19.742689606 +0000 UTC m=+0.133012961 container create f812ac2e4ae3797c8a5876c893af557326c18547d70f6f9cce7c70b95104138b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T11:23:20.292 INFO:journalctl@ceph.mds.c.smithi152.stdout:Apr 25 11:23:20 smithi152 podman[63580]: 2024-04-25 11:23:20.032010471 +0000 UTC m=+0.422333830 container init f812ac2e4ae3797c8a5876c893af557326c18547d70f6f9cce7c70b95104138b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-c, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-25T11:23:20.292 INFO:journalctl@ceph.mds.c.smithi152.stdout:Apr 25 11:23:20 smithi152 podman[63580]: 2024-04-25 11:23:20.034628311 +0000 UTC m=+0.424951675 container start f812ac2e4ae3797c8a5876c893af557326c18547d70f6f9cce7c70b95104138b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-c, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, io.buildah.version=1.33.5) 2024-04-25T11:23:20.292 INFO:journalctl@ceph.mds.c.smithi152.stdout:Apr 25 11:23:20 smithi152 ceph-mds[63599]: -- 172.21.15.152:0/3941076867 <== mon.1 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c299575a40 con 0x55c2995bda80 2024-04-25T11:23:20.292 INFO:journalctl@ceph.mds.c.smithi152.stdout:Apr 25 11:23:20 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-c[63595]: starting mds.c at 2024-04-25T11:23:20.292 INFO:journalctl@ceph.mds.c.smithi152.stdout:Apr 25 11:23:20 smithi152 bash[63580]: f812ac2e4ae3797c8a5876c893af557326c18547d70f6f9cce7c70b95104138b 2024-04-25T11:23:20.292 INFO:journalctl@ceph.mds.c.smithi152.stdout:Apr 25 11:23:20 smithi152 systemd[1]: Started Ceph mds.c for f7ed73dc-02f4-11ef-bc93-c7b262605968. 2024-04-25T11:23:20.465 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:23:20.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.591+0000 7f3dd0bce640 1 -- 172.21.15.69:0/1810684130 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3dcc105080 msgr2=0x7f3dcc105490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:20.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.591+0000 7f3dd0bce640 1 --2- 172.21.15.69:0/1810684130 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3dcc105080 0x7f3dcc105490 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc009900 tx=0x7f3dbc02a410 comp rx=0 tx=0).stop 2024-04-25T11:23:20.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.591+0000 7f3dd0bce640 1 -- 172.21.15.69:0/1810684130 shutdown_connections 2024-04-25T11:23:20.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.591+0000 7f3dd0bce640 1 --2- 172.21.15.69:0/1810684130 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3dcc102230 0x7f3dcc10bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:20.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.591+0000 7f3dd0bce640 1 --2- 172.21.15.69:0/1810684130 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3dcc101730 0x7f3dcc101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:20.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.591+0000 7f3dd0bce640 1 --2- 172.21.15.69:0/1810684130 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3dcc105080 0x7f3dcc105490 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:20.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.591+0000 7f3dd0bce640 1 -- 172.21.15.69:0/1810684130 >> 172.21.15.69:0/1810684130 conn(0x7f3dcc0fd330 msgr2=0x7f3dcc0ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:20.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.591+0000 7f3dd0bce640 1 -- 172.21.15.69:0/1810684130 shutdown_connections 2024-04-25T11:23:20.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.591+0000 7f3dd0bce640 1 -- 172.21.15.69:0/1810684130 wait complete. 2024-04-25T11:23:20.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.592+0000 7f3dd0bce640 1 Processor -- start 2024-04-25T11:23:20.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.592+0000 7f3dd0bce640 1 -- start start 2024-04-25T11:23:20.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.592+0000 7f3dd0bce640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3dcc101730 0x7f3dcc19a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:20.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.592+0000 7f3dd0bce640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3dcc102230 0x7f3dcc19a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:20.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.592+0000 7f3dd0bce640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3dcc105080 0x7f3dcc1a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:20.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.592+0000 7f3dd0bce640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3dcc10f4a0 con 0x7f3dcc101730 2024-04-25T11:23:20.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.592+0000 7f3dd0bce640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3dcc10f320 con 0x7f3dcc102230 2024-04-25T11:23:20.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.592+0000 7f3dd0bce640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3dcc10f620 con 0x7f3dcc105080 2024-04-25T11:23:20.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.592+0000 7f3dca59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3dcc101730 0x7f3dcc19a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:20.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.592+0000 7f3dca59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3dcc101730 0x7f3dcc19a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57252/0 (socket says 172.21.15.69:57252) 2024-04-25T11:23:20.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.592+0000 7f3dca59c640 1 -- 172.21.15.69:0/2379419619 learned_addr learned my addr 172.21.15.69:0/2379419619 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:20.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.592+0000 7f3dcad9d640 1 --2- 172.21.15.69:0/2379419619 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3dcc105080 0x7f3dcc1a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:20.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.592+0000 7f3dc9d9b640 1 --2- 172.21.15.69:0/2379419619 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3dcc102230 0x7f3dcc19a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:20.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.593+0000 7f3dca59c640 1 -- 172.21.15.69:0/2379419619 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3dcc105080 msgr2=0x7f3dcc1a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:20.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.593+0000 7f3dca59c640 1 --2- 172.21.15.69:0/2379419619 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3dcc105080 0x7f3dcc1a1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:20.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.593+0000 7f3dca59c640 1 -- 172.21.15.69:0/2379419619 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3dcc102230 msgr2=0x7f3dcc19a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:20.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.593+0000 7f3dca59c640 1 --2- 172.21.15.69:0/2379419619 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3dcc102230 0x7f3dcc19a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:20.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.593+0000 7f3dca59c640 1 -- 172.21.15.69:0/2379419619 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dcc1a1ce0 con 0x7f3dcc101730 2024-04-25T11:23:20.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.593+0000 7f3dca59c640 1 --2- 172.21.15.69:0/2379419619 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3dcc101730 0x7f3dcc19a1a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc005760 tx=0x7f3dbc004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:20.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.593+0000 7f3dbb7fe640 1 -- 172.21.15.69:0/2379419619 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc03f030 con 0x7f3dcc101730 2024-04-25T11:23:20.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.593+0000 7f3dbb7fe640 1 -- 172.21.15.69:0/2379419619 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3dbc004440 con 0x7f3dcc101730 2024-04-25T11:23:20.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.593+0000 7f3dd0bce640 1 -- 172.21.15.69:0/2379419619 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dcc1a1f70 con 0x7f3dcc101730 2024-04-25T11:23:20.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.593+0000 7f3dbb7fe640 1 -- 172.21.15.69:0/2379419619 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc035070 con 0x7f3dcc101730 2024-04-25T11:23:20.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.593+0000 7f3dd0bce640 1 -- 172.21.15.69:0/2379419619 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dcc1a23d0 con 0x7f3dcc101730 2024-04-25T11:23:20.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.594+0000 7f3dd0bce640 1 -- 172.21.15.69:0/2379419619 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dcc108a10 con 0x7f3dcc101730 2024-04-25T11:23:20.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.595+0000 7f3dbb7fe640 1 -- 172.21.15.69:0/2379419619 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f3dbc042030 con 0x7f3dcc101730 2024-04-25T11:23:20.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.596+0000 7f3dbb7fe640 1 --2- 172.21.15.69:0/2379419619 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f3db40afb70 0x7f3db40b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:20.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.596+0000 7f3dc9d9b640 1 --2- 172.21.15.69:0/2379419619 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f3db40afb70 0x7f3db40b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:20.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.596+0000 7f3dbb7fe640 1 -- 172.21.15.69:0/2379419619 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f3dbc0fee50 con 0x7f3dcc101730 2024-04-25T11:23:20.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.596+0000 7f3dc9d9b640 1 --2- 172.21.15.69:0/2379419619 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f3db40afb70 0x7f3db40b2080 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc19b240 tx=0x7f3dc000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:20.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.600+0000 7f3dbb7fe640 1 -- 172.21.15.69:0/2379419619 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3dbc0c7880 con 0x7f3dcc101730 2024-04-25T11:23:20.599 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:20 smithi146 ceph-mon[32821]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:20.599 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:20 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:20.600 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:20 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:20.600 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:20 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:20.600 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:20 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:23:20.600 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:20 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:23:20.600 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:20 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:23:20.693 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:20 smithi069 ceph-mon[29188]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:20.694 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:20 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:20.694 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:20 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:20.694 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:20 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:20.694 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:20 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:23:20.694 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:20 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:23:20.694 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:20 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:23:20.731 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:20 smithi152 ceph-mon[32701]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:20.731 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:20 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:20.731 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:20 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:20.731 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:20 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:20.731 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:20 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:23:20.731 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:20 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:23:20.732 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:20 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:23:20.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.816+0000 7f3dd0bce640 1 -- 172.21.15.69:0/2379419619 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3dcc02dc70 con 0x7f3dcc101730 2024-04-25T11:23:20.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.817+0000 7f3dbb7fe640 1 -- 172.21.15.69:0/2379419619 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6355 (secure 0 0 0) 0x7f3dbc0cbb80 con 0x7f3dcc101730 2024-04-25T11:23:20.811 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:23:20.811 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3399424094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3399424094},{"type":"v1","addr":"172.21.15.69:6835","nonce":3399424094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1463217771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1463217771},{"type":"v1","addr":"172.21.15.69:6837","nonce":1463217771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/4197482473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":4197482473},{"type":"v1","addr":"172.21.15.152:6833","nonce":4197482473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/4267885323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4267885323},{"type":"v1","addr":"172.21.15.146:6833","nonce":4267885323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24439,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/3188843362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3188843362},{"type":"v1","addr":"172.21.15.152:6835","nonce":3188843362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/563274982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":563274982},{"type":"v1","addr":"172.21.15.152:6837","nonce":563274982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1876533699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1876533699},{"type":"v1","addr":"172.21.15.146:6835","nonce":1876533699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-25T11:23:20.812 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 8 2024-04-25T11:23:20.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.819+0000 7f3dd0bce640 1 -- 172.21.15.69:0/2379419619 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f3db40afb70 msgr2=0x7f3db40b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:20.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.819+0000 7f3dd0bce640 1 --2- 172.21.15.69:0/2379419619 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f3db40afb70 0x7f3db40b2080 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc19b240 tx=0x7f3dc000b390 comp rx=0 tx=0).stop 2024-04-25T11:23:20.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.819+0000 7f3dd0bce640 1 -- 172.21.15.69:0/2379419619 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3dcc101730 msgr2=0x7f3dcc19a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:20.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.819+0000 7f3dd0bce640 1 --2- 172.21.15.69:0/2379419619 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3dcc101730 0x7f3dcc19a1a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc005760 tx=0x7f3dbc004040 comp rx=0 tx=0).stop 2024-04-25T11:23:20.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.819+0000 7f3dd0bce640 1 -- 172.21.15.69:0/2379419619 shutdown_connections 2024-04-25T11:23:20.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.819+0000 7f3dd0bce640 1 --2- 172.21.15.69:0/2379419619 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f3db40afb70 0x7f3db40b2080 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:20.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.819+0000 7f3dd0bce640 1 --2- 172.21.15.69:0/2379419619 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3dcc105080 0x7f3dcc1a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:20.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.819+0000 7f3dd0bce640 1 --2- 172.21.15.69:0/2379419619 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3dcc102230 0x7f3dcc19a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:20.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.819+0000 7f3dd0bce640 1 --2- 172.21.15.69:0/2379419619 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3dcc101730 0x7f3dcc19a1a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:20.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.819+0000 7f3dd0bce640 1 -- 172.21.15.69:0/2379419619 >> 172.21.15.69:0/2379419619 conn(0x7f3dcc0fd330 msgr2=0x7f3dcc105be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:20.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.819+0000 7f3dd0bce640 1 -- 172.21.15.69:0/2379419619 shutdown_connections 2024-04-25T11:23:20.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:20.819+0000 7f3dd0bce640 1 -- 172.21.15.69:0/2379419619 wait complete. 2024-04-25T11:23:21.481 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:21 smithi152 ceph-mon[32701]: Deploying daemon mds.h on smithi146 2024-04-25T11:23:21.481 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:21 smithi152 ceph-mon[32701]: mds.? [v2:172.21.15.152:6836/563274982,v1:172.21.15.152:6837/563274982] up:boot 2024-04-25T11:23:21.482 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:21 smithi152 ceph-mon[32701]: fsmap 7 up:standby 2024-04-25T11:23:21.482 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:21 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T11:23:21.482 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:21 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/2379419619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:21.599 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:21 smithi146 ceph-mon[32821]: Deploying daemon mds.h on smithi146 2024-04-25T11:23:21.600 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:21 smithi146 ceph-mon[32821]: mds.? [v2:172.21.15.152:6836/563274982,v1:172.21.15.152:6837/563274982] up:boot 2024-04-25T11:23:21.600 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:21 smithi146 ceph-mon[32821]: fsmap 7 up:standby 2024-04-25T11:23:21.600 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:21 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T11:23:21.600 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:21 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/2379419619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:21.694 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:21 smithi069 ceph-mon[29188]: Deploying daemon mds.h on smithi146 2024-04-25T11:23:21.694 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:21 smithi069 ceph-mon[29188]: mds.? [v2:172.21.15.152:6836/563274982,v1:172.21.15.152:6837/563274982] up:boot 2024-04-25T11:23:21.694 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:21 smithi069 ceph-mon[29188]: fsmap 7 up:standby 2024-04-25T11:23:21.694 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:21 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T11:23:21.694 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:21 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2379419619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:22.427 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:22 smithi146 ceph-mon[32821]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:22.694 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:22 smithi069 ceph-mon[29188]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:22.731 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:22 smithi152 ceph-mon[32701]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:22.827 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:23:22.830 INFO:journalctl@ceph.mds.h.smithi146.stdout:Apr 25 11:23:22 smithi146 systemd[1]: Starting Ceph mds.h for f7ed73dc-02f4-11ef-bc93-c7b262605968... 2024-04-25T11:23:22.830 INFO:journalctl@ceph.mds.h.smithi146.stdout:Apr 25 11:23:22 smithi146 podman[62565]: 2024-04-25 11:23:22.739130071 +0000 UTC m=+0.018503626 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T11:23:22.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:22.955+0000 7fcb8b667640 1 -- 172.21.15.69:0/4073399128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcb84109a30 msgr2=0x7fcb8410be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:22.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:22.955+0000 7fcb8b667640 1 --2- 172.21.15.69:0/4073399128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcb84109a30 0x7fcb8410be50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c009ca0 tx=0x7fcb7c02a8c0 comp rx=0 tx=0).stop 2024-04-25T11:23:22.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:22.955+0000 7fcb8b667640 1 -- 172.21.15.69:0/4073399128 shutdown_connections 2024-04-25T11:23:22.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:22.955+0000 7fcb8b667640 1 --2- 172.21.15.69:0/4073399128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcb84109a30 0x7fcb8410be50 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:22.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:22.955+0000 7fcb8b667640 1 --2- 172.21.15.69:0/4073399128 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb84104d30 0x7fcb84105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:22.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:22.955+0000 7fcb8b667640 1 --2- 172.21.15.69:0/4073399128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb84103af0 0x7fcb841047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:22.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:22.955+0000 7fcb8b667640 1 -- 172.21.15.69:0/4073399128 >> 172.21.15.69:0/4073399128 conn(0x7fcb840fed40 msgr2=0x7fcb84101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:22.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:22.955+0000 7fcb8b667640 1 -- 172.21.15.69:0/4073399128 shutdown_connections 2024-04-25T11:23:22.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:22.955+0000 7fcb8b667640 1 -- 172.21.15.69:0/4073399128 wait complete. 2024-04-25T11:23:22.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:22.955+0000 7fcb8b667640 1 Processor -- start 2024-04-25T11:23:22.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:22.955+0000 7fcb8b667640 1 -- start start 2024-04-25T11:23:22.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:22.956+0000 7fcb8b667640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb84103af0 0x7fcb8419a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:22.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:22.956+0000 7fcb8b667640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcb84104d30 0x7fcb8419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:22.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:22.956+0000 7fcb8b667640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb84109a30 0x7fcb841a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:22.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:22.956+0000 7fcb8b667640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8410f520 con 0x7fcb84104d30 2024-04-25T11:23:22.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:22.956+0000 7fcb8b667640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8410f3a0 con 0x7fcb84109a30 2024-04-25T11:23:22.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:22.956+0000 7fcb8b667640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8410f6a0 con 0x7fcb84103af0 2024-04-25T11:23:22.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:22.956+0000 7fcb88c02640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcb84104d30 0x7fcb8419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:22.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:22.956+0000 7fcb88c02640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcb84104d30 0x7fcb8419a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40698/0 (socket says 172.21.15.69:40698) 2024-04-25T11:23:22.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:22.956+0000 7fcb88c02640 1 -- 172.21.15.69:0/3165699120 learned_addr learned my addr 172.21.15.69:0/3165699120 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:22.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:22.956+0000 7fcb89c04640 1 --2- 172.21.15.69:0/3165699120 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb84109a30 0x7fcb841a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:22.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:22.956+0000 7fcb89403640 1 --2- 172.21.15.69:0/3165699120 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb84103af0 0x7fcb8419a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:22.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:22.956+0000 7fcb88c02640 1 -- 172.21.15.69:0/3165699120 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb84103af0 msgr2=0x7fcb8419a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:22.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:22.956+0000 7fcb88c02640 1 --2- 172.21.15.69:0/3165699120 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb84103af0 0x7fcb8419a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:22.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:22.956+0000 7fcb88c02640 1 -- 172.21.15.69:0/3165699120 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb84109a30 msgr2=0x7fcb841a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:22.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:22.956+0000 7fcb88c02640 1 --2- 172.21.15.69:0/3165699120 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb84109a30 0x7fcb841a1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:22.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:22.956+0000 7fcb88c02640 1 -- 172.21.15.69:0/3165699120 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb841a1c90 con 0x7fcb84104d30 2024-04-25T11:23:22.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:22.956+0000 7fcb88c02640 1 --2- 172.21.15.69:0/3165699120 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcb84104d30 0x7fcb8419a690 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcb8000e9f0 tx=0x7fcb8000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:22.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:22.956+0000 7fcb7a7fc640 1 -- 172.21.15.69:0/3165699120 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8000cd80 con 0x7fcb84104d30 2024-04-25T11:23:22.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:22.956+0000 7fcb7a7fc640 1 -- 172.21.15.69:0/3165699120 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcb800108c0 con 0x7fcb84104d30 2024-04-25T11:23:22.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:22.957+0000 7fcb8b667640 1 -- 172.21.15.69:0/3165699120 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb841a1f80 con 0x7fcb84104d30 2024-04-25T11:23:22.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:22.957+0000 7fcb7a7fc640 1 -- 172.21.15.69:0/3165699120 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb800187c0 con 0x7fcb84104d30 2024-04-25T11:23:22.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:22.957+0000 7fcb8b667640 1 -- 172.21.15.69:0/3165699120 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb841a23f0 con 0x7fcb84104d30 2024-04-25T11:23:22.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:22.958+0000 7fcb8b667640 1 -- 172.21.15.69:0/3165699120 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb84108840 con 0x7fcb84104d30 2024-04-25T11:23:22.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:22.960+0000 7fcb7a7fc640 1 -- 172.21.15.69:0/3165699120 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fcb80010430 con 0x7fcb84104d30 2024-04-25T11:23:22.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:22.960+0000 7fcb7a7fc640 1 --2- 172.21.15.69:0/3165699120 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fcb700afaf0 0x7fcb700b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:22.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:22.960+0000 7fcb7a7fc640 1 -- 172.21.15.69:0/3165699120 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fcb800defb0 con 0x7fcb84104d30 2024-04-25T11:23:22.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:22.960+0000 7fcb89403640 1 --2- 172.21.15.69:0/3165699120 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fcb700afaf0 0x7fcb700b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:22.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:22.961+0000 7fcb89403640 1 --2- 172.21.15.69:0/3165699120 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fcb700afaf0 0x7fcb700b2000 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcb841046f0 tx=0x7fcb74005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:22.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:22.964+0000 7fcb7a7fc640 1 -- 172.21.15.69:0/3165699120 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcb800a8030 con 0x7fcb84104d30 2024-04-25T11:23:23.100 INFO:journalctl@ceph.mds.h.smithi146.stdout:Apr 25 11:23:22 smithi146 podman[62565]: 2024-04-25 11:23:22.867471503 +0000 UTC m=+0.146845043 container create e2ddf7bbd96375b6f8dfdd19fb9df75158c8f510f8528f68a1812911b16d5357 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-h, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-25T11:23:23.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:23.180+0000 7fcb8b667640 1 -- 172.21.15.69:0/3165699120 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb8402db50 con 0x7fcb84104d30 2024-04-25T11:23:23.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:23.180+0000 7fcb7a7fc640 1 -- 172.21.15.69:0/3165699120 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6355 (secure 0 0 0) 0x7fcb800b00b0 con 0x7fcb84104d30 2024-04-25T11:23:23.175 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:23:23.175 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3399424094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3399424094},{"type":"v1","addr":"172.21.15.69:6835","nonce":3399424094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1463217771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1463217771},{"type":"v1","addr":"172.21.15.69:6837","nonce":1463217771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/4197482473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":4197482473},{"type":"v1","addr":"172.21.15.152:6833","nonce":4197482473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/4267885323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4267885323},{"type":"v1","addr":"172.21.15.146:6833","nonce":4267885323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24439,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/3188843362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3188843362},{"type":"v1","addr":"172.21.15.152:6835","nonce":3188843362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/563274982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":563274982},{"type":"v1","addr":"172.21.15.152:6837","nonce":563274982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1876533699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1876533699},{"type":"v1","addr":"172.21.15.146:6835","nonce":1876533699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-25T11:23:23.175 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 8 2024-04-25T11:23:23.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:23.182+0000 7fcb8b667640 1 -- 172.21.15.69:0/3165699120 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fcb700afaf0 msgr2=0x7fcb700b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:23.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:23.182+0000 7fcb8b667640 1 --2- 172.21.15.69:0/3165699120 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fcb700afaf0 0x7fcb700b2000 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcb841046f0 tx=0x7fcb74005f90 comp rx=0 tx=0).stop 2024-04-25T11:23:23.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:23.182+0000 7fcb8b667640 1 -- 172.21.15.69:0/3165699120 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcb84104d30 msgr2=0x7fcb8419a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:23.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:23.182+0000 7fcb8b667640 1 --2- 172.21.15.69:0/3165699120 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcb84104d30 0x7fcb8419a690 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcb8000e9f0 tx=0x7fcb8000eeb0 comp rx=0 tx=0).stop 2024-04-25T11:23:23.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:23.182+0000 7fcb8b667640 1 -- 172.21.15.69:0/3165699120 shutdown_connections 2024-04-25T11:23:23.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:23.182+0000 7fcb8b667640 1 --2- 172.21.15.69:0/3165699120 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fcb700afaf0 0x7fcb700b2000 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:23.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:23.182+0000 7fcb8b667640 1 --2- 172.21.15.69:0/3165699120 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb84109a30 0x7fcb841a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:23.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:23.182+0000 7fcb8b667640 1 --2- 172.21.15.69:0/3165699120 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcb84104d30 0x7fcb8419a690 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:23.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:23.182+0000 7fcb8b667640 1 --2- 172.21.15.69:0/3165699120 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb84103af0 0x7fcb8419a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:23.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:23.182+0000 7fcb8b667640 1 -- 172.21.15.69:0/3165699120 >> 172.21.15.69:0/3165699120 conn(0x7fcb840fed40 msgr2=0x7fcb84105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:23.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:23.182+0000 7fcb8b667640 1 -- 172.21.15.69:0/3165699120 shutdown_connections 2024-04-25T11:23:23.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:23.182+0000 7fcb8b667640 1 -- 172.21.15.69:0/3165699120 wait complete. 2024-04-25T11:23:23.444 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:23 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/3165699120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:23.444 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:23 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:23.444 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:23 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:23.444 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:23 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:23.444 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:23 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:23:23.444 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:23 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:23:23.444 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:23 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:23:23.600 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:23 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/3165699120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:23.600 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:23 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:23.600 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:23 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:23.600 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:23 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:23.600 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:23 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:23:23.600 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:23 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:23:23.600 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:23 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:23:23.600 INFO:journalctl@ceph.mds.h.smithi146.stdout:Apr 25 11:23:23 smithi146 podman[62565]: 2024-04-25 11:23:23.109453882 +0000 UTC m=+0.388827422 container init e2ddf7bbd96375b6f8dfdd19fb9df75158c8f510f8528f68a1812911b16d5357 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-25T11:23:23.600 INFO:journalctl@ceph.mds.h.smithi146.stdout:Apr 25 11:23:23 smithi146 podman[62565]: 2024-04-25 11:23:23.112068957 +0000 UTC m=+0.391442503 container start e2ddf7bbd96375b6f8dfdd19fb9df75158c8f510f8528f68a1812911b16d5357 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-h, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T11:23:23.601 INFO:journalctl@ceph.mds.h.smithi146.stdout:Apr 25 11:23:23 smithi146 ceph-mds[62584]: -- 172.21.15.146:0/1747763499 <== mon.2 v2:172.21.15.146:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5596e2e5fa40 con 0x5596e2ea7a80 2024-04-25T11:23:23.601 INFO:journalctl@ceph.mds.h.smithi146.stdout:Apr 25 11:23:23 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-h[62580]: starting mds.h at 2024-04-25T11:23:23.601 INFO:journalctl@ceph.mds.h.smithi146.stdout:Apr 25 11:23:23 smithi146 bash[62565]: e2ddf7bbd96375b6f8dfdd19fb9df75158c8f510f8528f68a1812911b16d5357 2024-04-25T11:23:23.601 INFO:journalctl@ceph.mds.h.smithi146.stdout:Apr 25 11:23:23 smithi146 systemd[1]: Started Ceph mds.h for f7ed73dc-02f4-11ef-bc93-c7b262605968. 2024-04-25T11:23:23.731 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:23 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/3165699120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:23.731 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:23 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:23.731 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:23 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:23.731 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:23 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:23.731 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:23 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:23:23.731 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:23 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:23:23.731 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:23 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:23:24.481 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:24 smithi152 ceph-mon[32701]: Deploying daemon mds.i on smithi152 2024-04-25T11:23:24.481 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:24 smithi152 ceph-mon[32701]: mds.? [v2:172.21.15.146:6836/2730317134,v1:172.21.15.146:6837/2730317134] up:boot 2024-04-25T11:23:24.481 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:24 smithi152 ceph-mon[32701]: fsmap 8 up:standby 2024-04-25T11:23:24.481 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:24 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T11:23:24.481 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:24 smithi152 ceph-mon[32701]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:24.600 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:24 smithi146 ceph-mon[32821]: Deploying daemon mds.i on smithi152 2024-04-25T11:23:24.600 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:24 smithi146 ceph-mon[32821]: mds.? [v2:172.21.15.146:6836/2730317134,v1:172.21.15.146:6837/2730317134] up:boot 2024-04-25T11:23:24.600 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:24 smithi146 ceph-mon[32821]: fsmap 8 up:standby 2024-04-25T11:23:24.600 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:24 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T11:23:24.600 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:24 smithi146 ceph-mon[32821]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:24.693 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:24 smithi069 ceph-mon[29188]: Deploying daemon mds.i on smithi152 2024-04-25T11:23:24.694 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:24 smithi069 ceph-mon[29188]: mds.? [v2:172.21.15.146:6836/2730317134,v1:172.21.15.146:6837/2730317134] up:boot 2024-04-25T11:23:24.694 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:24 smithi069 ceph-mon[29188]: fsmap 8 up:standby 2024-04-25T11:23:24.694 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:24 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T11:23:24.694 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:24 smithi069 ceph-mon[29188]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:25.189 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:23:25.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.318+0000 7fb4ad29f640 1 -- 172.21.15.69:0/2112550456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4a8101750 msgr2=0x7fb4a8101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:25.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.318+0000 7fb4ad29f640 1 --2- 172.21.15.69:0/2112550456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4a8101750 0x7fb4a8101bc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb49c009900 tx=0x7fb49c02a410 comp rx=0 tx=0).stop 2024-04-25T11:23:25.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.318+0000 7fb4ad29f640 1 -- 172.21.15.69:0/2112550456 shutdown_connections 2024-04-25T11:23:25.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.318+0000 7fb4ad29f640 1 --2- 172.21.15.69:0/2112550456 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4a8102250 0x7fb4a810bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:25.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.318+0000 7fb4ad29f640 1 --2- 172.21.15.69:0/2112550456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4a8101750 0x7fb4a8101bc0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:25.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.318+0000 7fb4ad29f640 1 --2- 172.21.15.69:0/2112550456 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4a81050a0 0x7fb4a81054b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:25.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.318+0000 7fb4ad29f640 1 -- 172.21.15.69:0/2112550456 >> 172.21.15.69:0/2112550456 conn(0x7fb4a80fd370 msgr2=0x7fb4a80ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:25.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.318+0000 7fb4ad29f640 1 -- 172.21.15.69:0/2112550456 shutdown_connections 2024-04-25T11:23:25.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.318+0000 7fb4ad29f640 1 -- 172.21.15.69:0/2112550456 wait complete. 2024-04-25T11:23:25.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.318+0000 7fb4ad29f640 1 Processor -- start 2024-04-25T11:23:25.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.319+0000 7fb4ad29f640 1 -- start start 2024-04-25T11:23:25.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.319+0000 7fb4ad29f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4a8101750 0x7fb4a819a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:25.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.319+0000 7fb4ad29f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4a8102250 0x7fb4a819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:25.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.319+0000 7fb4ad29f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4a81050a0 0x7fb4a81a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:25.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.319+0000 7fb4ad29f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a810f4e0 con 0x7fb4a81050a0 2024-04-25T11:23:25.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.319+0000 7fb4ad29f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a810f360 con 0x7fb4a8102250 2024-04-25T11:23:25.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.319+0000 7fb4ad29f640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a810f660 con 0x7fb4a8101750 2024-04-25T11:23:25.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.319+0000 7fb4a77fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4a81050a0 0x7fb4a81a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:25.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.319+0000 7fb4a77fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4a81050a0 0x7fb4a81a15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40708/0 (socket says 172.21.15.69:40708) 2024-04-25T11:23:25.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.319+0000 7fb4a77fe640 1 -- 172.21.15.69:0/3452225330 learned_addr learned my addr 172.21.15.69:0/3452225330 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:25.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.319+0000 7fb4a6ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4a8101750 0x7fb4a819a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:25.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.319+0000 7fb4a67fc640 1 --2- 172.21.15.69:0/3452225330 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4a8102250 0x7fb4a819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:25.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.319+0000 7fb4a77fe640 1 -- 172.21.15.69:0/3452225330 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4a8101750 msgr2=0x7fb4a819a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:25.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.319+0000 7fb4a77fe640 1 --2- 172.21.15.69:0/3452225330 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4a8101750 0x7fb4a819a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:25.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.319+0000 7fb4a77fe640 1 -- 172.21.15.69:0/3452225330 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4a8102250 msgr2=0x7fb4a819a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:25.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.319+0000 7fb4a77fe640 1 --2- 172.21.15.69:0/3452225330 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4a8102250 0x7fb4a819a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:25.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.319+0000 7fb4a77fe640 1 -- 172.21.15.69:0/3452225330 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a81a1c40 con 0x7fb4a81050a0 2024-04-25T11:23:25.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.319+0000 7fb4a67fc640 1 --2- 172.21.15.69:0/3452225330 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4a8102250 0x7fb4a819a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:23:25.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.319+0000 7fb4a77fe640 1 --2- 172.21.15.69:0/3452225330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4a81050a0 0x7fb4a81a15d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb4a000eb10 tx=0x7fb4a000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:25.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.320+0000 7fb48ffff640 1 -- 172.21.15.69:0/3452225330 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0004b20 con 0x7fb4a81050a0 2024-04-25T11:23:25.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.320+0000 7fb48ffff640 1 -- 172.21.15.69:0/3452225330 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb4a0004cc0 con 0x7fb4a81050a0 2024-04-25T11:23:25.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.320+0000 7fb4ad29f640 1 -- 172.21.15.69:0/3452225330 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4a81a1f30 con 0x7fb4a81050a0 2024-04-25T11:23:25.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.320+0000 7fb48ffff640 1 -- 172.21.15.69:0/3452225330 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0010690 con 0x7fb4a81050a0 2024-04-25T11:23:25.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.320+0000 7fb4ad29f640 1 -- 172.21.15.69:0/3452225330 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4a81a2390 con 0x7fb4a81050a0 2024-04-25T11:23:25.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.321+0000 7fb4ad29f640 1 -- 172.21.15.69:0/3452225330 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4a806a190 con 0x7fb4a81050a0 2024-04-25T11:23:25.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.322+0000 7fb48ffff640 1 -- 172.21.15.69:0/3452225330 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fb4a0005020 con 0x7fb4a81050a0 2024-04-25T11:23:25.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.323+0000 7fb48ffff640 1 --2- 172.21.15.69:0/3452225330 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fb4900afaf0 0x7fb4900b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:25.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.323+0000 7fb4a6ffd640 1 --2- 172.21.15.69:0/3452225330 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fb4900afaf0 0x7fb4900b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:25.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.323+0000 7fb48ffff640 1 -- 172.21.15.69:0/3452225330 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fb4a00ded90 con 0x7fb4a81050a0 2024-04-25T11:23:25.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.323+0000 7fb4a6ffd640 1 --2- 172.21.15.69:0/3452225330 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fb4900afaf0 0x7fb4900b2000 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb4a806b320 tx=0x7fb49800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:25.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.327+0000 7fb48ffff640 1 -- 172.21.15.69:0/3452225330 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb4a00a7e10 con 0x7fb4a81050a0 2024-04-25T11:23:25.481 INFO:journalctl@ceph.mds.i.smithi152.stdout:Apr 25 11:23:25 smithi152 systemd[1]: Starting Ceph mds.i for f7ed73dc-02f4-11ef-bc93-c7b262605968... 2024-04-25T11:23:25.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.538+0000 7fb4ad29f640 1 -- 172.21.15.69:0/3452225330 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb4a802dc10 con 0x7fb4a81050a0 2024-04-25T11:23:25.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.539+0000 7fb48ffff640 1 -- 172.21.15.69:0/3452225330 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7183 (secure 0 0 0) 0x7fb4a00a7710 con 0x7fb4a81050a0 2024-04-25T11:23:25.535 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:23:25.535 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3399424094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3399424094},{"type":"v1","addr":"172.21.15.69:6835","nonce":3399424094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1463217771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1463217771},{"type":"v1","addr":"172.21.15.69:6837","nonce":1463217771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/4197482473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":4197482473},{"type":"v1","addr":"172.21.15.152:6833","nonce":4197482473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/4267885323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4267885323},{"type":"v1","addr":"172.21.15.146:6833","nonce":4267885323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24439,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/3188843362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3188843362},{"type":"v1","addr":"172.21.15.152:6835","nonce":3188843362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/563274982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":563274982},{"type":"v1","addr":"172.21.15.152:6837","nonce":563274982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1876533699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1876533699},{"type":"v1","addr":"172.21.15.146:6835","nonce":1876533699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/2730317134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2730317134},{"type":"v1","addr":"172.21.15.146:6837","nonce":2730317134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-25T11:23:25.536 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 9 2024-04-25T11:23:25.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.541+0000 7fb4ad29f640 1 -- 172.21.15.69:0/3452225330 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fb4900afaf0 msgr2=0x7fb4900b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:25.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.541+0000 7fb4ad29f640 1 --2- 172.21.15.69:0/3452225330 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fb4900afaf0 0x7fb4900b2000 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb4a806b320 tx=0x7fb49800b390 comp rx=0 tx=0).stop 2024-04-25T11:23:25.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.541+0000 7fb4ad29f640 1 -- 172.21.15.69:0/3452225330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4a81050a0 msgr2=0x7fb4a81a15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:25.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.541+0000 7fb4ad29f640 1 --2- 172.21.15.69:0/3452225330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4a81050a0 0x7fb4a81a15d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb4a000eb10 tx=0x7fb4a000c2d0 comp rx=0 tx=0).stop 2024-04-25T11:23:25.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.541+0000 7fb4ad29f640 1 -- 172.21.15.69:0/3452225330 shutdown_connections 2024-04-25T11:23:25.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.541+0000 7fb4ad29f640 1 --2- 172.21.15.69:0/3452225330 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fb4900afaf0 0x7fb4900b2000 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:25.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.541+0000 7fb4ad29f640 1 --2- 172.21.15.69:0/3452225330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4a81050a0 0x7fb4a81a15d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:25.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.541+0000 7fb4ad29f640 1 --2- 172.21.15.69:0/3452225330 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb4a8102250 0x7fb4a819a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:25.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.541+0000 7fb4ad29f640 1 --2- 172.21.15.69:0/3452225330 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4a8101750 0x7fb4a819a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:25.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.541+0000 7fb4ad29f640 1 -- 172.21.15.69:0/3452225330 >> 172.21.15.69:0/3452225330 conn(0x7fb4a80fd370 msgr2=0x7fb4a8105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:25.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.541+0000 7fb4ad29f640 1 -- 172.21.15.69:0/3452225330 shutdown_connections 2024-04-25T11:23:25.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:25.541+0000 7fb4ad29f640 1 -- 172.21.15.69:0/3452225330 wait complete. 2024-04-25T11:23:25.850 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:25 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/3452225330' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:25.907 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:25 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/3452225330' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:25.908 INFO:journalctl@ceph.mds.i.smithi152.stdout:Apr 25 11:23:25 smithi152 podman[63890]: 2024-04-25 11:23:25.533320892 +0000 UTC m=+0.018167789 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T11:23:25.908 INFO:journalctl@ceph.mds.i.smithi152.stdout:Apr 25 11:23:25 smithi152 podman[63890]: 2024-04-25 11:23:25.652243833 +0000 UTC m=+0.137090712 container create 9c74b6b22d60ac474b098fcf8899c5ef7f34cff878a93f2d3349b068063447e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-25T11:23:25.946 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:25 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/3452225330' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:26.231 INFO:journalctl@ceph.mds.i.smithi152.stdout:Apr 25 11:23:25 smithi152 podman[63890]: 2024-04-25 11:23:25.913772595 +0000 UTC m=+0.398619474 container init 9c74b6b22d60ac474b098fcf8899c5ef7f34cff878a93f2d3349b068063447e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-i, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-25T11:23:26.231 INFO:journalctl@ceph.mds.i.smithi152.stdout:Apr 25 11:23:25 smithi152 podman[63890]: 2024-04-25 11:23:25.916992058 +0000 UTC m=+0.401838938 container start 9c74b6b22d60ac474b098fcf8899c5ef7f34cff878a93f2d3349b068063447e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-i, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T11:23:26.231 INFO:journalctl@ceph.mds.i.smithi152.stdout:Apr 25 11:23:25 smithi152 ceph-mds[63910]: -- 172.21.15.152:0/2736060024 <== mon.1 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560040ae3a40 con 0x560040b2b180 2024-04-25T11:23:26.231 INFO:journalctl@ceph.mds.i.smithi152.stdout:Apr 25 11:23:25 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-i[63906]: starting mds.i at 2024-04-25T11:23:26.232 INFO:journalctl@ceph.mds.i.smithi152.stdout:Apr 25 11:23:26 smithi152 bash[63890]: 9c74b6b22d60ac474b098fcf8899c5ef7f34cff878a93f2d3349b068063447e2 2024-04-25T11:23:26.232 INFO:journalctl@ceph.mds.i.smithi152.stdout:Apr 25 11:23:26 smithi152 systemd[1]: Started Ceph mds.i for f7ed73dc-02f4-11ef-bc93-c7b262605968. 2024-04-25T11:23:26.850 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:26 smithi146 ceph-mon[32821]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:26.850 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:26 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:26.850 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:26 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:26.850 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:26 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:26.850 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:26 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:23:26.850 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:26 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:23:26.850 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:26 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:23:26.892 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:26 smithi069 ceph-mon[29188]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:26.892 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:26 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:26.892 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:26 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:26.892 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:26 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:26.892 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:26 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:23:26.892 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:26 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:23:26.892 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:26 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:23:26.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:26 smithi152 ceph-mon[32701]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:26.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:26 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:26.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:26 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:26.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:26 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:26.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:26 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:23:26.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:26 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:23:26.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:26 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:23:27.548 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:23:27.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.665+0000 7ffac12aa640 1 -- 172.21.15.69:0/3573395116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffabc101740 msgr2=0x7ffabc101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:27.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.665+0000 7ffac12aa640 1 --2- 172.21.15.69:0/3573395116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffabc101740 0x7ffabc101bb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ffab0009900 tx=0x7ffab002a410 comp rx=0 tx=0).stop 2024-04-25T11:23:27.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.665+0000 7ffac12aa640 1 -- 172.21.15.69:0/3573395116 shutdown_connections 2024-04-25T11:23:27.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.665+0000 7ffac12aa640 1 --2- 172.21.15.69:0/3573395116 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffabc102240 0x7ffabc10bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:27.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.665+0000 7ffac12aa640 1 --2- 172.21.15.69:0/3573395116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffabc101740 0x7ffabc101bb0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:27.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.665+0000 7ffac12aa640 1 --2- 172.21.15.69:0/3573395116 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffabc105090 0x7ffabc1054a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:27.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.665+0000 7ffac12aa640 1 -- 172.21.15.69:0/3573395116 >> 172.21.15.69:0/3573395116 conn(0x7ffabc0fd340 msgr2=0x7ffabc0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:27.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.665+0000 7ffac12aa640 1 -- 172.21.15.69:0/3573395116 shutdown_connections 2024-04-25T11:23:27.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.665+0000 7ffac12aa640 1 -- 172.21.15.69:0/3573395116 wait complete. 2024-04-25T11:23:27.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.666+0000 7ffac12aa640 1 Processor -- start 2024-04-25T11:23:27.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.666+0000 7ffac12aa640 1 -- start start 2024-04-25T11:23:27.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.666+0000 7ffac12aa640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffabc101740 0x7ffabc19a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:27.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.666+0000 7ffac12aa640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffabc102240 0x7ffabc19a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:27.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.666+0000 7ffac12aa640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffabc105090 0x7ffabc1a15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:27.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.666+0000 7ffac12aa640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ffabc10f4b0 con 0x7ffabc101740 2024-04-25T11:23:27.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.666+0000 7ffac12aa640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ffabc10f330 con 0x7ffabc102240 2024-04-25T11:23:27.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.666+0000 7ffac12aa640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ffabc10f630 con 0x7ffabc105090 2024-04-25T11:23:27.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.667+0000 7ffabaffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffabc101740 0x7ffabc19a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:27.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.667+0000 7ffabaffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffabc101740 0x7ffabc19a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40716/0 (socket says 172.21.15.69:40716) 2024-04-25T11:23:27.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.667+0000 7ffabaffd640 1 -- 172.21.15.69:0/4054608624 learned_addr learned my addr 172.21.15.69:0/4054608624 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:27.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.667+0000 7ffaba7fc640 1 --2- 172.21.15.69:0/4054608624 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffabc102240 0x7ffabc19a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:27.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.667+0000 7ffabb7fe640 1 --2- 172.21.15.69:0/4054608624 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffabc105090 0x7ffabc1a15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:27.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.667+0000 7ffabaffd640 1 -- 172.21.15.69:0/4054608624 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffabc105090 msgr2=0x7ffabc1a15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:27.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.667+0000 7ffabaffd640 1 --2- 172.21.15.69:0/4054608624 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffabc105090 0x7ffabc1a15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:27.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.667+0000 7ffabaffd640 1 -- 172.21.15.69:0/4054608624 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffabc102240 msgr2=0x7ffabc19a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:27.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.667+0000 7ffabaffd640 1 --2- 172.21.15.69:0/4054608624 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffabc102240 0x7ffabc19a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:27.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.667+0000 7ffabaffd640 1 -- 172.21.15.69:0/4054608624 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffabc1a1bb0 con 0x7ffabc101740 2024-04-25T11:23:27.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.667+0000 7ffabb7fe640 1 --2- 172.21.15.69:0/4054608624 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffabc105090 0x7ffabc1a15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:23:27.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.667+0000 7ffabaffd640 1 --2- 172.21.15.69:0/4054608624 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffabc101740 0x7ffabc19a120 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ffaac00ea50 tx=0x7ffaac00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:27.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.667+0000 7ffaa3fff640 1 -- 172.21.15.69:0/4054608624 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaac00cdc0 con 0x7ffabc101740 2024-04-25T11:23:27.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.667+0000 7ffaa3fff640 1 -- 172.21.15.69:0/4054608624 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffaac011900 con 0x7ffabc101740 2024-04-25T11:23:27.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.667+0000 7ffac12aa640 1 -- 172.21.15.69:0/4054608624 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffabc1a1ea0 con 0x7ffabc101740 2024-04-25T11:23:27.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.667+0000 7ffaa3fff640 1 -- 172.21.15.69:0/4054608624 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaac01a970 con 0x7ffabc101740 2024-04-25T11:23:27.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.667+0000 7ffac12aa640 1 -- 172.21.15.69:0/4054608624 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffabc1a2360 con 0x7ffabc101740 2024-04-25T11:23:27.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.668+0000 7ffac12aa640 1 -- 172.21.15.69:0/4054608624 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffabc108a20 con 0x7ffabc101740 2024-04-25T11:23:27.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.670+0000 7ffaa3fff640 1 -- 172.21.15.69:0/4054608624 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7ffaac011470 con 0x7ffabc101740 2024-04-25T11:23:27.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.670+0000 7ffaa3fff640 1 --2- 172.21.15.69:0/4054608624 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ffaa40afb70 0x7ffaa40b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:27.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.670+0000 7ffaba7fc640 1 --2- 172.21.15.69:0/4054608624 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ffaa40afb70 0x7ffaa40b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:27.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.670+0000 7ffaa3fff640 1 -- 172.21.15.69:0/4054608624 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7ffaac0dedb0 con 0x7ffabc101740 2024-04-25T11:23:27.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.671+0000 7ffaba7fc640 1 --2- 172.21.15.69:0/4054608624 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ffaa40afb70 0x7ffaa40b2080 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ffabc19b1c0 tx=0x7ffab0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:27.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.674+0000 7ffaa3fff640 1 -- 172.21.15.69:0/4054608624 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffaac055ff0 con 0x7ffabc101740 2024-04-25T11:23:27.849 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:27 smithi146 ceph-mon[32821]: Deploying daemon mds.g on smithi069 2024-04-25T11:23:27.850 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:27 smithi146 ceph-mon[32821]: mds.? [v2:172.21.15.152:6838/3008615657,v1:172.21.15.152:6839/3008615657] up:boot 2024-04-25T11:23:27.850 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:27 smithi146 ceph-mon[32821]: fsmap 9 up:standby 2024-04-25T11:23:27.850 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:27 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T11:23:27.861 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:27 smithi069 ceph-mon[29188]: Deploying daemon mds.g on smithi069 2024-04-25T11:23:27.861 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:27 smithi069 ceph-mon[29188]: mds.? [v2:172.21.15.152:6838/3008615657,v1:172.21.15.152:6839/3008615657] up:boot 2024-04-25T11:23:27.861 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:27 smithi069 ceph-mon[29188]: fsmap 9 up:standby 2024-04-25T11:23:27.861 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:27 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T11:23:27.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.895+0000 7ffac12aa640 1 -- 172.21.15.69:0/4054608624 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffabc02dda0 con 0x7ffabc101740 2024-04-25T11:23:27.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.896+0000 7ffaa3fff640 1 -- 172.21.15.69:0/4054608624 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8013 (secure 0 0 0) 0x7ffaac016030 con 0x7ffabc101740 2024-04-25T11:23:27.890 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:23:27.891 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3399424094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3399424094},{"type":"v1","addr":"172.21.15.69:6835","nonce":3399424094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1463217771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1463217771},{"type":"v1","addr":"172.21.15.69:6837","nonce":1463217771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/4197482473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":4197482473},{"type":"v1","addr":"172.21.15.152:6833","nonce":4197482473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/4267885323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4267885323},{"type":"v1","addr":"172.21.15.146:6833","nonce":4267885323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24439,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/3188843362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3188843362},{"type":"v1","addr":"172.21.15.152:6835","nonce":3188843362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/563274982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":563274982},{"type":"v1","addr":"172.21.15.152:6837","nonce":563274982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1876533699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1876533699},{"type":"v1","addr":"172.21.15.146:6835","nonce":1876533699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24475,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6839/3008615657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":3008615657},{"type":"v1","addr":"172.21.15.152:6839","nonce":3008615657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/2730317134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2730317134},{"type":"v1","addr":"172.21.15.146:6837","nonce":2730317134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-25T11:23:27.891 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 10 2024-04-25T11:23:27.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.898+0000 7ffac12aa640 1 -- 172.21.15.69:0/4054608624 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ffaa40afb70 msgr2=0x7ffaa40b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:27.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.898+0000 7ffac12aa640 1 --2- 172.21.15.69:0/4054608624 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ffaa40afb70 0x7ffaa40b2080 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ffabc19b1c0 tx=0x7ffab0000bc0 comp rx=0 tx=0).stop 2024-04-25T11:23:27.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.898+0000 7ffac12aa640 1 -- 172.21.15.69:0/4054608624 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffabc101740 msgr2=0x7ffabc19a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:27.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.898+0000 7ffac12aa640 1 --2- 172.21.15.69:0/4054608624 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffabc101740 0x7ffabc19a120 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ffaac00ea50 tx=0x7ffaac00ef10 comp rx=0 tx=0).stop 2024-04-25T11:23:27.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.898+0000 7ffac12aa640 1 -- 172.21.15.69:0/4054608624 shutdown_connections 2024-04-25T11:23:27.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.898+0000 7ffac12aa640 1 --2- 172.21.15.69:0/4054608624 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ffaa40afb70 0x7ffaa40b2080 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:27.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.898+0000 7ffac12aa640 1 --2- 172.21.15.69:0/4054608624 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffabc105090 0x7ffabc1a15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:27.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.898+0000 7ffac12aa640 1 --2- 172.21.15.69:0/4054608624 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffabc102240 0x7ffabc19a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:27.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.898+0000 7ffac12aa640 1 --2- 172.21.15.69:0/4054608624 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffabc101740 0x7ffabc19a120 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:27.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.898+0000 7ffac12aa640 1 -- 172.21.15.69:0/4054608624 >> 172.21.15.69:0/4054608624 conn(0x7ffabc0fd340 msgr2=0x7ffabc105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:27.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.898+0000 7ffac12aa640 1 -- 172.21.15.69:0/4054608624 shutdown_connections 2024-04-25T11:23:27.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:27.898+0000 7ffac12aa640 1 -- 172.21.15.69:0/4054608624 wait complete. 2024-04-25T11:23:27.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:27 smithi152 ceph-mon[32701]: Deploying daemon mds.g on smithi069 2024-04-25T11:23:27.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:27 smithi152 ceph-mon[32701]: mds.? [v2:172.21.15.152:6838/3008615657,v1:172.21.15.152:6839/3008615657] up:boot 2024-04-25T11:23:27.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:27 smithi152 ceph-mon[32701]: fsmap 9 up:standby 2024-04-25T11:23:27.982 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:27 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T11:23:28.444 INFO:journalctl@ceph.mds.g.smithi069.stdout:Apr 25 11:23:28 smithi069 systemd[1]: Starting Ceph mds.g for f7ed73dc-02f4-11ef-bc93-c7b262605968... 2024-04-25T11:23:28.849 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:28 smithi146 ceph-mon[32821]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:28.850 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:28 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/4054608624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:28.871 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:28 smithi069 ceph-mon[29188]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:28.871 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:28 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/4054608624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:28.871 INFO:journalctl@ceph.mds.g.smithi069.stdout:Apr 25 11:23:28 smithi069 podman[70502]: 2024-04-25 11:23:28.470040097 +0000 UTC m=+0.017963955 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T11:23:28.871 INFO:journalctl@ceph.mds.g.smithi069.stdout:Apr 25 11:23:28 smithi069 podman[70502]: 2024-04-25 11:23:28.595180536 +0000 UTC m=+0.143104436 container create bf0ac5d14701c33778f0c8a19479d9eeec80c1d48bd3c69ce2875d652f85172f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-g, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5) 2024-04-25T11:23:28.871 INFO:journalctl@ceph.mds.g.smithi069.stdout:Apr 25 11:23:28 smithi069 podman[70502]: 2024-04-25 11:23:28.877500002 +0000 UTC m=+0.425423859 container init bf0ac5d14701c33778f0c8a19479d9eeec80c1d48bd3c69ce2875d652f85172f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-g, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-25T11:23:28.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:28 smithi152 ceph-mon[32701]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:28.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:28 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/4054608624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:29.193 INFO:journalctl@ceph.mds.g.smithi069.stdout:Apr 25 11:23:28 smithi069 podman[70502]: 2024-04-25 11:23:28.880674479 +0000 UTC m=+0.428598346 container start bf0ac5d14701c33778f0c8a19479d9eeec80c1d48bd3c69ce2875d652f85172f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-g, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-25T11:23:29.194 INFO:journalctl@ceph.mds.g.smithi069.stdout:Apr 25 11:23:28 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-g[70517]: starting mds.g at 2024-04-25T11:23:29.194 INFO:journalctl@ceph.mds.g.smithi069.stdout:Apr 25 11:23:28 smithi069 bash[70502]: bf0ac5d14701c33778f0c8a19479d9eeec80c1d48bd3c69ce2875d652f85172f 2024-04-25T11:23:29.194 INFO:journalctl@ceph.mds.g.smithi069.stdout:Apr 25 11:23:29 smithi069 systemd[1]: Started Ceph mds.g for f7ed73dc-02f4-11ef-bc93-c7b262605968. 2024-04-25T11:23:29.904 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:23:30.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.026+0000 7f1d847e3640 1 -- 172.21.15.69:0/3957368088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d7c102a10 msgr2=0x7f1d7c103710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:30.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.026+0000 7f1d847e3640 1 --2- 172.21.15.69:0/3957368088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d7c102a10 0x7f1d7c103710 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1d74009ca0 tx=0x7f1d7402a8c0 comp rx=0 tx=0).stop 2024-04-25T11:23:30.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.026+0000 7f1d847e3640 1 -- 172.21.15.69:0/3957368088 shutdown_connections 2024-04-25T11:23:30.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.026+0000 7f1d847e3640 1 --2- 172.21.15.69:0/3957368088 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d7c108950 0x7f1d7c10ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:30.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.026+0000 7f1d847e3640 1 --2- 172.21.15.69:0/3957368088 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d7c103c50 0x7f1d7c104060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:30.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.026+0000 7f1d847e3640 1 --2- 172.21.15.69:0/3957368088 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d7c102a10 0x7f1d7c103710 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:30.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.026+0000 7f1d847e3640 1 -- 172.21.15.69:0/3957368088 >> 172.21.15.69:0/3957368088 conn(0x7f1d7c0fdc00 msgr2=0x7f1d7c1000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:30.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.026+0000 7f1d847e3640 1 -- 172.21.15.69:0/3957368088 shutdown_connections 2024-04-25T11:23:30.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.026+0000 7f1d847e3640 1 -- 172.21.15.69:0/3957368088 wait complete. 2024-04-25T11:23:30.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.027+0000 7f1d847e3640 1 Processor -- start 2024-04-25T11:23:30.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.027+0000 7f1d847e3640 1 -- start start 2024-04-25T11:23:30.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.027+0000 7f1d847e3640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d7c102a10 0x7f1d7c0ffd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:30.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.027+0000 7f1d847e3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d7c103c50 0x7f1d7c0fe420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:30.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.027+0000 7f1d847e3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d7c108950 0x7f1d7c0fe960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:30.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.027+0000 7f1d847e3640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1d7c10e410 con 0x7f1d7c108950 2024-04-25T11:23:30.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.027+0000 7f1d847e3640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1d7c10e290 con 0x7f1d7c103c50 2024-04-25T11:23:30.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.027+0000 7f1d847e3640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1d7c10e590 con 0x7f1d7c102a10 2024-04-25T11:23:30.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.027+0000 7f1d82d80640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d7c108950 0x7f1d7c0fe960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:30.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.027+0000 7f1d82d80640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d7c108950 0x7f1d7c0fe960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40748/0 (socket says 172.21.15.69:40748) 2024-04-25T11:23:30.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.027+0000 7f1d82d80640 1 -- 172.21.15.69:0/1859557772 learned_addr learned my addr 172.21.15.69:0/1859557772 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:30.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.027+0000 7f1d81d7e640 1 --2- 172.21.15.69:0/1859557772 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d7c103c50 0x7f1d7c0fe420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:30.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.027+0000 7f1d82d80640 1 -- 172.21.15.69:0/1859557772 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d7c102a10 msgr2=0x7f1d7c0ffd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:30.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.027+0000 7f1d8257f640 1 --2- 172.21.15.69:0/1859557772 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d7c102a10 0x7f1d7c0ffd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:30.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.027+0000 7f1d82d80640 1 --2- 172.21.15.69:0/1859557772 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d7c102a10 0x7f1d7c0ffd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:30.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.027+0000 7f1d82d80640 1 -- 172.21.15.69:0/1859557772 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d7c103c50 msgr2=0x7f1d7c0fe420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:30.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.027+0000 7f1d82d80640 1 --2- 172.21.15.69:0/1859557772 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d7c103c50 0x7f1d7c0fe420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:30.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.027+0000 7f1d82d80640 1 -- 172.21.15.69:0/1859557772 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d7c0ff090 con 0x7f1d7c108950 2024-04-25T11:23:30.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.027+0000 7f1d8257f640 1 --2- 172.21.15.69:0/1859557772 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d7c102a10 0x7f1d7c0ffd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:23:30.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.028+0000 7f1d82d80640 1 --2- 172.21.15.69:0/1859557772 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d7c108950 0x7f1d7c0fe960 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c00c970 tx=0x7f1d6c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:30.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.028+0000 7f1d737fe640 1 -- 172.21.15.69:0/1859557772 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c007bb0 con 0x7f1d7c108950 2024-04-25T11:23:30.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.028+0000 7f1d737fe640 1 -- 172.21.15.69:0/1859557772 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1d6c007d50 con 0x7f1d7c108950 2024-04-25T11:23:30.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.028+0000 7f1d847e3640 1 -- 172.21.15.69:0/1859557772 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d7c1a42e0 con 0x7f1d7c108950 2024-04-25T11:23:30.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.028+0000 7f1d737fe640 1 -- 172.21.15.69:0/1859557772 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c004d50 con 0x7f1d7c108950 2024-04-25T11:23:30.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.028+0000 7f1d847e3640 1 -- 172.21.15.69:0/1859557772 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d7c1a47a0 con 0x7f1d7c108950 2024-04-25T11:23:30.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.029+0000 7f1d847e3640 1 -- 172.21.15.69:0/1859557772 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d7c1a4a60 con 0x7f1d7c108950 2024-04-25T11:23:30.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.030+0000 7f1d737fe640 1 -- 172.21.15.69:0/1859557772 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f1d6c005000 con 0x7f1d7c108950 2024-04-25T11:23:30.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.031+0000 7f1d737fe640 1 --2- 172.21.15.69:0/1859557772 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f1d640afb70 0x7f1d640b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:30.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.031+0000 7f1d8257f640 1 --2- 172.21.15.69:0/1859557772 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f1d640afb70 0x7f1d640b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:30.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.031+0000 7f1d737fe640 1 -- 172.21.15.69:0/1859557772 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f1d6c0de860 con 0x7f1d7c108950 2024-04-25T11:23:30.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.031+0000 7f1d8257f640 1 --2- 172.21.15.69:0/1859557772 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f1d640afb70 0x7f1d640b2080 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1d74006010 tx=0x7f1d74000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:30.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.034+0000 7f1d737fe640 1 -- 172.21.15.69:0/1859557772 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1d6c014030 con 0x7f1d7c108950 2024-04-25T11:23:30.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.252+0000 7f1d847e3640 1 -- 172.21.15.69:0/1859557772 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d7c1a4c30 con 0x7f1d7c108950 2024-04-25T11:23:30.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.253+0000 7f1d737fe640 1 -- 172.21.15.69:0/1859557772 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8836 (secure 0 0 0) 0x7f1d6c0a7960 con 0x7f1d7c108950 2024-04-25T11:23:30.247 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:23:30.247 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3399424094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3399424094},{"type":"v1","addr":"172.21.15.69:6835","nonce":3399424094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1463217771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1463217771},{"type":"v1","addr":"172.21.15.69:6837","nonce":1463217771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/966316001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":966316001},{"type":"v1","addr":"172.21.15.69:6839","nonce":966316001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/4197482473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":4197482473},{"type":"v1","addr":"172.21.15.152:6833","nonce":4197482473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/4267885323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4267885323},{"type":"v1","addr":"172.21.15.146:6833","nonce":4267885323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24439,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/3188843362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3188843362},{"type":"v1","addr":"172.21.15.152:6835","nonce":3188843362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/563274982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":563274982},{"type":"v1","addr":"172.21.15.152:6837","nonce":563274982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1876533699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1876533699},{"type":"v1","addr":"172.21.15.146:6835","nonce":1876533699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24475,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6839/3008615657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":3008615657},{"type":"v1","addr":"172.21.15.152:6839","nonce":3008615657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/2730317134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2730317134},{"type":"v1","addr":"172.21.15.146:6837","nonce":2730317134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-25T11:23:30.247 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 11 2024-04-25T11:23:30.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.255+0000 7f1d847e3640 1 -- 172.21.15.69:0/1859557772 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f1d640afb70 msgr2=0x7f1d640b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:30.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.255+0000 7f1d847e3640 1 --2- 172.21.15.69:0/1859557772 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f1d640afb70 0x7f1d640b2080 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1d74006010 tx=0x7f1d74000f40 comp rx=0 tx=0).stop 2024-04-25T11:23:30.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.255+0000 7f1d847e3640 1 -- 172.21.15.69:0/1859557772 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d7c108950 msgr2=0x7f1d7c0fe960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:30.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.255+0000 7f1d847e3640 1 --2- 172.21.15.69:0/1859557772 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d7c108950 0x7f1d7c0fe960 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c00c970 tx=0x7f1d6c00ce30 comp rx=0 tx=0).stop 2024-04-25T11:23:30.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.255+0000 7f1d847e3640 1 -- 172.21.15.69:0/1859557772 shutdown_connections 2024-04-25T11:23:30.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.255+0000 7f1d847e3640 1 --2- 172.21.15.69:0/1859557772 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f1d640afb70 0x7f1d640b2080 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:30.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.255+0000 7f1d847e3640 1 --2- 172.21.15.69:0/1859557772 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d7c108950 0x7f1d7c0fe960 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:30.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.255+0000 7f1d847e3640 1 --2- 172.21.15.69:0/1859557772 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d7c103c50 0x7f1d7c0fe420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:30.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.255+0000 7f1d847e3640 1 --2- 172.21.15.69:0/1859557772 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d7c102a10 0x7f1d7c0ffd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:30.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.255+0000 7f1d847e3640 1 -- 172.21.15.69:0/1859557772 >> 172.21.15.69:0/1859557772 conn(0x7f1d7c0fdc00 msgr2=0x7f1d7c06b230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:30.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.255+0000 7f1d847e3640 1 -- 172.21.15.69:0/1859557772 shutdown_connections 2024-04-25T11:23:30.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:30.255+0000 7f1d847e3640 1 -- 172.21.15.69:0/1859557772 wait complete. 2024-04-25T11:23:30.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:30 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:30.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:30 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:30.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:30 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:30.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:30 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:23:30.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:30 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:23:30.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:30 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:23:30.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:30 smithi069 ceph-mon[29188]: Deploying daemon mds.a on smithi069 2024-04-25T11:23:30.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:30 smithi069 ceph-mon[29188]: mds.? [v2:172.21.15.69:6838/966316001,v1:172.21.15.69:6839/966316001] up:boot 2024-04-25T11:23:30.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:30 smithi069 ceph-mon[29188]: fsmap 10 up:standby 2024-04-25T11:23:30.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:30 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T11:23:30.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:30 smithi069 ceph-mon[29188]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:30.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:30 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:30.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:30 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:30.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:30 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:30.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:30 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:23:30.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:30 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:23:30.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:30 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:23:30.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:30 smithi146 ceph-mon[32821]: Deploying daemon mds.a on smithi069 2024-04-25T11:23:30.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:30 smithi146 ceph-mon[32821]: mds.? [v2:172.21.15.69:6838/966316001,v1:172.21.15.69:6839/966316001] up:boot 2024-04-25T11:23:30.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:30 smithi146 ceph-mon[32821]: fsmap 10 up:standby 2024-04-25T11:23:30.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:30 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T11:23:30.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:30 smithi146 ceph-mon[32821]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:30.481 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:30 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:30.481 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:30 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:30.481 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:30 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:30.481 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:30 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:23:30.482 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:30 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:23:30.482 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:30 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:23:30.482 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:30 smithi152 ceph-mon[32701]: Deploying daemon mds.a on smithi069 2024-04-25T11:23:30.482 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:30 smithi152 ceph-mon[32701]: mds.? [v2:172.21.15.69:6838/966316001,v1:172.21.15.69:6839/966316001] up:boot 2024-04-25T11:23:30.482 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:30 smithi152 ceph-mon[32701]: fsmap 10 up:standby 2024-04-25T11:23:30.482 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:30 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T11:23:30.482 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:30 smithi152 ceph-mon[32701]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:31.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:31 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/1859557772' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:31.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:31 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/1859557772' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:31.482 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:31 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/1859557772' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:31.486 INFO:journalctl@ceph.mds.a.smithi069.stdout:Apr 25 11:23:31 smithi069 podman[70861]: 2024-04-25 11:23:31.372812764 +0000 UTC m=+0.018220710 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T11:23:31.486 INFO:journalctl@ceph.mds.a.smithi069.stdout:Apr 25 11:23:31 smithi069 podman[70861]: 2024-04-25 11:23:31.493229592 +0000 UTC m=+0.138637536 container create 4700ba9fb4542b04c7647c680d7ef56151496850689f124d848dcb75a2535af1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True) 2024-04-25T11:23:32.049 INFO:journalctl@ceph.mds.a.smithi069.stdout:Apr 25 11:23:31 smithi069 podman[70861]: 2024-04-25 11:23:31.746216892 +0000 UTC m=+0.391624836 container init 4700ba9fb4542b04c7647c680d7ef56151496850689f124d848dcb75a2535af1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-25T11:23:32.049 INFO:journalctl@ceph.mds.a.smithi069.stdout:Apr 25 11:23:31 smithi069 podman[70861]: 2024-04-25 11:23:31.74905102 +0000 UTC m=+0.394458965 container start 4700ba9fb4542b04c7647c680d7ef56151496850689f124d848dcb75a2535af1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-25T11:23:32.050 INFO:journalctl@ceph.mds.a.smithi069.stdout:Apr 25 11:23:31 smithi069 ceph-mds[70881]: -- 172.21.15.69:0/1880844599 <== mon.0 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d8cc489a40 con 0x55d8cc4d1180 2024-04-25T11:23:32.050 INFO:journalctl@ceph.mds.a.smithi069.stdout:Apr 25 11:23:31 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-a[70877]: starting mds.a at 2024-04-25T11:23:32.050 INFO:journalctl@ceph.mds.a.smithi069.stdout:Apr 25 11:23:31 smithi069 bash[70861]: 4700ba9fb4542b04c7647c680d7ef56151496850689f124d848dcb75a2535af1 2024-04-25T11:23:32.050 INFO:journalctl@ceph.mds.a.smithi069.stdout:Apr 25 11:23:31 smithi069 systemd[1]: Started Ceph mds.a for f7ed73dc-02f4-11ef-bc93-c7b262605968. 2024-04-25T11:23:32.263 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:23:32.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:32 smithi146 ceph-mon[32821]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:32.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:32 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:32.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:32 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:32.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:32 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:32.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:32 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:23:32.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:32 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:23:32.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:32 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:23:32.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.382+0000 7f487552e640 1 -- 172.21.15.69:0/2344101266 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48701022b0 msgr2=0x7f487010bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:32.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.382+0000 7f487552e640 1 --2- 172.21.15.69:0/2344101266 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48701022b0 0x7f487010bd10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4868009900 tx=0x7f486802a410 comp rx=0 tx=0).stop 2024-04-25T11:23:32.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.382+0000 7f487552e640 1 -- 172.21.15.69:0/2344101266 shutdown_connections 2024-04-25T11:23:32.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.382+0000 7f487552e640 1 --2- 172.21.15.69:0/2344101266 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48701022b0 0x7f487010bd10 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:32.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.382+0000 7f487552e640 1 --2- 172.21.15.69:0/2344101266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f48701017b0 0x7f4870101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:32.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.382+0000 7f487552e640 1 --2- 172.21.15.69:0/2344101266 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4870105100 0x7f4870105510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:32.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.382+0000 7f487552e640 1 -- 172.21.15.69:0/2344101266 >> 172.21.15.69:0/2344101266 conn(0x7f48700fd3b0 msgr2=0x7f48700ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:32.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.382+0000 7f487552e640 1 -- 172.21.15.69:0/2344101266 shutdown_connections 2024-04-25T11:23:32.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.382+0000 7f487552e640 1 -- 172.21.15.69:0/2344101266 wait complete. 2024-04-25T11:23:32.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.382+0000 7f487552e640 1 Processor -- start 2024-04-25T11:23:32.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.383+0000 7f487552e640 1 -- start start 2024-04-25T11:23:32.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.383+0000 7f487552e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48701017b0 0x7f48701042b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:32.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.383+0000 7f487552e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f48701022b0 0x7f4870102950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:32.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.383+0000 7f487552e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4870105100 0x7f4870102e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:32.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.383+0000 7f487552e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f487010f4d0 con 0x7f4870105100 2024-04-25T11:23:32.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.383+0000 7f487552e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f487010f350 con 0x7f48701017b0 2024-04-25T11:23:32.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.383+0000 7f487552e640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f487010f650 con 0x7f48701022b0 2024-04-25T11:23:32.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.383+0000 7f486f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4870105100 0x7f4870102e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:32.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.383+0000 7f486f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4870105100 0x7f4870102e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37662/0 (socket says 172.21.15.69:37662) 2024-04-25T11:23:32.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.383+0000 7f486f7fe640 1 -- 172.21.15.69:0/138465528 learned_addr learned my addr 172.21.15.69:0/138465528 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:32.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.383+0000 7f486effd640 1 --2- 172.21.15.69:0/138465528 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48701017b0 0x7f48701042b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:32.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.383+0000 7f486e7fc640 1 --2- 172.21.15.69:0/138465528 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f48701022b0 0x7f4870102950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:32.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.383+0000 7f486f7fe640 1 -- 172.21.15.69:0/138465528 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f48701022b0 msgr2=0x7f4870102950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:32.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.383+0000 7f486f7fe640 1 --2- 172.21.15.69:0/138465528 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f48701022b0 0x7f4870102950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:32.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.383+0000 7f486f7fe640 1 -- 172.21.15.69:0/138465528 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48701017b0 msgr2=0x7f48701042b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:32.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.383+0000 7f486f7fe640 1 --2- 172.21.15.69:0/138465528 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48701017b0 0x7f48701042b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:32.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.383+0000 7f486f7fe640 1 -- 172.21.15.69:0/138465528 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48701035c0 con 0x7f4870105100 2024-04-25T11:23:32.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.383+0000 7f486f7fe640 1 --2- 172.21.15.69:0/138465528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4870105100 0x7f4870102e90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4868005ab0 tx=0x7f48680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:32.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.384+0000 7f4857fff640 1 -- 172.21.15.69:0/138465528 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486803f030 con 0x7f4870105100 2024-04-25T11:23:32.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.384+0000 7f487552e640 1 -- 172.21.15.69:0/138465528 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48701a4190 con 0x7f4870105100 2024-04-25T11:23:32.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.384+0000 7f4857fff640 1 -- 172.21.15.69:0/138465528 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4868003c90 con 0x7f4870105100 2024-04-25T11:23:32.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.384+0000 7f4857fff640 1 -- 172.21.15.69:0/138465528 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4868035070 con 0x7f4870105100 2024-04-25T11:23:32.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.384+0000 7f487552e640 1 -- 172.21.15.69:0/138465528 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48701a45a0 con 0x7f4870105100 2024-04-25T11:23:32.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.385+0000 7f487552e640 1 -- 172.21.15.69:0/138465528 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f487010aaa0 con 0x7f4870105100 2024-04-25T11:23:32.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.386+0000 7f4857fff640 1 -- 172.21.15.69:0/138465528 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f4868042030 con 0x7f4870105100 2024-04-25T11:23:32.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.387+0000 7f4857fff640 1 --2- 172.21.15.69:0/138465528 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f48580afaf0 0x7f48580b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:32.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.387+0000 7f486effd640 1 --2- 172.21.15.69:0/138465528 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f48580afaf0 0x7f48580b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:32.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.387+0000 7f4857fff640 1 -- 172.21.15.69:0/138465528 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f48680ff200 con 0x7f4870105100 2024-04-25T11:23:32.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.387+0000 7f486effd640 1 --2- 172.21.15.69:0/138465528 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f48580afaf0 0x7f48580b2000 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f487006b2a0 tx=0x7f4860009670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:32.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.390+0000 7f4857fff640 1 -- 172.21.15.69:0/138465528 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f48680c8280 con 0x7f4870105100 2024-04-25T11:23:32.443 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:32 smithi069 ceph-mon[29188]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:32.443 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:32 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:32.444 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:32 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:32.444 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:32 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:32.444 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:32 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:23:32.444 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:32 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:23:32.444 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:32 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:23:32.481 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:32 smithi152 ceph-mon[32701]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:32.481 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:32 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:32.481 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:32 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:32.481 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:32 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:32.481 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:32 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:23:32.481 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:32 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:23:32.481 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:32 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:23:32.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.609+0000 7f487552e640 1 -- 172.21.15.69:0/138465528 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f487002db50 con 0x7f4870105100 2024-04-25T11:23:32.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.610+0000 7f4857fff640 1 -- 172.21.15.69:0/138465528 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9662 (secure 0 0 0) 0x7f48680c7b80 con 0x7f4870105100 2024-04-25T11:23:32.604 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:23:32.605 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3399424094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3399424094},{"type":"v1","addr":"172.21.15.69:6835","nonce":3399424094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1463217771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1463217771},{"type":"v1","addr":"172.21.15.69:6837","nonce":1463217771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/966316001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":966316001},{"type":"v1","addr":"172.21.15.69:6839","nonce":966316001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/2060225181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":2060225181},{"type":"v1","addr":"172.21.15.69:6841","nonce":2060225181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/4197482473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":4197482473},{"type":"v1","addr":"172.21.15.152:6833","nonce":4197482473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/4267885323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4267885323},{"type":"v1","addr":"172.21.15.146:6833","nonce":4267885323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24439,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/3188843362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3188843362},{"type":"v1","addr":"172.21.15.152:6835","nonce":3188843362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/563274982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":563274982},{"type":"v1","addr":"172.21.15.152:6837","nonce":563274982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1876533699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1876533699},{"type":"v1","addr":"172.21.15.146:6835","nonce":1876533699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24475,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6839/3008615657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":3008615657},{"type":"v1","addr":"172.21.15.152:6839","nonce":3008615657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/2730317134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2730317134},{"type":"v1","addr":"172.21.15.146:6837","nonce":2730317134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-25T11:23:32.605 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 12 2024-04-25T11:23:32.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.612+0000 7f487552e640 1 -- 172.21.15.69:0/138465528 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f48580afaf0 msgr2=0x7f48580b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:32.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.612+0000 7f487552e640 1 --2- 172.21.15.69:0/138465528 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f48580afaf0 0x7f48580b2000 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f487006b2a0 tx=0x7f4860009670 comp rx=0 tx=0).stop 2024-04-25T11:23:32.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.612+0000 7f487552e640 1 -- 172.21.15.69:0/138465528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4870105100 msgr2=0x7f4870102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:32.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.612+0000 7f487552e640 1 --2- 172.21.15.69:0/138465528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4870105100 0x7f4870102e90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4868005ab0 tx=0x7f48680042f0 comp rx=0 tx=0).stop 2024-04-25T11:23:32.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.612+0000 7f487552e640 1 -- 172.21.15.69:0/138465528 shutdown_connections 2024-04-25T11:23:32.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.612+0000 7f487552e640 1 --2- 172.21.15.69:0/138465528 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f48580afaf0 0x7f48580b2000 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:32.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.612+0000 7f487552e640 1 --2- 172.21.15.69:0/138465528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4870105100 0x7f4870102e90 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:32.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.612+0000 7f487552e640 1 --2- 172.21.15.69:0/138465528 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f48701022b0 0x7f4870102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:32.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.612+0000 7f487552e640 1 --2- 172.21.15.69:0/138465528 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48701017b0 0x7f48701042b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:32.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.612+0000 7f487552e640 1 -- 172.21.15.69:0/138465528 >> 172.21.15.69:0/138465528 conn(0x7f48700fd3b0 msgr2=0x7f4870107c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:32.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.612+0000 7f487552e640 1 -- 172.21.15.69:0/138465528 shutdown_connections 2024-04-25T11:23:32.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:32.612+0000 7f487552e640 1 -- 172.21.15.69:0/138465528 wait complete. 2024-04-25T11:23:33.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:33 smithi146 ceph-mon[32821]: Deploying daemon mds.b on smithi146 2024-04-25T11:23:33.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:33 smithi146 ceph-mon[32821]: mds.? [v2:172.21.15.69:6840/2060225181,v1:172.21.15.69:6841/2060225181] up:boot 2024-04-25T11:23:33.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:33 smithi146 ceph-mon[32821]: fsmap 11 up:standby 2024-04-25T11:23:33.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:33 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T11:23:33.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:33 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/138465528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:33.443 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:33 smithi069 ceph-mon[29188]: Deploying daemon mds.b on smithi146 2024-04-25T11:23:33.444 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:33 smithi069 ceph-mon[29188]: mds.? [v2:172.21.15.69:6840/2060225181,v1:172.21.15.69:6841/2060225181] up:boot 2024-04-25T11:23:33.444 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:33 smithi069 ceph-mon[29188]: fsmap 11 up:standby 2024-04-25T11:23:33.444 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:33 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T11:23:33.444 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:33 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/138465528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:33.481 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:33 smithi152 ceph-mon[32701]: Deploying daemon mds.b on smithi146 2024-04-25T11:23:33.481 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:33 smithi152 ceph-mon[32701]: mds.? [v2:172.21.15.69:6840/2060225181,v1:172.21.15.69:6841/2060225181] up:boot 2024-04-25T11:23:33.481 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:33 smithi152 ceph-mon[32701]: fsmap 11 up:standby 2024-04-25T11:23:33.481 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:33 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T11:23:33.482 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:33 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/138465528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:34.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:34 smithi146 ceph-mon[32821]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:34.350 INFO:journalctl@ceph.mds.b.smithi146.stdout:Apr 25 11:23:34 smithi146 systemd[1]: Starting Ceph mds.b for f7ed73dc-02f4-11ef-bc93-c7b262605968... 2024-04-25T11:23:34.443 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:34 smithi069 ceph-mon[29188]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:34.481 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:34 smithi152 ceph-mon[32701]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:34.619 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:23:34.669 INFO:journalctl@ceph.mds.b.smithi146.stdout:Apr 25 11:23:34 smithi146 podman[62875]: 2024-04-25 11:23:34.318464677 +0000 UTC m=+0.019035349 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T11:23:34.670 INFO:journalctl@ceph.mds.b.smithi146.stdout:Apr 25 11:23:34 smithi146 podman[62875]: 2024-04-25 11:23:34.420672468 +0000 UTC m=+0.121243126 container create 26a675f1aaf4783579a8997fad18a4002a69e90fc6dfe997eca2184b5ba356d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T11:23:34.670 INFO:journalctl@ceph.mds.b.smithi146.stdout:Apr 25 11:23:34 smithi146 podman[62875]: 2024-04-25 11:23:34.640076468 +0000 UTC m=+0.340647128 container init 26a675f1aaf4783579a8997fad18a4002a69e90fc6dfe997eca2184b5ba356d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-b, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T11:23:34.670 INFO:journalctl@ceph.mds.b.smithi146.stdout:Apr 25 11:23:34 smithi146 podman[62875]: 2024-04-25 11:23:34.6426707 +0000 UTC m=+0.343241359 container start 26a675f1aaf4783579a8997fad18a4002a69e90fc6dfe997eca2184b5ba356d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-25T11:23:34.670 INFO:journalctl@ceph.mds.b.smithi146.stdout:Apr 25 11:23:34 smithi146 ceph-mds[62896]: -- 172.21.15.146:0/3318717201 <== mon.2 v2:172.21.15.146:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5555bc031a40 con 0x5555bc079180 2024-04-25T11:23:34.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.747+0000 7fc2bc6d1640 1 -- 172.21.15.69:0/3580531901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2b4103c00 msgr2=0x7fc2b4104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:34.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.747+0000 7fc2bc6d1640 1 --2- 172.21.15.69:0/3580531901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2b4103c00 0x7fc2b4104010 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0009900 tx=0x7fc2b002a410 comp rx=0 tx=0).stop 2024-04-25T11:23:34.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.747+0000 7fc2bc6d1640 1 -- 172.21.15.69:0/3580531901 shutdown_connections 2024-04-25T11:23:34.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.747+0000 7fc2bc6d1640 1 --2- 172.21.15.69:0/3580531901 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc2b4108900 0x7fc2b410ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:34.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.747+0000 7fc2bc6d1640 1 --2- 172.21.15.69:0/3580531901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2b4103c00 0x7fc2b4104010 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:34.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.747+0000 7fc2bc6d1640 1 --2- 172.21.15.69:0/3580531901 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2b41029c0 0x7fc2b41036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:34.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.747+0000 7fc2bc6d1640 1 -- 172.21.15.69:0/3580531901 >> 172.21.15.69:0/3580531901 conn(0x7fc2b40fdbd0 msgr2=0x7fc2b4100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:34.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.747+0000 7fc2bc6d1640 1 -- 172.21.15.69:0/3580531901 shutdown_connections 2024-04-25T11:23:34.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.747+0000 7fc2bc6d1640 1 -- 172.21.15.69:0/3580531901 wait complete. 2024-04-25T11:23:34.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.748+0000 7fc2bc6d1640 1 Processor -- start 2024-04-25T11:23:34.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.748+0000 7fc2bc6d1640 1 -- start start 2024-04-25T11:23:34.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.748+0000 7fc2bc6d1640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc2b41029c0 0x7fc2b4199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:34.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.748+0000 7fc2bc6d1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2b4103c00 0x7fc2b4199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:34.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.748+0000 7fc2bc6d1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2b4108900 0x7fc2b41a0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:34.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.748+0000 7fc2bc6d1640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b410e3a0 con 0x7fc2b4103c00 2024-04-25T11:23:34.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.748+0000 7fc2bc6d1640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b410e220 con 0x7fc2b4108900 2024-04-25T11:23:34.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.748+0000 7fc2bc6d1640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b410e520 con 0x7fc2b41029c0 2024-04-25T11:23:34.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.748+0000 7fc2b9c6c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2b4103c00 0x7fc2b4199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:34.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.748+0000 7fc2b9c6c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2b4103c00 0x7fc2b4199570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37676/0 (socket says 172.21.15.69:37676) 2024-04-25T11:23:34.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.748+0000 7fc2b9c6c640 1 -- 172.21.15.69:0/3044882383 learned_addr learned my addr 172.21.15.69:0/3044882383 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:34.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.748+0000 7fc2bac6e640 1 --2- 172.21.15.69:0/3044882383 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2b4108900 0x7fc2b41a0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:34.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.748+0000 7fc2b9c6c640 1 -- 172.21.15.69:0/3044882383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc2b41029c0 msgr2=0x7fc2b4199030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:34.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.748+0000 7fc2ba46d640 1 --2- 172.21.15.69:0/3044882383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc2b41029c0 0x7fc2b4199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:34.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.748+0000 7fc2b9c6c640 1 --2- 172.21.15.69:0/3044882383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc2b41029c0 0x7fc2b4199030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:34.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.748+0000 7fc2b9c6c640 1 -- 172.21.15.69:0/3044882383 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2b4108900 msgr2=0x7fc2b41a0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:34.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.748+0000 7fc2b9c6c640 1 --2- 172.21.15.69:0/3044882383 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2b4108900 0x7fc2b41a0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:34.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.748+0000 7fc2b9c6c640 1 -- 172.21.15.69:0/3044882383 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2b41a0b70 con 0x7fc2b4103c00 2024-04-25T11:23:34.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.749+0000 7fc2b9c6c640 1 --2- 172.21.15.69:0/3044882383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2b4103c00 0x7fc2b4199570 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0009230 tx=0x7fc2b00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:34.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.749+0000 7fc2ab7fe640 1 -- 172.21.15.69:0/3044882383 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b003f030 con 0x7fc2b4103c00 2024-04-25T11:23:34.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.749+0000 7fc2bc6d1640 1 -- 172.21.15.69:0/3044882383 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2b41a0e00 con 0x7fc2b4103c00 2024-04-25T11:23:34.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.749+0000 7fc2ab7fe640 1 -- 172.21.15.69:0/3044882383 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc2b0003c90 con 0x7fc2b4103c00 2024-04-25T11:23:34.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.749+0000 7fc2ab7fe640 1 -- 172.21.15.69:0/3044882383 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b0035070 con 0x7fc2b4103c00 2024-04-25T11:23:34.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.749+0000 7fc2bc6d1640 1 -- 172.21.15.69:0/3044882383 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2b41a1210 con 0x7fc2b4103c00 2024-04-25T11:23:34.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.750+0000 7fc2bc6d1640 1 -- 172.21.15.69:0/3044882383 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2b4107710 con 0x7fc2b4103c00 2024-04-25T11:23:34.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.751+0000 7fc2ab7fe640 1 -- 172.21.15.69:0/3044882383 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fc2b0042030 con 0x7fc2b4103c00 2024-04-25T11:23:34.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.752+0000 7fc2ab7fe640 1 --2- 172.21.15.69:0/3044882383 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc2a00afaf0 0x7fc2a00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:34.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.752+0000 7fc2ba46d640 1 --2- 172.21.15.69:0/3044882383 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc2a00afaf0 0x7fc2a00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:34.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.752+0000 7fc2ab7fe640 1 -- 172.21.15.69:0/3044882383 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fc2b00ff220 con 0x7fc2b4103c00 2024-04-25T11:23:34.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.752+0000 7fc2ba46d640 1 --2- 172.21.15.69:0/3044882383 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc2a00afaf0 0x7fc2a00b2000 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4005fd0 tx=0x7fc2a4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:34.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.756+0000 7fc2ab7fe640 1 -- 172.21.15.69:0/3044882383 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc2b00c82a0 con 0x7fc2b4103c00 2024-04-25T11:23:34.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.978+0000 7fc2bc6d1640 1 -- 172.21.15.69:0/3044882383 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc2b402db50 con 0x7fc2b4103c00 2024-04-25T11:23:34.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.979+0000 7fc2ab7fe640 1 -- 172.21.15.69:0/3044882383 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9662 (secure 0 0 0) 0x7fc2b00c7ba0 con 0x7fc2b4103c00 2024-04-25T11:23:34.973 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:23:34.974 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3399424094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3399424094},{"type":"v1","addr":"172.21.15.69:6835","nonce":3399424094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1463217771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1463217771},{"type":"v1","addr":"172.21.15.69:6837","nonce":1463217771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/966316001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":966316001},{"type":"v1","addr":"172.21.15.69:6839","nonce":966316001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/2060225181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":2060225181},{"type":"v1","addr":"172.21.15.69:6841","nonce":2060225181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/4197482473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":4197482473},{"type":"v1","addr":"172.21.15.152:6833","nonce":4197482473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/4267885323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4267885323},{"type":"v1","addr":"172.21.15.146:6833","nonce":4267885323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24439,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/3188843362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3188843362},{"type":"v1","addr":"172.21.15.152:6835","nonce":3188843362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/563274982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":563274982},{"type":"v1","addr":"172.21.15.152:6837","nonce":563274982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1876533699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1876533699},{"type":"v1","addr":"172.21.15.146:6835","nonce":1876533699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24475,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6839/3008615657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":3008615657},{"type":"v1","addr":"172.21.15.152:6839","nonce":3008615657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/2730317134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2730317134},{"type":"v1","addr":"172.21.15.146:6837","nonce":2730317134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-25T11:23:34.974 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 12 2024-04-25T11:23:34.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.981+0000 7fc2bc6d1640 1 -- 172.21.15.69:0/3044882383 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc2a00afaf0 msgr2=0x7fc2a00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:34.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.981+0000 7fc2bc6d1640 1 --2- 172.21.15.69:0/3044882383 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc2a00afaf0 0x7fc2a00b2000 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4005fd0 tx=0x7fc2a4005850 comp rx=0 tx=0).stop 2024-04-25T11:23:34.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.981+0000 7fc2bc6d1640 1 -- 172.21.15.69:0/3044882383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2b4103c00 msgr2=0x7fc2b4199570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:34.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.981+0000 7fc2bc6d1640 1 --2- 172.21.15.69:0/3044882383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2b4103c00 0x7fc2b4199570 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0009230 tx=0x7fc2b00042f0 comp rx=0 tx=0).stop 2024-04-25T11:23:34.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.981+0000 7fc2bc6d1640 1 -- 172.21.15.69:0/3044882383 shutdown_connections 2024-04-25T11:23:34.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.981+0000 7fc2bc6d1640 1 --2- 172.21.15.69:0/3044882383 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc2a00afaf0 0x7fc2a00b2000 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:34.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.981+0000 7fc2bc6d1640 1 --2- 172.21.15.69:0/3044882383 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2b4108900 0x7fc2b41a0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:34.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.981+0000 7fc2bc6d1640 1 --2- 172.21.15.69:0/3044882383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2b4103c00 0x7fc2b4199570 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:34.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.981+0000 7fc2bc6d1640 1 --2- 172.21.15.69:0/3044882383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc2b41029c0 0x7fc2b4199030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:34.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.981+0000 7fc2bc6d1640 1 -- 172.21.15.69:0/3044882383 >> 172.21.15.69:0/3044882383 conn(0x7fc2b40fdbd0 msgr2=0x7fc2b41048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:34.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.981+0000 7fc2bc6d1640 1 -- 172.21.15.69:0/3044882383 shutdown_connections 2024-04-25T11:23:34.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:34.981+0000 7fc2bc6d1640 1 -- 172.21.15.69:0/3044882383 wait complete. 2024-04-25T11:23:35.099 INFO:journalctl@ceph.mds.b.smithi146.stdout:Apr 25 11:23:34 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-b[62892]: starting mds.b at 2024-04-25T11:23:35.099 INFO:journalctl@ceph.mds.b.smithi146.stdout:Apr 25 11:23:34 smithi146 bash[62875]: 26a675f1aaf4783579a8997fad18a4002a69e90fc6dfe997eca2184b5ba356d9 2024-04-25T11:23:35.099 INFO:journalctl@ceph.mds.b.smithi146.stdout:Apr 25 11:23:34 smithi146 systemd[1]: Started Ceph mds.b for f7ed73dc-02f4-11ef-bc93-c7b262605968. 2024-04-25T11:23:36.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:35 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:36.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:35 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:36.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:35 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:36.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:35 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:36.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:35 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:36.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:35 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:23:36.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:35 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/3044882383' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:36.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:35 smithi069 ceph-mon[29188]: mds.? [v2:172.21.15.146:6838/2013283448,v1:172.21.15.146:6839/2013283448] up:boot 2024-04-25T11:23:36.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:35 smithi069 ceph-mon[29188]: fsmap 12 up:standby 2024-04-25T11:23:36.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:35 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T11:23:36.027 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:35 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:36.027 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:35 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:36.027 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:35 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:36.027 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:35 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:36.027 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:35 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:36.027 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:35 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:23:36.027 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:35 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/3044882383' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:36.028 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:35 smithi152 ceph-mon[32701]: mds.? [v2:172.21.15.146:6838/2013283448,v1:172.21.15.146:6839/2013283448] up:boot 2024-04-25T11:23:36.028 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:35 smithi152 ceph-mon[32701]: fsmap 12 up:standby 2024-04-25T11:23:36.028 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:35 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T11:23:36.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:35 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:36.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:35 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:36.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:35 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:36.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:35 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:36.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:35 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:36.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:35 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:23:36.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:35 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/3044882383' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:36.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:35 smithi146 ceph-mon[32821]: mds.? [v2:172.21.15.146:6838/2013283448,v1:172.21.15.146:6839/2013283448] up:boot 2024-04-25T11:23:36.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:35 smithi146 ceph-mon[32821]: fsmap 12 up:standby 2024-04-25T11:23:36.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:35 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T11:23:36.988 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:23:37.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:36 smithi146 ceph-mon[32821]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:37.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:36 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:37.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:36 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:37.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:36 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:37.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:36 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:37.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:36 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:37.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:36 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:37.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:36 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:23:37.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:36 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:23:37.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:36 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:37.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:36 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:37.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.105+0000 7f4b48b0d640 1 -- 172.21.15.69:0/1126463062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b441029c0 msgr2=0x7f4b441036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:37.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.105+0000 7f4b48b0d640 1 --2- 172.21.15.69:0/1126463062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b441029c0 0x7f4b441036c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4b34009900 tx=0x7f4b3402a410 comp rx=0 tx=0).stop 2024-04-25T11:23:37.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.105+0000 7f4b48b0d640 1 -- 172.21.15.69:0/1126463062 shutdown_connections 2024-04-25T11:23:37.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.105+0000 7f4b48b0d640 1 --2- 172.21.15.69:0/1126463062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b44108900 0x7f4b4410ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:37.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.105+0000 7f4b48b0d640 1 --2- 172.21.15.69:0/1126463062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b44103c00 0x7f4b44104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:37.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.105+0000 7f4b48b0d640 1 --2- 172.21.15.69:0/1126463062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b441029c0 0x7f4b441036c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:37.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.105+0000 7f4b48b0d640 1 -- 172.21.15.69:0/1126463062 >> 172.21.15.69:0/1126463062 conn(0x7f4b440fdbd0 msgr2=0x7f4b44100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:37.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.105+0000 7f4b48b0d640 1 -- 172.21.15.69:0/1126463062 shutdown_connections 2024-04-25T11:23:37.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.105+0000 7f4b48b0d640 1 -- 172.21.15.69:0/1126463062 wait complete. 2024-04-25T11:23:37.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.106+0000 7f4b48b0d640 1 Processor -- start 2024-04-25T11:23:37.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.106+0000 7f4b48b0d640 1 -- start start 2024-04-25T11:23:37.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.106+0000 7f4b48b0d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b441029c0 0x7f4b4406a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:37.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.106+0000 7f4b48b0d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b44103c00 0x7f4b4419ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:37.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.106+0000 7f4b48b0d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b44108900 0x7f4b441a1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:37.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.106+0000 7f4b48b0d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4410e350 con 0x7f4b441029c0 2024-04-25T11:23:37.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.106+0000 7f4b48b0d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4410e1d0 con 0x7f4b44108900 2024-04-25T11:23:37.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.106+0000 7f4b48b0d640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4410e4d0 con 0x7f4b44103c00 2024-04-25T11:23:37.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.107+0000 7f4b4259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b441029c0 0x7f4b4406a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:37.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.107+0000 7f4b4259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b441029c0 0x7f4b4406a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37690/0 (socket says 172.21.15.69:37690) 2024-04-25T11:23:37.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.107+0000 7f4b4259c640 1 -- 172.21.15.69:0/3466202761 learned_addr learned my addr 172.21.15.69:0/3466202761 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:37.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.107+0000 7f4b3bfff640 1 --2- 172.21.15.69:0/3466202761 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b44103c00 0x7f4b4419ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:37.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.107+0000 7f4b42d9d640 1 --2- 172.21.15.69:0/3466202761 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b44108900 0x7f4b441a1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:37.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.107+0000 7f4b4259c640 1 -- 172.21.15.69:0/3466202761 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b44103c00 msgr2=0x7f4b4419ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:37.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.107+0000 7f4b4259c640 1 --2- 172.21.15.69:0/3466202761 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b44103c00 0x7f4b4419ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:37.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.107+0000 7f4b4259c640 1 -- 172.21.15.69:0/3466202761 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b44108900 msgr2=0x7f4b441a1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:37.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.107+0000 7f4b4259c640 1 --2- 172.21.15.69:0/3466202761 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b44108900 0x7f4b441a1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:37.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.107+0000 7f4b4259c640 1 -- 172.21.15.69:0/3466202761 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b441a1c60 con 0x7f4b441029c0 2024-04-25T11:23:37.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.107+0000 7f4b4259c640 1 --2- 172.21.15.69:0/3466202761 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b441029c0 0x7f4b4406a960 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4b34005760 tx=0x7f4b34004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:37.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.107+0000 7f4b3b7fe640 1 -- 172.21.15.69:0/3466202761 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3403f030 con 0x7f4b441029c0 2024-04-25T11:23:37.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.107+0000 7f4b3b7fe640 1 -- 172.21.15.69:0/3466202761 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4b34003a80 con 0x7f4b441029c0 2024-04-25T11:23:37.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.107+0000 7f4b48b0d640 1 -- 172.21.15.69:0/3466202761 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b441a1ef0 con 0x7f4b441029c0 2024-04-25T11:23:37.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.107+0000 7f4b3b7fe640 1 -- 172.21.15.69:0/3466202761 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b34035070 con 0x7f4b441029c0 2024-04-25T11:23:37.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.107+0000 7f4b48b0d640 1 -- 172.21.15.69:0/3466202761 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b441a22f0 con 0x7f4b441029c0 2024-04-25T11:23:37.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.109+0000 7f4b48b0d640 1 -- 172.21.15.69:0/3466202761 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b44107710 con 0x7f4b441029c0 2024-04-25T11:23:37.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.110+0000 7f4b3b7fe640 1 -- 172.21.15.69:0/3466202761 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f4b34042030 con 0x7f4b441029c0 2024-04-25T11:23:37.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.110+0000 7f4b3b7fe640 1 --2- 172.21.15.69:0/3466202761 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f4b300afaf0 0x7f4b300b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:37.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.110+0000 7f4b3bfff640 1 --2- 172.21.15.69:0/3466202761 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f4b300afaf0 0x7f4b300b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:37.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.111+0000 7f4b3b7fe640 1 -- 172.21.15.69:0/3466202761 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f4b340fedf0 con 0x7f4b441029c0 2024-04-25T11:23:37.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.111+0000 7f4b3bfff640 1 --2- 172.21.15.69:0/3466202761 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f4b300afaf0 0x7f4b300b2000 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4b4406a300 tx=0x7f4b2c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:37.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.115+0000 7f4b3b7fe640 1 -- 172.21.15.69:0/3466202761 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4b340c7820 con 0x7f4b441029c0 2024-04-25T11:23:37.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:36 smithi069 ceph-mon[29188]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:37.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:36 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:37.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:36 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:37.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:36 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:37.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:36 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:37.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:36 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:37.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:36 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:37.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:36 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:23:37.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:36 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:23:37.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:36 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:37.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:36 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:37.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:36 smithi152 ceph-mon[32701]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:37.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:36 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:37.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:36 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:37.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:36 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:37.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:36 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:37.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:36 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:37.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:36 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:37.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:36 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:23:37.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:36 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:23:37.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:36 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:37.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:36 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:37.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.346+0000 7f4b48b0d640 1 -- 172.21.15.69:0/3466202761 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b4402db50 con 0x7f4b441029c0 2024-04-25T11:23:37.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.347+0000 7f4b3b7fe640 1 -- 172.21.15.69:0/3466202761 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10491 (secure 0 0 0) 0x7f4b340cbb20 con 0x7f4b441029c0 2024-04-25T11:23:37.341 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:23:37.341 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3399424094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3399424094},{"type":"v1","addr":"172.21.15.69:6835","nonce":3399424094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1463217771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1463217771},{"type":"v1","addr":"172.21.15.69:6837","nonce":1463217771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/966316001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":966316001},{"type":"v1","addr":"172.21.15.69:6839","nonce":966316001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/2060225181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":2060225181},{"type":"v1","addr":"172.21.15.69:6841","nonce":2060225181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/4197482473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":4197482473},{"type":"v1","addr":"172.21.15.152:6833","nonce":4197482473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/4267885323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4267885323},{"type":"v1","addr":"172.21.15.146:6833","nonce":4267885323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24439,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/3188843362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3188843362},{"type":"v1","addr":"172.21.15.152:6835","nonce":3188843362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/563274982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":563274982},{"type":"v1","addr":"172.21.15.152:6837","nonce":563274982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1876533699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1876533699},{"type":"v1","addr":"172.21.15.146:6835","nonce":1876533699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24475,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6839/3008615657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":3008615657},{"type":"v1","addr":"172.21.15.152:6839","nonce":3008615657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/2730317134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2730317134},{"type":"v1","addr":"172.21.15.146:6837","nonce":2730317134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24515,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6839/2013283448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":2013283448},{"type":"v1","addr":"172.21.15.146:6839","nonce":2013283448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-25T11:23:37.342 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 13 2024-04-25T11:23:37.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.349+0000 7f4b48b0d640 1 -- 172.21.15.69:0/3466202761 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f4b300afaf0 msgr2=0x7f4b300b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:37.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.349+0000 7f4b48b0d640 1 --2- 172.21.15.69:0/3466202761 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f4b300afaf0 0x7f4b300b2000 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4b4406a300 tx=0x7f4b2c00b390 comp rx=0 tx=0).stop 2024-04-25T11:23:37.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.349+0000 7f4b48b0d640 1 -- 172.21.15.69:0/3466202761 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b441029c0 msgr2=0x7f4b4406a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:37.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.349+0000 7f4b48b0d640 1 --2- 172.21.15.69:0/3466202761 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b441029c0 0x7f4b4406a960 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4b34005760 tx=0x7f4b34004270 comp rx=0 tx=0).stop 2024-04-25T11:23:37.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.349+0000 7f4b48b0d640 1 -- 172.21.15.69:0/3466202761 shutdown_connections 2024-04-25T11:23:37.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.349+0000 7f4b48b0d640 1 --2- 172.21.15.69:0/3466202761 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f4b300afaf0 0x7f4b300b2000 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:37.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.349+0000 7f4b48b0d640 1 --2- 172.21.15.69:0/3466202761 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b44108900 0x7f4b441a1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:37.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.349+0000 7f4b48b0d640 1 --2- 172.21.15.69:0/3466202761 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b44103c00 0x7f4b4419ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:37.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.349+0000 7f4b48b0d640 1 --2- 172.21.15.69:0/3466202761 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b441029c0 0x7f4b4406a960 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:37.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.349+0000 7f4b48b0d640 1 -- 172.21.15.69:0/3466202761 >> 172.21.15.69:0/3466202761 conn(0x7f4b440fdbd0 msgr2=0x7f4b441048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:37.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.349+0000 7f4b48b0d640 1 -- 172.21.15.69:0/3466202761 shutdown_connections 2024-04-25T11:23:37.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.349+0000 7f4b48b0d640 1 -- 172.21.15.69:0/3466202761 wait complete. 2024-04-25T11:23:37.356 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:23:37.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.475+0000 7f7436a9f640 1 -- 172.21.15.69:0/2188072505 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7430106f30 msgr2=0x7f7430109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:37.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.475+0000 7f7436a9f640 1 --2- 172.21.15.69:0/2188072505 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7430106f30 0x7f7430109350 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f741c009900 tx=0x7f741c02a410 comp rx=0 tx=0).stop 2024-04-25T11:23:37.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.475+0000 7f7436a9f640 1 -- 172.21.15.69:0/2188072505 shutdown_connections 2024-04-25T11:23:37.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.475+0000 7f7436a9f640 1 --2- 172.21.15.69:0/2188072505 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7430109890 0x7f743010bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:37.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.475+0000 7f7436a9f640 1 --2- 172.21.15.69:0/2188072505 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7430106f30 0x7f7430109350 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:37.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.475+0000 7f7436a9f640 1 --2- 172.21.15.69:0/2188072505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74301045d0 0x7f74301069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:37.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.475+0000 7f7436a9f640 1 -- 172.21.15.69:0/2188072505 >> 172.21.15.69:0/2188072505 conn(0x7f74300fdbd0 msgr2=0x7f7430100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:37.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.476+0000 7f7436a9f640 1 -- 172.21.15.69:0/2188072505 shutdown_connections 2024-04-25T11:23:37.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.476+0000 7f7436a9f640 1 -- 172.21.15.69:0/2188072505 wait complete. 2024-04-25T11:23:37.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.476+0000 7f7436a9f640 1 Processor -- start 2024-04-25T11:23:37.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.476+0000 7f7436a9f640 1 -- start start 2024-04-25T11:23:37.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.476+0000 7f7436a9f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74301045d0 0x7f743019a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:37.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.476+0000 7f7436a9f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7430106f30 0x7f743019a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:37.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.476+0000 7f7436a9f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7430109890 0x7f74301a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:37.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.476+0000 7f7436a9f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f743010f530 con 0x7f74301045d0 2024-04-25T11:23:37.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.476+0000 7f7436a9f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f743010f3b0 con 0x7f7430109890 2024-04-25T11:23:37.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.476+0000 7f7436a9f640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f743010f6b0 con 0x7f7430106f30 2024-04-25T11:23:37.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.476+0000 7f743483b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74301045d0 0x7f743019a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:37.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.476+0000 7f743483b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74301045d0 0x7f743019a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37712/0 (socket says 172.21.15.69:37712) 2024-04-25T11:23:37.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.476+0000 7f743483b640 1 -- 172.21.15.69:0/3324613811 learned_addr learned my addr 172.21.15.69:0/3324613811 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:37.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.477+0000 7f743503c640 1 --2- 172.21.15.69:0/3324613811 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7430109890 0x7f74301a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:37.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.477+0000 7f742ffff640 1 --2- 172.21.15.69:0/3324613811 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7430106f30 0x7f743019a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:37.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.477+0000 7f743483b640 1 -- 172.21.15.69:0/3324613811 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7430106f30 msgr2=0x7f743019a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:37.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.477+0000 7f743483b640 1 --2- 172.21.15.69:0/3324613811 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7430106f30 0x7f743019a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:37.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.477+0000 7f743483b640 1 -- 172.21.15.69:0/3324613811 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7430109890 msgr2=0x7f74301a1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:37.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.477+0000 7f743483b640 1 --2- 172.21.15.69:0/3324613811 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7430109890 0x7f74301a1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:37.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.477+0000 7f743483b640 1 -- 172.21.15.69:0/3324613811 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74301a1c20 con 0x7f74301045d0 2024-04-25T11:23:37.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.477+0000 7f743483b640 1 --2- 172.21.15.69:0/3324613811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74301045d0 0x7f743019a190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f742400c970 tx=0x7f742400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:37.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.477+0000 7f742dffb640 1 -- 172.21.15.69:0/3324613811 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7424014070 con 0x7f74301045d0 2024-04-25T11:23:37.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.477+0000 7f742dffb640 1 -- 172.21.15.69:0/3324613811 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7424004440 con 0x7f74301045d0 2024-04-25T11:23:37.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.477+0000 7f742dffb640 1 -- 172.21.15.69:0/3324613811 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7424003d40 con 0x7f74301045d0 2024-04-25T11:23:37.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.477+0000 7f7436a9f640 1 -- 172.21.15.69:0/3324613811 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74301a1f10 con 0x7f74301045d0 2024-04-25T11:23:37.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.477+0000 7f7436a9f640 1 -- 172.21.15.69:0/3324613811 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74301a2450 con 0x7f74301045d0 2024-04-25T11:23:37.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.478+0000 7f7436a9f640 1 -- 172.21.15.69:0/3324613811 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74300ff6f0 con 0x7f74301045d0 2024-04-25T11:23:37.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.479+0000 7f742dffb640 1 -- 172.21.15.69:0/3324613811 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f7424017070 con 0x7f74301045d0 2024-04-25T11:23:37.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.480+0000 7f742dffb640 1 --2- 172.21.15.69:0/3324613811 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f74200afb70 0x7f74200b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:37.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.480+0000 7f742ffff640 1 --2- 172.21.15.69:0/3324613811 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f74200afb70 0x7f74200b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:37.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.480+0000 7f742dffb640 1 -- 172.21.15.69:0/3324613811 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f74240de5a0 con 0x7f74301045d0 2024-04-25T11:23:37.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.481+0000 7f742ffff640 1 --2- 172.21.15.69:0/3324613811 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f74200afb70 0x7f74200b2080 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f741c006010 tx=0x7f741c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:37.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.485+0000 7f742dffb640 1 -- 172.21.15.69:0/3324613811 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f742401e020 con 0x7f74301045d0 2024-04-25T11:23:37.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.664+0000 7f7436a9f640 1 -- 172.21.15.69:0/3324613811 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f743002dd70 con 0x7f74301045d0 2024-04-25T11:23:37.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.665+0000 7f742dffb640 1 -- 172.21.15.69:0/3324613811 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+18458 (secure 0 0 0) 0x7f74240a7620 con 0x7f74301045d0 2024-04-25T11:23:37.659 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:23:37.659 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":71,"fsid":"f7ed73dc-02f4-11ef-bc93-c7b262605968","created":"2024-04-25T11:15:37.965281+0000","modified":"2024-04-25T11:22:51.698858+0000","last_up_change":"2024-04-25T11:22:50.691869+0000","last_in_change":"2024-04-25T11:22:30.271959+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:18:57.781263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a6a0d179-4998-4754-bf82-2ea3dac54e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6803","nonce":3756025235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6805","nonce":3756025235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6809","nonce":3756025235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6807","nonce":3756025235}]},"public_addr":"172.21.15.69:6803/3756025235","cluster_addr":"172.21.15.69:6805/3756025235","heartbeat_back_addr":"172.21.15.69:6809/3756025235","heartbeat_front_addr":"172.21.15.69:6807/3756025235","state":["exists","up"]},{"osd":1,"uuid":"c0c66930-4233-493f-b2bb-99c473994b59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6801","nonce":3166305213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6803","nonce":3166305213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6807","nonce":3166305213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6805","nonce":3166305213}]},"public_addr":"172.21.15.146:6801/3166305213","cluster_addr":"172.21.15.146:6803/3166305213","heartbeat_back_addr":"172.21.15.146:6807/3166305213","heartbeat_front_addr":"172.21.15.146:6805/3166305213","state":["exists","up"]},{"osd":2,"uuid":"d4ef735b-9786-4cb9-83ef-7c3f0691a729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6801","nonce":1474319175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6803","nonce":1474319175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6807","nonce":1474319175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6805","nonce":1474319175}]},"public_addr":"172.21.15.152:6801/1474319175","cluster_addr":"172.21.15.152:6803/1474319175","heartbeat_back_addr":"172.21.15.152:6807/1474319175","heartbeat_front_addr":"172.21.15.152:6805/1474319175","state":["exists","up"]},{"osd":3,"uuid":"0ab85fb8-5bbb-4ffc-bddf-cae7644a64ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6811","nonce":1809071296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6813","nonce":1809071296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6817","nonce":1809071296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6815","nonce":1809071296}]},"public_addr":"172.21.15.69:6811/1809071296","cluster_addr":"172.21.15.69:6813/1809071296","heartbeat_back_addr":"172.21.15.69:6817/1809071296","heartbeat_front_addr":"172.21.15.69:6815/1809071296","state":["exists","up"]},{"osd":4,"uuid":"273e0592-bbbc-4188-8eeb-46a3879cf632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6809","nonce":1898520972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6811","nonce":1898520972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6815","nonce":1898520972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6813","nonce":1898520972}]},"public_addr":"172.21.15.146:6809/1898520972","cluster_addr":"172.21.15.146:6811/1898520972","heartbeat_back_addr":"172.21.15.146:6815/1898520972","heartbeat_front_addr":"172.21.15.146:6813/1898520972","state":["exists","up"]},{"osd":5,"uuid":"bbeae945-9fa2-48f9-a1d6-8bf18dcaef19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6809","nonce":1996555108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6811","nonce":1996555108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6815","nonce":1996555108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6813","nonce":1996555108}]},"public_addr":"172.21.15.152:6809/1996555108","cluster_addr":"172.21.15.152:6811/1996555108","heartbeat_back_addr":"172.21.15.152:6815/1996555108","heartbeat_front_addr":"172.21.15.152:6813/1996555108","state":["exists","up"]},{"osd":6,"uuid":"dbb61253-30bc-418e-bd75-118c80bbf931","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6819","nonce":491215762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6821","nonce":491215762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6825","nonce":491215762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6823","nonce":491215762}]},"public_addr":"172.21.15.69:6819/491215762","cluster_addr":"172.21.15.69:6821/491215762","heartbeat_back_addr":"172.21.15.69:6825/491215762","heartbeat_front_addr":"172.21.15.69:6823/491215762","state":["exists","up"]},{"osd":7,"uuid":"75f23854-e283-4e3e-94cf-7b4079d83876","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6817","nonce":2330294690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6819","nonce":2330294690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6823","nonce":2330294690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6821","nonce":2330294690}]},"public_addr":"172.21.15.146:6817/2330294690","cluster_addr":"172.21.15.146:6819/2330294690","heartbeat_back_addr":"172.21.15.146:6823/2330294690","heartbeat_front_addr":"172.21.15.146:6821/2330294690","state":["exists","up"]},{"osd":8,"uuid":"1aa1060c-0c14-4abd-be56-5ebfeb78759c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6817","nonce":950669381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6819","nonce":950669381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6823","nonce":950669381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6821","nonce":950669381}]},"public_addr":"172.21.15.152:6817/950669381","cluster_addr":"172.21.15.152:6819/950669381","heartbeat_back_addr":"172.21.15.152:6823/950669381","heartbeat_front_addr":"172.21.15.152:6821/950669381","state":["exists","up"]},{"osd":9,"uuid":"3f4bf800-4090-4138-9bb3-d001f13f450c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6827","nonce":2414186758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6829","nonce":2414186758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6833","nonce":2414186758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6831","nonce":2414186758}]},"public_addr":"172.21.15.69:6827/2414186758","cluster_addr":"172.21.15.69:6829/2414186758","heartbeat_back_addr":"172.21.15.69:6833/2414186758","heartbeat_front_addr":"172.21.15.69:6831/2414186758","state":["exists","up"]},{"osd":10,"uuid":"19e967e5-51ce-4506-80c9-0a95441baf99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6825","nonce":707256400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6827","nonce":707256400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6831","nonce":707256400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6829","nonce":707256400}]},"public_addr":"172.21.15.146:6825/707256400","cluster_addr":"172.21.15.146:6827/707256400","heartbeat_back_addr":"172.21.15.146:6831/707256400","heartbeat_front_addr":"172.21.15.146:6829/707256400","state":["exists","up"]},{"osd":11,"uuid":"7788c2de-6930-4808-be0f-f1fd1ae0cb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6825","nonce":2934336833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6827","nonce":2934336833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6831","nonce":2934336833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6829","nonce":2934336833}]},"public_addr":"172.21.15.152:6825/2934336833","cluster_addr":"172.21.15.152:6827/2934336833","heartbeat_back_addr":"172.21.15.152:6831/2934336833","heartbeat_front_addr":"172.21.15.152:6829/2934336833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:07.497448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:31.227494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:55.027978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:20.713437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:46.214704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:20:11.199983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:20:37.177070+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:02.951240+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:28.963680+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:56.095807+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:22:22.221744+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:22:48.929334+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/1372727546":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6801/1097124842":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6800/1508198215":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3103465990":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3540132960":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3899912190":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6800/1097124842":"2024-04-26T11:16:31.688159+0000","172.21.15.69:0/1199907202":"2024-04-26T11:16:04.860792+0000","172.21.15.69:6801/1508198215":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/2879102529":"2024-04-26T11:16:31.688159+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:23:37.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.667+0000 7f7436a9f640 1 -- 172.21.15.69:0/3324613811 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f74200afb70 msgr2=0x7f74200b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:37.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.667+0000 7f7436a9f640 1 --2- 172.21.15.69:0/3324613811 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f74200afb70 0x7f74200b2080 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f741c006010 tx=0x7f741c000bc0 comp rx=0 tx=0).stop 2024-04-25T11:23:37.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.667+0000 7f7436a9f640 1 -- 172.21.15.69:0/3324613811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74301045d0 msgr2=0x7f743019a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:37.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.667+0000 7f7436a9f640 1 --2- 172.21.15.69:0/3324613811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74301045d0 0x7f743019a190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f742400c970 tx=0x7f742400ce30 comp rx=0 tx=0).stop 2024-04-25T11:23:37.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.667+0000 7f7436a9f640 1 -- 172.21.15.69:0/3324613811 shutdown_connections 2024-04-25T11:23:37.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.667+0000 7f7436a9f640 1 --2- 172.21.15.69:0/3324613811 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f74200afb70 0x7f74200b2080 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:37.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.667+0000 7f7436a9f640 1 --2- 172.21.15.69:0/3324613811 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7430109890 0x7f74301a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:37.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.667+0000 7f7436a9f640 1 --2- 172.21.15.69:0/3324613811 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7430106f30 0x7f743019a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:37.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.667+0000 7f7436a9f640 1 --2- 172.21.15.69:0/3324613811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74301045d0 0x7f743019a190 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:37.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.667+0000 7f7436a9f640 1 -- 172.21.15.69:0/3324613811 >> 172.21.15.69:0/3324613811 conn(0x7f74300fdbd0 msgr2=0x7f74301052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:37.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.667+0000 7f7436a9f640 1 -- 172.21.15.69:0/3324613811 shutdown_connections 2024-04-25T11:23:37.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.667+0000 7f7436a9f640 1 -- 172.21.15.69:0/3324613811 wait complete. 2024-04-25T11:23:37.674 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:18:57.781263+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:23:37.674 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:23:37.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.796+0000 7f24f5ed8640 1 -- 172.21.15.69:0/2411707721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24f0103af0 msgr2=0x7f24f01047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:37.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.796+0000 7f24f5ed8640 1 --2- 172.21.15.69:0/2411707721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24f0103af0 0x7f24f01047f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f24e0009900 tx=0x7f24e002a410 comp rx=0 tx=0).stop 2024-04-25T11:23:37.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.796+0000 7f24f5ed8640 1 -- 172.21.15.69:0/2411707721 shutdown_connections 2024-04-25T11:23:37.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.796+0000 7f24f5ed8640 1 --2- 172.21.15.69:0/2411707721 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24f0109a30 0x7f24f010be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:37.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.796+0000 7f24f5ed8640 1 --2- 172.21.15.69:0/2411707721 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24f0104d30 0x7f24f0105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:37.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.796+0000 7f24f5ed8640 1 --2- 172.21.15.69:0/2411707721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24f0103af0 0x7f24f01047f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:37.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.796+0000 7f24f5ed8640 1 -- 172.21.15.69:0/2411707721 >> 172.21.15.69:0/2411707721 conn(0x7f24f00fece0 msgr2=0x7f24f0101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:37.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.796+0000 7f24f5ed8640 1 -- 172.21.15.69:0/2411707721 shutdown_connections 2024-04-25T11:23:37.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.796+0000 7f24f5ed8640 1 -- 172.21.15.69:0/2411707721 wait complete. 2024-04-25T11:23:37.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.796+0000 7f24f5ed8640 1 Processor -- start 2024-04-25T11:23:37.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.796+0000 7f24f5ed8640 1 -- start start 2024-04-25T11:23:37.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.797+0000 7f24f5ed8640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24f0103af0 0x7f24f019a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:37.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.797+0000 7f24f5ed8640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24f0104d30 0x7f24f019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:37.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.797+0000 7f24f5ed8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24f0109a30 0x7f24f01a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:37.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.797+0000 7f24f5ed8640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f24f010f4a0 con 0x7f24f0104d30 2024-04-25T11:23:37.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.797+0000 7f24f5ed8640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f24f010f320 con 0x7f24f0109a30 2024-04-25T11:23:37.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.797+0000 7f24f5ed8640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f24f010f620 con 0x7f24f0103af0 2024-04-25T11:23:37.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.797+0000 7f24eeffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24f0104d30 0x7f24f019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:37.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.797+0000 7f24eeffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24f0104d30 0x7f24f019a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37732/0 (socket says 172.21.15.69:37732) 2024-04-25T11:23:37.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.797+0000 7f24eeffd640 1 -- 172.21.15.69:0/3792064916 learned_addr learned my addr 172.21.15.69:0/3792064916 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:37.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.797+0000 7f24ef7fe640 1 --2- 172.21.15.69:0/3792064916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24f0103af0 0x7f24f019a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:37.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.797+0000 7f24effff640 1 --2- 172.21.15.69:0/3792064916 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24f0109a30 0x7f24f01a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:37.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.797+0000 7f24eeffd640 1 -- 172.21.15.69:0/3792064916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24f0103af0 msgr2=0x7f24f019a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:37.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.797+0000 7f24eeffd640 1 --2- 172.21.15.69:0/3792064916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24f0103af0 0x7f24f019a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:37.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.797+0000 7f24eeffd640 1 -- 172.21.15.69:0/3792064916 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24f0109a30 msgr2=0x7f24f01a1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:37.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.797+0000 7f24eeffd640 1 --2- 172.21.15.69:0/3792064916 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24f0109a30 0x7f24f01a1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:37.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.797+0000 7f24eeffd640 1 -- 172.21.15.69:0/3792064916 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24f01a1cd0 con 0x7f24f0104d30 2024-04-25T11:23:37.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.797+0000 7f24ef7fe640 1 --2- 172.21.15.69:0/3792064916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24f0103af0 0x7f24f019a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:23:37.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.797+0000 7f24eeffd640 1 --2- 172.21.15.69:0/3792064916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24f0104d30 0x7f24f019a740 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f24e400e9f0 tx=0x7f24e400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:37.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.798+0000 7f24ecff9640 1 -- 172.21.15.69:0/3792064916 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e400cd80 con 0x7f24f0104d30 2024-04-25T11:23:37.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.798+0000 7f24ecff9640 1 -- 172.21.15.69:0/3792064916 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24e40054b0 con 0x7f24f0104d30 2024-04-25T11:23:37.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.798+0000 7f24f5ed8640 1 -- 172.21.15.69:0/3792064916 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24f0075f40 con 0x7f24f0104d30 2024-04-25T11:23:37.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.798+0000 7f24ecff9640 1 -- 172.21.15.69:0/3792064916 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e4010430 con 0x7f24f0104d30 2024-04-25T11:23:37.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.798+0000 7f24f5ed8640 1 -- 172.21.15.69:0/3792064916 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24f0076440 con 0x7f24f0104d30 2024-04-25T11:23:37.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.799+0000 7f24f5ed8640 1 -- 172.21.15.69:0/3792064916 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24f0108840 con 0x7f24f0104d30 2024-04-25T11:23:37.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.801+0000 7f24ecff9640 1 -- 172.21.15.69:0/3792064916 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f24e40046c0 con 0x7f24f0104d30 2024-04-25T11:23:37.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.801+0000 7f24ecff9640 1 --2- 172.21.15.69:0/3792064916 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f24d80afaf0 0x7f24d80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:37.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.802+0000 7f24ef7fe640 1 --2- 172.21.15.69:0/3792064916 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f24d80afaf0 0x7f24d80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:37.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.802+0000 7f24ecff9640 1 -- 172.21.15.69:0/3792064916 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f24e40decf0 con 0x7f24f0104d30 2024-04-25T11:23:37.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.802+0000 7f24ef7fe640 1 --2- 172.21.15.69:0/3792064916 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f24d80afaf0 0x7f24d80b2000 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f24e0005760 tx=0x7f24e0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:37.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.805+0000 7f24ecff9640 1 -- 172.21.15.69:0/3792064916 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f24e4053ff0 con 0x7f24f0104d30 2024-04-25T11:23:37.894 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:37 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:23:37.894 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:37 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/3466202761' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:37.894 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:37 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/3324613811' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:23:37.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:37 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:23:37.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:37 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/3466202761' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:37.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:37 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/3324613811' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:23:37.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.985+0000 7f24f5ed8640 1 -- 172.21.15.69:0/3792064916 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f24f0077880 con 0x7f24f0104d30 2024-04-25T11:23:37.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.985+0000 7f24ecff9640 1 -- 172.21.15.69:0/3792064916 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7f24e40a7d70 con 0x7f24f0104d30 2024-04-25T11:23:37.980 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-04-25T11:23:37.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.987+0000 7f24f5ed8640 1 -- 172.21.15.69:0/3792064916 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f24d80afaf0 msgr2=0x7f24d80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:37.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.987+0000 7f24f5ed8640 1 --2- 172.21.15.69:0/3792064916 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f24d80afaf0 0x7f24d80b2000 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f24e0005760 tx=0x7f24e0000bc0 comp rx=0 tx=0).stop 2024-04-25T11:23:37.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.987+0000 7f24f5ed8640 1 -- 172.21.15.69:0/3792064916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24f0104d30 msgr2=0x7f24f019a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:37.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.987+0000 7f24f5ed8640 1 --2- 172.21.15.69:0/3792064916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24f0104d30 0x7f24f019a740 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f24e400e9f0 tx=0x7f24e400edf0 comp rx=0 tx=0).stop 2024-04-25T11:23:37.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.987+0000 7f24f5ed8640 1 -- 172.21.15.69:0/3792064916 shutdown_connections 2024-04-25T11:23:37.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.987+0000 7f24f5ed8640 1 --2- 172.21.15.69:0/3792064916 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f24d80afaf0 0x7f24d80b2000 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:37.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.987+0000 7f24f5ed8640 1 --2- 172.21.15.69:0/3792064916 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24f0109a30 0x7f24f01a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:37.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.987+0000 7f24f5ed8640 1 --2- 172.21.15.69:0/3792064916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f24f0104d30 0x7f24f019a740 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:37.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.987+0000 7f24f5ed8640 1 --2- 172.21.15.69:0/3792064916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24f0103af0 0x7f24f019a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:37.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.987+0000 7f24f5ed8640 1 -- 172.21.15.69:0/3792064916 >> 172.21.15.69:0/3792064916 conn(0x7f24f00fece0 msgr2=0x7f24f0105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:37.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.987+0000 7f24f5ed8640 1 -- 172.21.15.69:0/3792064916 shutdown_connections 2024-04-25T11:23:37.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:37.988+0000 7f24f5ed8640 1 -- 172.21.15.69:0/3792064916 wait complete. 2024-04-25T11:23:37.993 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T11:23:38.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:37 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:23:38.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:37 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/3466202761' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:38.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:37 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/3324613811' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:23:38.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.107+0000 7f20e1d34640 1 -- 172.21.15.69:0/2924110134 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20dc102160 msgr2=0x7f20dc102e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:38.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.107+0000 7f20e1d34640 1 --2- 172.21.15.69:0/2924110134 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20dc102160 0x7f20dc102e60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f20cc009900 tx=0x7f20cc02a410 comp rx=0 tx=0).stop 2024-04-25T11:23:38.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.107+0000 7f20e1d34640 1 -- 172.21.15.69:0/2924110134 shutdown_connections 2024-04-25T11:23:38.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.107+0000 7f20e1d34640 1 --2- 172.21.15.69:0/2924110134 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20dc1080a0 0x7f20dc10a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:38.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.107+0000 7f20e1d34640 1 --2- 172.21.15.69:0/2924110134 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20dc1033a0 0x7f20dc1037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:38.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.107+0000 7f20e1d34640 1 --2- 172.21.15.69:0/2924110134 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20dc102160 0x7f20dc102e60 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:38.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.107+0000 7f20e1d34640 1 -- 172.21.15.69:0/2924110134 >> 172.21.15.69:0/2924110134 conn(0x7f20dc0fd330 msgr2=0x7f20dc0ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:38.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.107+0000 7f20e1d34640 1 -- 172.21.15.69:0/2924110134 shutdown_connections 2024-04-25T11:23:38.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.107+0000 7f20e1d34640 1 -- 172.21.15.69:0/2924110134 wait complete. 2024-04-25T11:23:38.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.108+0000 7f20e1d34640 1 Processor -- start 2024-04-25T11:23:38.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.108+0000 7f20e1d34640 1 -- start start 2024-04-25T11:23:38.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.108+0000 7f20e1d34640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20dc102160 0x7f20dc10b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:38.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.108+0000 7f20e1d34640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20dc1033a0 0x7f20dc19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:38.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.108+0000 7f20e1d34640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20dc1080a0 0x7f20dc1a1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:38.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.108+0000 7f20e1d34640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f20dc10e3a0 con 0x7f20dc102160 2024-04-25T11:23:38.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.108+0000 7f20e1d34640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f20dc10e220 con 0x7f20dc1080a0 2024-04-25T11:23:38.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.108+0000 7f20e1d34640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f20dc10e520 con 0x7f20dc1033a0 2024-04-25T11:23:38.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.108+0000 7f20db7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20dc102160 0x7f20dc10b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:38.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.108+0000 7f20db7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20dc102160 0x7f20dc10b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37740/0 (socket says 172.21.15.69:37740) 2024-04-25T11:23:38.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.108+0000 7f20db7fe640 1 -- 172.21.15.69:0/813420774 learned_addr learned my addr 172.21.15.69:0/813420774 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:38.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.108+0000 7f20d3fff640 1 --2- 172.21.15.69:0/813420774 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20dc1033a0 0x7f20dc19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:38.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.108+0000 7f20dbfff640 1 --2- 172.21.15.69:0/813420774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20dc1080a0 0x7f20dc1a1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:38.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.108+0000 7f20db7fe640 1 -- 172.21.15.69:0/813420774 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20dc1033a0 msgr2=0x7f20dc19eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:38.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.108+0000 7f20db7fe640 1 --2- 172.21.15.69:0/813420774 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20dc1033a0 0x7f20dc19eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:38.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.108+0000 7f20db7fe640 1 -- 172.21.15.69:0/813420774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20dc1080a0 msgr2=0x7f20dc1a1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:38.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.108+0000 7f20db7fe640 1 --2- 172.21.15.69:0/813420774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20dc1080a0 0x7f20dc1a1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:38.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.108+0000 7f20db7fe640 1 -- 172.21.15.69:0/813420774 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20dc1a1cb0 con 0x7f20dc102160 2024-04-25T11:23:38.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.108+0000 7f20db7fe640 1 --2- 172.21.15.69:0/813420774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20dc102160 0x7f20dc10b680 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f20cc005760 tx=0x7f20cc004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:38.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.109+0000 7f20d97fa640 1 -- 172.21.15.69:0/813420774 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20cc03f030 con 0x7f20dc102160 2024-04-25T11:23:38.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.109+0000 7f20d97fa640 1 -- 172.21.15.69:0/813420774 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20cc004440 con 0x7f20dc102160 2024-04-25T11:23:38.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.109+0000 7f20e1d34640 1 -- 172.21.15.69:0/813420774 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20dc1a1f40 con 0x7f20dc102160 2024-04-25T11:23:38.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.109+0000 7f20d97fa640 1 -- 172.21.15.69:0/813420774 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20cc035070 con 0x7f20dc102160 2024-04-25T11:23:38.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.109+0000 7f20e1d34640 1 -- 172.21.15.69:0/813420774 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20dc1a2340 con 0x7f20dc102160 2024-04-25T11:23:38.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.110+0000 7f20e1d34640 1 -- 172.21.15.69:0/813420774 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20dc106eb0 con 0x7f20dc102160 2024-04-25T11:23:38.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.112+0000 7f20d97fa640 1 -- 172.21.15.69:0/813420774 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f20cc042030 con 0x7f20dc102160 2024-04-25T11:23:38.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.112+0000 7f20d97fa640 1 --2- 172.21.15.69:0/813420774 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f20c80afaf0 0x7f20c80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:38.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.113+0000 7f20d3fff640 1 --2- 172.21.15.69:0/813420774 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f20c80afaf0 0x7f20c80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:38.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.113+0000 7f20d97fa640 1 -- 172.21.15.69:0/813420774 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f20cc0fee90 con 0x7f20dc102160 2024-04-25T11:23:38.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.113+0000 7f20d3fff640 1 --2- 172.21.15.69:0/813420774 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f20c80afaf0 0x7f20c80b2000 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f20dc10b020 tx=0x7f20c400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:38.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.117+0000 7f20d97fa640 1 -- 172.21.15.69:0/813420774 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f20cc0c78c0 con 0x7f20dc102160 2024-04-25T11:23:38.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.324+0000 7f20e1d34640 1 -- 172.21.15.69:0/813420774 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f20dc0637d0 con 0x7f20dc102160 2024-04-25T11:23:38.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.324+0000 7f20d97fa640 1 -- 172.21.15.69:0/813420774 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v71) ==== 84+0+65 (secure 0 0 0) 0x7f20cc0cbbc0 con 0x7f20dc102160 2024-04-25T11:23:38.319 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:23:38.319 INFO:teuthology.orchestra.run.smithi069.stdout:[ 2024-04-25T11:23:38.319 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:23:38.319 INFO:teuthology.orchestra.run.smithi069.stdout: "poolnum": 1, 2024-04-25T11:23:38.319 INFO:teuthology.orchestra.run.smithi069.stdout: "poolname": ".mgr" 2024-04-25T11:23:38.319 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:23:38.319 INFO:teuthology.orchestra.run.smithi069.stdout:] 2024-04-25T11:23:38.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.327+0000 7f20e1d34640 1 -- 172.21.15.69:0/813420774 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f20c80afaf0 msgr2=0x7f20c80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:38.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.327+0000 7f20e1d34640 1 --2- 172.21.15.69:0/813420774 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f20c80afaf0 0x7f20c80b2000 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f20dc10b020 tx=0x7f20c400b390 comp rx=0 tx=0).stop 2024-04-25T11:23:38.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.327+0000 7f20e1d34640 1 -- 172.21.15.69:0/813420774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20dc102160 msgr2=0x7f20dc10b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:38.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.327+0000 7f20e1d34640 1 --2- 172.21.15.69:0/813420774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20dc102160 0x7f20dc10b680 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f20cc005760 tx=0x7f20cc004040 comp rx=0 tx=0).stop 2024-04-25T11:23:38.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.327+0000 7f20e1d34640 1 -- 172.21.15.69:0/813420774 shutdown_connections 2024-04-25T11:23:38.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.327+0000 7f20e1d34640 1 --2- 172.21.15.69:0/813420774 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f20c80afaf0 0x7f20c80b2000 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:38.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.327+0000 7f20e1d34640 1 --2- 172.21.15.69:0/813420774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20dc1080a0 0x7f20dc1a1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:38.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.327+0000 7f20e1d34640 1 --2- 172.21.15.69:0/813420774 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20dc1033a0 0x7f20dc19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:38.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.327+0000 7f20e1d34640 1 --2- 172.21.15.69:0/813420774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20dc102160 0x7f20dc10b680 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:38.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.327+0000 7f20e1d34640 1 -- 172.21.15.69:0/813420774 >> 172.21.15.69:0/813420774 conn(0x7f20dc0fd330 msgr2=0x7f20dc104080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:38.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.327+0000 7f20e1d34640 1 -- 172.21.15.69:0/813420774 shutdown_connections 2024-04-25T11:23:38.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.327+0000 7f20e1d34640 1 -- 172.21.15.69:0/813420774 wait complete. 2024-04-25T11:23:38.333 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T11:23:38.333 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T11:23:38.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.459+0000 7fc5fefb4640 1 -- 172.21.15.69:0/1404882722 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5f8104460 msgr2=0x7fc5f810aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:38.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.459+0000 7fc5fefb4640 1 --2- 172.21.15.69:0/1404882722 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5f8104460 0x7fc5f810aba0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0009ca0 tx=0x7fc5f002a8c0 comp rx=0 tx=0).stop 2024-04-25T11:23:38.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.459+0000 7fc5fefb4640 1 -- 172.21.15.69:0/1404882722 shutdown_connections 2024-04-25T11:23:38.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.459+0000 7fc5fefb4640 1 --2- 172.21.15.69:0/1404882722 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5f8104460 0x7fc5f810aba0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:38.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.459+0000 7fc5fefb4640 1 --2- 172.21.15.69:0/1404882722 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5f8103920 0x7fc5f8103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:38.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.459+0000 7fc5fefb4640 1 --2- 172.21.15.69:0/1404882722 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5f8102f40 0x7fc5f8103350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:38.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.459+0000 7fc5fefb4640 1 -- 172.21.15.69:0/1404882722 >> 172.21.15.69:0/1404882722 conn(0x7fc5f80fd340 msgr2=0x7fc5f80ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:38.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.459+0000 7fc5fefb4640 1 -- 172.21.15.69:0/1404882722 shutdown_connections 2024-04-25T11:23:38.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.459+0000 7fc5fefb4640 1 -- 172.21.15.69:0/1404882722 wait complete. 2024-04-25T11:23:38.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.460+0000 7fc5fefb4640 1 Processor -- start 2024-04-25T11:23:38.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.460+0000 7fc5fefb4640 1 -- start start 2024-04-25T11:23:38.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.460+0000 7fc5fefb4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5f8102f40 0x7fc5f819a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:38.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.460+0000 7fc5fefb4640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5f8103920 0x7fc5f819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:38.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.460+0000 7fc5fefb4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5f8104460 0x7fc5f81a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:38.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.460+0000 7fc5fefb4640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f810e430 con 0x7fc5f8102f40 2024-04-25T11:23:38.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.460+0000 7fc5fefb4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f810e2b0 con 0x7fc5f8104460 2024-04-25T11:23:38.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.460+0000 7fc5fefb4640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f810e5b0 con 0x7fc5f8103920 2024-04-25T11:23:38.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.460+0000 7fc5fcd50640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5f8102f40 0x7fc5f819a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:38.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.460+0000 7fc5fcd50640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5f8102f40 0x7fc5f819a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37760/0 (socket says 172.21.15.69:37760) 2024-04-25T11:23:38.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.460+0000 7fc5fcd50640 1 -- 172.21.15.69:0/2249360224 learned_addr learned my addr 172.21.15.69:0/2249360224 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:38.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.460+0000 7fc5fd551640 1 --2- 172.21.15.69:0/2249360224 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5f8104460 0x7fc5f81a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:38.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.460+0000 7fc5f7fff640 1 --2- 172.21.15.69:0/2249360224 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5f8103920 0x7fc5f819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:38.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.460+0000 7fc5fcd50640 1 -- 172.21.15.69:0/2249360224 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5f8103920 msgr2=0x7fc5f819a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:38.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.460+0000 7fc5fcd50640 1 --2- 172.21.15.69:0/2249360224 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5f8103920 0x7fc5f819a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:38.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.460+0000 7fc5fcd50640 1 -- 172.21.15.69:0/2249360224 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5f8104460 msgr2=0x7fc5f81a1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:38.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.460+0000 7fc5fcd50640 1 --2- 172.21.15.69:0/2249360224 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5f8104460 0x7fc5f81a1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:38.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.460+0000 7fc5fcd50640 1 -- 172.21.15.69:0/2249360224 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5f81a1c00 con 0x7fc5f8102f40 2024-04-25T11:23:38.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.461+0000 7fc5fcd50640 1 --2- 172.21.15.69:0/2249360224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5f8102f40 0x7fc5f819a0c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec00e9f0 tx=0x7fc5ec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:38.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.461+0000 7fc5f5ffb640 1 -- 172.21.15.69:0/2249360224 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec00ccc0 con 0x7fc5f8102f40 2024-04-25T11:23:38.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.461+0000 7fc5f5ffb640 1 -- 172.21.15.69:0/2249360224 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5ec0108c0 con 0x7fc5f8102f40 2024-04-25T11:23:38.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.461+0000 7fc5fefb4640 1 -- 172.21.15.69:0/2249360224 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5f81a1ef0 con 0x7fc5f8102f40 2024-04-25T11:23:38.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.461+0000 7fc5f5ffb640 1 -- 172.21.15.69:0/2249360224 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec018750 con 0x7fc5f8102f40 2024-04-25T11:23:38.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.461+0000 7fc5fefb4640 1 -- 172.21.15.69:0/2249360224 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5f81a2350 con 0x7fc5f8102f40 2024-04-25T11:23:38.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.463+0000 7fc5fefb4640 1 -- 172.21.15.69:0/2249360224 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5f81a2640 con 0x7fc5f8102f40 2024-04-25T11:23:38.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.464+0000 7fc5f5ffb640 1 -- 172.21.15.69:0/2249360224 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fc5ec010430 con 0x7fc5f8102f40 2024-04-25T11:23:38.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.464+0000 7fc5f5ffb640 1 --2- 172.21.15.69:0/2249360224 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc5e80afaf0 0x7fc5e80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:38.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.465+0000 7fc5f7fff640 1 --2- 172.21.15.69:0/2249360224 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc5e80afaf0 0x7fc5e80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:38.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.465+0000 7fc5f5ffb640 1 -- 172.21.15.69:0/2249360224 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fc5ec0dea40 con 0x7fc5f8102f40 2024-04-25T11:23:38.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.465+0000 7fc5f7fff640 1 --2- 172.21.15.69:0/2249360224 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc5e80afaf0 0x7fc5e80b2000 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc5f819b160 tx=0x7fc5e4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:38.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.470+0000 7fc5f5ffb640 1 -- 172.21.15.69:0/2249360224 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc5ec0a7470 con 0x7fc5f8102f40 2024-04-25T11:23:38.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:38.680+0000 7fc5fefb4640 1 -- 172.21.15.69:0/2249360224 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fc5f81a2810 con 0x7fc5f8102f40 2024-04-25T11:23:39.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:38 smithi146 ceph-mon[32821]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:39.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:38 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/3792064916' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:23:39.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:38 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/813420774' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T11:23:39.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:38 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:39.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:38 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:39.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:38 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:39.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:38 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:39.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:38 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:39.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:38 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:39.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:38 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:23:39.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:38 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:23:39.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:38 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:39.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:38 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/2249360224' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T11:23:39.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:38 smithi069 ceph-mon[29188]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:39.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:38 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/3792064916' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:23:39.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:38 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/813420774' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T11:23:39.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:38 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:39.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:38 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:39.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:38 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:39.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:38 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:39.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:38 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:39.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:38 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:39.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:38 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:23:39.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:38 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:23:39.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:38 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:39.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:38 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2249360224' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T11:23:39.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:38 smithi152 ceph-mon[32701]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:39.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:38 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/3792064916' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:23:39.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:38 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/813420774' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T11:23:39.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:38 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:39.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:38 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:39.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:38 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:39.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:38 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:39.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:38 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:39.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:38 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:39.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:38 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:23:39.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:38 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:23:39.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:38 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:23:39.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:38 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/2249360224' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T11:23:39.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.608+0000 7fc5f5ffb640 1 -- 172.21.15.69:0/2249360224 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v72) ==== 138+0+0 (secure 0 0 0) 0x7fc5ec0ab770 con 0x7fc5f8102f40 2024-04-25T11:23:39.603 INFO:teuthology.orchestra.run.smithi069.stderr:pool 'cephfs_metadata' created 2024-04-25T11:23:39.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.612+0000 7fc5fefb4640 1 -- 172.21.15.69:0/2249360224 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc5e80afaf0 msgr2=0x7fc5e80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:39.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.612+0000 7fc5fefb4640 1 --2- 172.21.15.69:0/2249360224 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc5e80afaf0 0x7fc5e80b2000 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc5f819b160 tx=0x7fc5e4005f90 comp rx=0 tx=0).stop 2024-04-25T11:23:39.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.612+0000 7fc5fefb4640 1 -- 172.21.15.69:0/2249360224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5f8102f40 msgr2=0x7fc5f819a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:39.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.612+0000 7fc5fefb4640 1 --2- 172.21.15.69:0/2249360224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5f8102f40 0x7fc5f819a0c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec00e9f0 tx=0x7fc5ec00eeb0 comp rx=0 tx=0).stop 2024-04-25T11:23:39.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.612+0000 7fc5fefb4640 1 -- 172.21.15.69:0/2249360224 shutdown_connections 2024-04-25T11:23:39.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.612+0000 7fc5fefb4640 1 --2- 172.21.15.69:0/2249360224 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc5e80afaf0 0x7fc5e80b2000 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:39.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.612+0000 7fc5fefb4640 1 --2- 172.21.15.69:0/2249360224 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5f8104460 0x7fc5f81a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:39.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.612+0000 7fc5fefb4640 1 --2- 172.21.15.69:0/2249360224 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5f8103920 0x7fc5f819a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:39.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.612+0000 7fc5fefb4640 1 --2- 172.21.15.69:0/2249360224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5f8102f40 0x7fc5f819a0c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:39.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.612+0000 7fc5fefb4640 1 -- 172.21.15.69:0/2249360224 >> 172.21.15.69:0/2249360224 conn(0x7fc5f80fd340 msgr2=0x7fc5f8101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:39.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.613+0000 7fc5fefb4640 1 -- 172.21.15.69:0/2249360224 shutdown_connections 2024-04-25T11:23:39.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.613+0000 7fc5fefb4640 1 -- 172.21.15.69:0/2249360224 wait complete. 2024-04-25T11:23:39.623 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-25T11:23:39.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.749+0000 7f41c7f7d640 1 -- 172.21.15.69:0/2117905841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41c0104d50 msgr2=0x7f41c0105160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:39.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.749+0000 7f41c7f7d640 1 --2- 172.21.15.69:0/2117905841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41c0104d50 0x7f41c0105160 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f41b0009900 tx=0x7f41b002a410 comp rx=0 tx=0).stop 2024-04-25T11:23:39.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.749+0000 7f41c7f7d640 1 -- 172.21.15.69:0/2117905841 shutdown_connections 2024-04-25T11:23:39.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.749+0000 7f41c7f7d640 1 --2- 172.21.15.69:0/2117905841 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41c0109a50 0x7f41c010be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:39.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.749+0000 7f41c7f7d640 1 --2- 172.21.15.69:0/2117905841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41c0104d50 0x7f41c0105160 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:39.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.749+0000 7f41c7f7d640 1 --2- 172.21.15.69:0/2117905841 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41c0103b10 0x7f41c0104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:39.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.749+0000 7f41c7f7d640 1 -- 172.21.15.69:0/2117905841 >> 172.21.15.69:0/2117905841 conn(0x7f41c00fed40 msgr2=0x7f41c01011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:39.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.749+0000 7f41c7f7d640 1 -- 172.21.15.69:0/2117905841 shutdown_connections 2024-04-25T11:23:39.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.749+0000 7f41c7f7d640 1 -- 172.21.15.69:0/2117905841 wait complete. 2024-04-25T11:23:39.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.749+0000 7f41c7f7d640 1 Processor -- start 2024-04-25T11:23:39.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.750+0000 7f41c7f7d640 1 -- start start 2024-04-25T11:23:39.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.750+0000 7f41c7f7d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41c0103b10 0x7f41c019a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:39.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.750+0000 7f41c7f7d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41c0104d50 0x7f41c019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:39.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.750+0000 7f41c7f7d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41c0109a50 0x7f41c01a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:39.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.750+0000 7f41c7f7d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f41c010f4c0 con 0x7f41c0109a50 2024-04-25T11:23:39.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.750+0000 7f41c7f7d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f41c010f340 con 0x7f41c0104d50 2024-04-25T11:23:39.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.750+0000 7f41c7f7d640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f41c010f640 con 0x7f41c0103b10 2024-04-25T11:23:39.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.750+0000 7f41c651a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41c0109a50 0x7f41c01a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:39.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.750+0000 7f41c651a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41c0109a50 0x7f41c01a1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37770/0 (socket says 172.21.15.69:37770) 2024-04-25T11:23:39.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.750+0000 7f41c651a640 1 -- 172.21.15.69:0/2363803650 learned_addr learned my addr 172.21.15.69:0/2363803650 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:39.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.750+0000 7f41c5d19640 1 --2- 172.21.15.69:0/2363803650 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41c0103b10 0x7f41c019a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:39.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.750+0000 7f41c5518640 1 --2- 172.21.15.69:0/2363803650 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41c0104d50 0x7f41c019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:39.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.750+0000 7f41c651a640 1 -- 172.21.15.69:0/2363803650 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41c0103b10 msgr2=0x7f41c019a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:39.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.750+0000 7f41c651a640 1 --2- 172.21.15.69:0/2363803650 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41c0103b10 0x7f41c019a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:39.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.750+0000 7f41c651a640 1 -- 172.21.15.69:0/2363803650 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41c0104d50 msgr2=0x7f41c019a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:39.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.750+0000 7f41c651a640 1 --2- 172.21.15.69:0/2363803650 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41c0104d50 0x7f41c019a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:39.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.750+0000 7f41c651a640 1 -- 172.21.15.69:0/2363803650 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c01a1c90 con 0x7f41c0109a50 2024-04-25T11:23:39.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.750+0000 7f41c651a640 1 --2- 172.21.15.69:0/2363803650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41c0109a50 0x7f41c01a1620 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f41b800eaa0 tx=0x7f41b800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:39.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.750+0000 7f41b6ffd640 1 -- 172.21.15.69:0/2363803650 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b800cdb0 con 0x7f41c0109a50 2024-04-25T11:23:39.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.750+0000 7f41b6ffd640 1 -- 172.21.15.69:0/2363803650 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f41b8005020 con 0x7f41c0109a50 2024-04-25T11:23:39.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.750+0000 7f41c7f7d640 1 -- 172.21.15.69:0/2363803650 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41c01a1f80 con 0x7f41c0109a50 2024-04-25T11:23:39.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.750+0000 7f41b6ffd640 1 -- 172.21.15.69:0/2363803650 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b80116d0 con 0x7f41c0109a50 2024-04-25T11:23:39.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.750+0000 7f41c7f7d640 1 -- 172.21.15.69:0/2363803650 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41c01a23e0 con 0x7f41c0109a50 2024-04-25T11:23:39.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.751+0000 7f41c7f7d640 1 -- 172.21.15.69:0/2363803650 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41c006a1c0 con 0x7f41c0109a50 2024-04-25T11:23:39.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.753+0000 7f41b6ffd640 1 -- 172.21.15.69:0/2363803650 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f41b80051d0 con 0x7f41c0109a50 2024-04-25T11:23:39.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.753+0000 7f41b6ffd640 1 --2- 172.21.15.69:0/2363803650 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f41ac0afb70 0x7f41ac0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:39.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.754+0000 7f41c5d19640 1 --2- 172.21.15.69:0/2363803650 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f41ac0afb70 0x7f41ac0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:39.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.754+0000 7f41b6ffd640 1 -- 172.21.15.69:0/2363803650 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7497+0+0 (secure 0 0 0) 0x7f41b80deee0 con 0x7f41c0109a50 2024-04-25T11:23:39.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.754+0000 7f41c5d19640 1 --2- 172.21.15.69:0/2363803650 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f41ac0afb70 0x7f41ac0b2080 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f41bc005fd0 tx=0x7f41bc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:39.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.757+0000 7f41b6ffd640 1 -- 172.21.15.69:0/2363803650 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f41c006a1c0 con 0x7f41c0109a50 2024-04-25T11:23:39.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:39.939+0000 7f41c7f7d640 1 -- 172.21.15.69:0/2363803650 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f41c01088b0 con 0x7f41c0109a50 2024-04-25T11:23:40.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.615+0000 7f41b6ffd640 1 -- 172.21.15.69:0/2363803650 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v73) ==== 171+0+0 (secure 0 0 0) 0x7f41c01088b0 con 0x7f41c0109a50 2024-04-25T11:23:40.610 INFO:teuthology.orchestra.run.smithi069.stderr:pool 'cephfs_data' created 2024-04-25T11:23:40.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.619+0000 7f41c7f7d640 1 -- 172.21.15.69:0/2363803650 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f41ac0afb70 msgr2=0x7f41ac0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:40.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.619+0000 7f41c7f7d640 1 --2- 172.21.15.69:0/2363803650 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f41ac0afb70 0x7f41ac0b2080 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f41bc005fd0 tx=0x7f41bc005850 comp rx=0 tx=0).stop 2024-04-25T11:23:40.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.619+0000 7f41c7f7d640 1 -- 172.21.15.69:0/2363803650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41c0109a50 msgr2=0x7f41c01a1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:40.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.619+0000 7f41c7f7d640 1 --2- 172.21.15.69:0/2363803650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41c0109a50 0x7f41c01a1620 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f41b800eaa0 tx=0x7f41b800ef60 comp rx=0 tx=0).stop 2024-04-25T11:23:40.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.619+0000 7f41c7f7d640 1 -- 172.21.15.69:0/2363803650 shutdown_connections 2024-04-25T11:23:40.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.619+0000 7f41c7f7d640 1 --2- 172.21.15.69:0/2363803650 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f41ac0afb70 0x7f41ac0b2080 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:40.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.619+0000 7f41c7f7d640 1 --2- 172.21.15.69:0/2363803650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41c0109a50 0x7f41c01a1620 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:40.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.619+0000 7f41c7f7d640 1 --2- 172.21.15.69:0/2363803650 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f41c0104d50 0x7f41c019a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:40.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.619+0000 7f41c7f7d640 1 --2- 172.21.15.69:0/2363803650 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41c0103b10 0x7f41c019a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:40.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.619+0000 7f41c7f7d640 1 -- 172.21.15.69:0/2363803650 >> 172.21.15.69:0/2363803650 conn(0x7f41c00fed40 msgr2=0x7f41c0105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:40.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.619+0000 7f41c7f7d640 1 -- 172.21.15.69:0/2363803650 shutdown_connections 2024-04-25T11:23:40.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.619+0000 7f41c7f7d640 1 -- 172.21.15.69:0/2363803650 wait complete. 2024-04-25T11:23:40.635 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T11:23:40.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.757+0000 7fc06f59e640 1 -- 172.21.15.69:0/441450532 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0701096d0 msgr2=0x7fc07010baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:40.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.757+0000 7fc06f59e640 1 --2- 172.21.15.69:0/441450532 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0701096d0 0x7fc07010baf0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc068009ca0 tx=0x7fc06802a8c0 comp rx=0 tx=0).stop 2024-04-25T11:23:40.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.757+0000 7fc06f59e640 1 -- 172.21.15.69:0/441450532 shutdown_connections 2024-04-25T11:23:40.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.757+0000 7fc06f59e640 1 --2- 172.21.15.69:0/441450532 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0701096d0 0x7fc07010baf0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:40.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.757+0000 7fc06f59e640 1 --2- 172.21.15.69:0/441450532 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc070106d70 0x7fc070109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:40.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.757+0000 7fc06f59e640 1 --2- 172.21.15.69:0/441450532 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc070104410 0x7fc070106830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:40.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.757+0000 7fc06f59e640 1 -- 172.21.15.69:0/441450532 >> 172.21.15.69:0/441450532 conn(0x7fc0700fda10 msgr2=0x7fc0700ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:40.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.757+0000 7fc06f59e640 1 -- 172.21.15.69:0/441450532 shutdown_connections 2024-04-25T11:23:40.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.757+0000 7fc06f59e640 1 -- 172.21.15.69:0/441450532 wait complete. 2024-04-25T11:23:40.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.758+0000 7fc06f59e640 1 Processor -- start 2024-04-25T11:23:40.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.758+0000 7fc06f59e640 1 -- start start 2024-04-25T11:23:40.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.758+0000 7fc06f59e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc070104410 0x7fc070199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:40.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.758+0000 7fc06f59e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc070106d70 0x7fc07019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:40.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.758+0000 7fc06f59e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0701096d0 0x7fc0701a1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:40.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.758+0000 7fc06f59e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc07010f350 con 0x7fc0701096d0 2024-04-25T11:23:40.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.758+0000 7fc06f59e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc07010f1d0 con 0x7fc070106d70 2024-04-25T11:23:40.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.758+0000 7fc06f59e640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc07010f4d0 con 0x7fc070104410 2024-04-25T11:23:40.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.758+0000 7fc06ed9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0701096d0 0x7fc0701a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:40.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.758+0000 7fc06ed9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0701096d0 0x7fc0701a1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37784/0 (socket says 172.21.15.69:37784) 2024-04-25T11:23:40.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.758+0000 7fc06ed9d640 1 -- 172.21.15.69:0/2048308962 learned_addr learned my addr 172.21.15.69:0/2048308962 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:40.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.758+0000 7fc06e59c640 1 --2- 172.21.15.69:0/2048308962 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc070104410 0x7fc070199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:40.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.759+0000 7fc06dd9b640 1 --2- 172.21.15.69:0/2048308962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc070106d70 0x7fc07019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:40.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.759+0000 7fc06ed9d640 1 -- 172.21.15.69:0/2048308962 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc070104410 msgr2=0x7fc070199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:40.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.759+0000 7fc06ed9d640 1 --2- 172.21.15.69:0/2048308962 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc070104410 0x7fc070199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:40.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.759+0000 7fc06ed9d640 1 -- 172.21.15.69:0/2048308962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc070106d70 msgr2=0x7fc07019a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:40.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.759+0000 7fc06ed9d640 1 --2- 172.21.15.69:0/2048308962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc070106d70 0x7fc07019a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:40.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.759+0000 7fc06ed9d640 1 -- 172.21.15.69:0/2048308962 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0701a1ac0 con 0x7fc0701096d0 2024-04-25T11:23:40.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.759+0000 7fc06e59c640 1 --2- 172.21.15.69:0/2048308962 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc070104410 0x7fc070199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:23:40.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.759+0000 7fc06ed9d640 1 --2- 172.21.15.69:0/2048308962 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0701096d0 0x7fc0701a1450 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc068031040 tx=0x7fc068003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:40.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.759+0000 7fc05f7fe640 1 -- 172.21.15.69:0/2048308962 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc068032ed0 con 0x7fc0701096d0 2024-04-25T11:23:40.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.759+0000 7fc05f7fe640 1 -- 172.21.15.69:0/2048308962 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc0680044d0 con 0x7fc0701096d0 2024-04-25T11:23:40.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.759+0000 7fc06f59e640 1 -- 172.21.15.69:0/2048308962 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0701a1d50 con 0x7fc0701096d0 2024-04-25T11:23:40.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.759+0000 7fc05f7fe640 1 -- 172.21.15.69:0/2048308962 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc068030ca0 con 0x7fc0701096d0 2024-04-25T11:23:40.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.759+0000 7fc06f59e640 1 -- 172.21.15.69:0/2048308962 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0701a21e0 con 0x7fc0701096d0 2024-04-25T11:23:40.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.760+0000 7fc06f59e640 1 -- 172.21.15.69:0/2048308962 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0700ff530 con 0x7fc0701096d0 2024-04-25T11:23:40.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.761+0000 7fc05f7fe640 1 -- 172.21.15.69:0/2048308962 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fc068004680 con 0x7fc0701096d0 2024-04-25T11:23:40.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.762+0000 7fc05f7fe640 1 --2- 172.21.15.69:0/2048308962 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc0580afae0 0x7fc0580b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:40.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.762+0000 7fc06e59c640 1 --2- 172.21.15.69:0/2048308962 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc0580afae0 0x7fc0580b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:40.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.762+0000 7fc05f7fe640 1 -- 172.21.15.69:0/2048308962 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7879+0+0 (secure 0 0 0) 0x7fc068100360 con 0x7fc0701096d0 2024-04-25T11:23:40.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.762+0000 7fc06e59c640 1 --2- 172.21.15.69:0/2048308962 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc0580afae0 0x7fc0580b1ff0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc060005d00 tx=0x7fc060005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:40.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.766+0000 7fc05f7fe640 1 -- 172.21.15.69:0/2048308962 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc06807fff0 con 0x7fc0701096d0 2024-04-25T11:23:40.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:40 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2249360224' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T11:23:40.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:40 smithi069 ceph-mon[29188]: osdmap e72: 12 total, 12 up, 12 in 2024-04-25T11:23:40.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:40 smithi069 ceph-mon[29188]: pgmap v276: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:40.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:40 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2363803650' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-25T11:23:40.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:40.983+0000 7fc06f59e640 1 -- 172.21.15.69:0/2048308962 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fc07002db50 con 0x7fc0701096d0 2024-04-25T11:23:40.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:40 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/2249360224' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T11:23:40.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:40 smithi152 ceph-mon[32701]: osdmap e72: 12 total, 12 up, 12 in 2024-04-25T11:23:40.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:40 smithi152 ceph-mon[32701]: pgmap v276: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:40.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:40 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/2363803650' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-25T11:23:41.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:40 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/2249360224' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T11:23:41.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:40 smithi146 ceph-mon[32821]: osdmap e72: 12 total, 12 up, 12 in 2024-04-25T11:23:41.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:40 smithi146 ceph-mon[32821]: pgmap v276: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:41.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:40 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/2363803650' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-25T11:23:41.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.617+0000 7fc05f7fe640 1 -- 172.21.15.69:0/2048308962 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fc0680c90f0 con 0x7fc0701096d0 2024-04-25T11:23:41.611 INFO:teuthology.orchestra.run.smithi069.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T11:23:41.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.620+0000 7fc06f59e640 1 -- 172.21.15.69:0/2048308962 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc0580afae0 msgr2=0x7fc0580b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:41.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.620+0000 7fc06f59e640 1 --2- 172.21.15.69:0/2048308962 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc0580afae0 0x7fc0580b1ff0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc060005d00 tx=0x7fc060005c70 comp rx=0 tx=0).stop 2024-04-25T11:23:41.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.620+0000 7fc06f59e640 1 -- 172.21.15.69:0/2048308962 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0701096d0 msgr2=0x7fc0701a1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:41.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.620+0000 7fc06f59e640 1 --2- 172.21.15.69:0/2048308962 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0701096d0 0x7fc0701a1450 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc068031040 tx=0x7fc068003ba0 comp rx=0 tx=0).stop 2024-04-25T11:23:41.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.620+0000 7fc06f59e640 1 -- 172.21.15.69:0/2048308962 shutdown_connections 2024-04-25T11:23:41.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.620+0000 7fc06f59e640 1 --2- 172.21.15.69:0/2048308962 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc0580afae0 0x7fc0580b1ff0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:41.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.620+0000 7fc06f59e640 1 --2- 172.21.15.69:0/2048308962 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0701096d0 0x7fc0701a1450 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:41.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.620+0000 7fc06f59e640 1 --2- 172.21.15.69:0/2048308962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc070106d70 0x7fc07019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:41.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.620+0000 7fc06f59e640 1 --2- 172.21.15.69:0/2048308962 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc070104410 0x7fc070199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:41.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.620+0000 7fc06f59e640 1 -- 172.21.15.69:0/2048308962 >> 172.21.15.69:0/2048308962 conn(0x7fc0700fda10 msgr2=0x7fc0701063e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:41.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.620+0000 7fc06f59e640 1 -- 172.21.15.69:0/2048308962 shutdown_connections 2024-04-25T11:23:41.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.620+0000 7fc06f59e640 1 -- 172.21.15.69:0/2048308962 wait complete. 2024-04-25T11:23:41.634 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-25T11:23:41.634 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-25T11:23:41.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.757+0000 7f8943714640 1 -- 172.21.15.69:0/3027799790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f893c1046a0 msgr2=0x7f893c10adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:41.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.757+0000 7f8943714640 1 --2- 172.21.15.69:0/3027799790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f893c1046a0 0x7f893c10adf0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8934009ca0 tx=0x7f893402a8c0 comp rx=0 tx=0).stop 2024-04-25T11:23:41.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.757+0000 7f8943714640 1 -- 172.21.15.69:0/3027799790 shutdown_connections 2024-04-25T11:23:41.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.757+0000 7f8943714640 1 --2- 172.21.15.69:0/3027799790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f893c1046a0 0x7f893c10adf0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:41.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.757+0000 7f8943714640 1 --2- 172.21.15.69:0/3027799790 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f893c103b40 0x7f893c104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:41.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.757+0000 7f8943714640 1 --2- 172.21.15.69:0/3027799790 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f893c102fe0 0x7f893c1033f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:41.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.757+0000 7f8943714640 1 -- 172.21.15.69:0/3027799790 >> 172.21.15.69:0/3027799790 conn(0x7f893c0fd380 msgr2=0x7f893c0ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:41.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.757+0000 7f8943714640 1 -- 172.21.15.69:0/3027799790 shutdown_connections 2024-04-25T11:23:41.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.757+0000 7f8943714640 1 -- 172.21.15.69:0/3027799790 wait complete. 2024-04-25T11:23:41.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.758+0000 7f8943714640 1 Processor -- start 2024-04-25T11:23:41.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.758+0000 7f8943714640 1 -- start start 2024-04-25T11:23:41.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.758+0000 7f8943714640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f893c102fe0 0x7f893c1022a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:41.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.758+0000 7f8943714640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f893c103b40 0x7f893c19ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:41.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.758+0000 7f8943714640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f893c19f1f0 0x7f893c1a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:41.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.758+0000 7f8943714640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f893c10e670 con 0x7f893c19f1f0 2024-04-25T11:23:41.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.758+0000 7f8943714640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f893c10e4f0 con 0x7f893c103b40 2024-04-25T11:23:41.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.758+0000 7f8943714640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f893c10e7f0 con 0x7f893c102fe0 2024-04-25T11:23:41.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.758+0000 7f8941cb1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f893c19f1f0 0x7f893c1a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:41.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.758+0000 7f8941cb1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f893c19f1f0 0x7f893c1a1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37794/0 (socket says 172.21.15.69:37794) 2024-04-25T11:23:41.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.758+0000 7f8941cb1640 1 -- 172.21.15.69:0/3812468084 learned_addr learned my addr 172.21.15.69:0/3812468084 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:41.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.758+0000 7f8940caf640 1 --2- 172.21.15.69:0/3812468084 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f893c103b40 0x7f893c19ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:41.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.758+0000 7f8941cb1640 1 -- 172.21.15.69:0/3812468084 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f893c102fe0 msgr2=0x7f893c1022a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:41.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.758+0000 7f8941cb1640 1 --2- 172.21.15.69:0/3812468084 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f893c102fe0 0x7f893c1022a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:41.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.758+0000 7f8941cb1640 1 -- 172.21.15.69:0/3812468084 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f893c103b40 msgr2=0x7f893c19ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:41.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.758+0000 7f8941cb1640 1 --2- 172.21.15.69:0/3812468084 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f893c103b40 0x7f893c19ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:41.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.758+0000 7f8941cb1640 1 -- 172.21.15.69:0/3812468084 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f893c1a1e50 con 0x7f893c19f1f0 2024-04-25T11:23:41.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.758+0000 7f8940caf640 1 --2- 172.21.15.69:0/3812468084 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f893c103b40 0x7f893c19ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:23:41.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.759+0000 7f8941cb1640 1 --2- 172.21.15.69:0/3812468084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f893c19f1f0 0x7f893c1a1630 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8934031040 tx=0x7f8934003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:41.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.759+0000 7f89327fc640 1 -- 172.21.15.69:0/3812468084 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8934004040 con 0x7f893c19f1f0 2024-04-25T11:23:41.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.759+0000 7f89327fc640 1 -- 172.21.15.69:0/3812468084 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8934004670 con 0x7f893c19f1f0 2024-04-25T11:23:41.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.759+0000 7f8943714640 1 -- 172.21.15.69:0/3812468084 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f893c1a20e0 con 0x7f893c19f1f0 2024-04-25T11:23:41.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.759+0000 7f89327fc640 1 -- 172.21.15.69:0/3812468084 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8934030d50 con 0x7f893c19f1f0 2024-04-25T11:23:41.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.759+0000 7f8943714640 1 -- 172.21.15.69:0/3812468084 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f893c1a2630 con 0x7f893c19f1f0 2024-04-25T11:23:41.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.760+0000 7f8943714640 1 -- 172.21.15.69:0/3812468084 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f893c06b350 con 0x7f893c19f1f0 2024-04-25T11:23:41.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.761+0000 7f89327fc640 1 -- 172.21.15.69:0/3812468084 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f89340305c0 con 0x7f893c19f1f0 2024-04-25T11:23:41.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.762+0000 7f89327fc640 1 --2- 172.21.15.69:0/3812468084 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f89280afaf0 0x7f89280b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:41.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.762+0000 7f89414b0640 1 --2- 172.21.15.69:0/3812468084 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f89280afaf0 0x7f89280b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:41.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.762+0000 7f89327fc640 1 -- 172.21.15.69:0/3812468084 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7979+0+0 (secure 0 0 0) 0x7f89341002d0 con 0x7f893c19f1f0 2024-04-25T11:23:41.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.762+0000 7f89414b0640 1 --2- 172.21.15.69:0/3812468084 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f89280afaf0 0x7f89280b2000 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8938005fd0 tx=0x7f8938005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:41.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.766+0000 7f89327fc640 1 -- 172.21.15.69:0/3812468084 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f89340c9000 con 0x7f893c19f1f0 2024-04-25T11:23:41.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:41 smithi152 ceph-mon[32701]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T11:23:41.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:41 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/2363803650' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-25T11:23:41.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:41 smithi152 ceph-mon[32701]: osdmap e73: 12 total, 12 up, 12 in 2024-04-25T11:23:41.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:41 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/2048308962' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T11:23:41.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:41 smithi152 ceph-mon[32701]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T11:23:41.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:41 smithi069 ceph-mon[29188]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T11:23:41.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:41 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2363803650' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-25T11:23:41.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:41 smithi069 ceph-mon[29188]: osdmap e73: 12 total, 12 up, 12 in 2024-04-25T11:23:41.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:41 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2048308962' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T11:23:41.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:41 smithi069 ceph-mon[29188]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T11:23:41.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:41.954+0000 7f8943714640 1 -- 172.21.15.69:0/3812468084 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f893c02dec0 con 0x7f893c19f1f0 2024-04-25T11:23:42.001 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:41 smithi146 ceph-mon[32821]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T11:23:42.001 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:41 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/2363803650' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-25T11:23:42.001 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:41 smithi146 ceph-mon[32821]: osdmap e73: 12 total, 12 up, 12 in 2024-04-25T11:23:42.001 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:41 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/2048308962' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T11:23:42.001 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:41 smithi146 ceph-mon[32821]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T11:23:42.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.632+0000 7f89327fc640 1 -- 172.21.15.69:0/3812468084 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v75) ==== 157+0+0 (secure 0 0 0) 0x7f89340d1080 con 0x7f893c19f1f0 2024-04-25T11:23:42.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.634+0000 7f8943714640 1 -- 172.21.15.69:0/3812468084 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f89280afaf0 msgr2=0x7f89280b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:42.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.634+0000 7f8943714640 1 --2- 172.21.15.69:0/3812468084 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f89280afaf0 0x7f89280b2000 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8938005fd0 tx=0x7f8938005850 comp rx=0 tx=0).stop 2024-04-25T11:23:42.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.634+0000 7f8943714640 1 -- 172.21.15.69:0/3812468084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f893c19f1f0 msgr2=0x7f893c1a1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:42.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.634+0000 7f8943714640 1 --2- 172.21.15.69:0/3812468084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f893c19f1f0 0x7f893c1a1630 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8934031040 tx=0x7f8934003ba0 comp rx=0 tx=0).stop 2024-04-25T11:23:42.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.635+0000 7f8943714640 1 -- 172.21.15.69:0/3812468084 shutdown_connections 2024-04-25T11:23:42.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.635+0000 7f8943714640 1 --2- 172.21.15.69:0/3812468084 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f89280afaf0 0x7f89280b2000 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:42.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.635+0000 7f8943714640 1 --2- 172.21.15.69:0/3812468084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f893c19f1f0 0x7f893c1a1630 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:42.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.635+0000 7f8943714640 1 --2- 172.21.15.69:0/3812468084 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f893c103b40 0x7f893c19ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:42.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.635+0000 7f8943714640 1 --2- 172.21.15.69:0/3812468084 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f893c102fe0 0x7f893c1022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:42.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.635+0000 7f8943714640 1 -- 172.21.15.69:0/3812468084 >> 172.21.15.69:0/3812468084 conn(0x7f893c0fd380 msgr2=0x7f893c106ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:42.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.635+0000 7f8943714640 1 -- 172.21.15.69:0/3812468084 shutdown_connections 2024-04-25T11:23:42.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.635+0000 7f8943714640 1 -- 172.21.15.69:0/3812468084 wait complete. 2024-04-25T11:23:42.641 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T11:23:42.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.763+0000 7fb4826f2640 1 -- 172.21.15.69:0/3272052926 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb47c101e60 msgr2=0x7fb47c102270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:42.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.763+0000 7fb4826f2640 1 --2- 172.21.15.69:0/3272052926 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb47c101e60 0x7fb47c102270 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb46c009900 tx=0x7fb46c02a410 comp rx=0 tx=0).stop 2024-04-25T11:23:42.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.763+0000 7fb4826f2640 1 -- 172.21.15.69:0/3272052926 shutdown_connections 2024-04-25T11:23:42.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.763+0000 7fb4826f2640 1 --2- 172.21.15.69:0/3272052926 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb47c1097b0 0x7fb47c10bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:42.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.763+0000 7fb4826f2640 1 --2- 172.21.15.69:0/3272052926 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb47c106c40 0x7fb47c109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:42.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.763+0000 7fb4826f2640 1 --2- 172.21.15.69:0/3272052926 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb47c101e60 0x7fb47c102270 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:42.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.763+0000 7fb4826f2640 1 -- 172.21.15.69:0/3272052926 >> 172.21.15.69:0/3272052926 conn(0x7fb47c0fd370 msgr2=0x7fb47c0ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:42.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.763+0000 7fb4826f2640 1 -- 172.21.15.69:0/3272052926 shutdown_connections 2024-04-25T11:23:42.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.763+0000 7fb4826f2640 1 -- 172.21.15.69:0/3272052926 wait complete. 2024-04-25T11:23:42.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.764+0000 7fb4826f2640 1 Processor -- start 2024-04-25T11:23:42.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.764+0000 7fb4826f2640 1 -- start start 2024-04-25T11:23:42.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.764+0000 7fb4826f2640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb47c101e60 0x7fb47c19a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:42.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.764+0000 7fb4826f2640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb47c106c40 0x7fb47c19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:42.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.764+0000 7fb4826f2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb47c1097b0 0x7fb47c1a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:42.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.764+0000 7fb4826f2640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb47c10f4a0 con 0x7fb47c1097b0 2024-04-25T11:23:42.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.764+0000 7fb4826f2640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb47c10f320 con 0x7fb47c106c40 2024-04-25T11:23:42.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.764+0000 7fb4826f2640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb47c10f620 con 0x7fb47c101e60 2024-04-25T11:23:42.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.764+0000 7fb480c8f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb47c1097b0 0x7fb47c1a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:42.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.764+0000 7fb480c8f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb47c1097b0 0x7fb47c1a15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48702/0 (socket says 172.21.15.69:48702) 2024-04-25T11:23:42.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.764+0000 7fb480c8f640 1 -- 172.21.15.69:0/167015546 learned_addr learned my addr 172.21.15.69:0/167015546 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:42.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.764+0000 7fb47bfff640 1 --2- 172.21.15.69:0/167015546 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb47c101e60 0x7fb47c19a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:42.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.764+0000 7fb47b7fe640 1 --2- 172.21.15.69:0/167015546 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb47c106c40 0x7fb47c19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:42.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.764+0000 7fb480c8f640 1 -- 172.21.15.69:0/167015546 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb47c101e60 msgr2=0x7fb47c19a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:42.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.764+0000 7fb480c8f640 1 --2- 172.21.15.69:0/167015546 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb47c101e60 0x7fb47c19a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:42.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.764+0000 7fb480c8f640 1 -- 172.21.15.69:0/167015546 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb47c106c40 msgr2=0x7fb47c19a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:42.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.764+0000 7fb480c8f640 1 --2- 172.21.15.69:0/167015546 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb47c106c40 0x7fb47c19a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:42.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.764+0000 7fb480c8f640 1 -- 172.21.15.69:0/167015546 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb47c1a1c40 con 0x7fb47c1097b0 2024-04-25T11:23:42.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.765+0000 7fb480c8f640 1 --2- 172.21.15.69:0/167015546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb47c1097b0 0x7fb47c1a15d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb474009fd0 tx=0x7fb47400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:42.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.765+0000 7fb4797fa640 1 -- 172.21.15.69:0/167015546 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb474017070 con 0x7fb47c1097b0 2024-04-25T11:23:42.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.765+0000 7fb4797fa640 1 -- 172.21.15.69:0/167015546 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb474003e00 con 0x7fb47c1097b0 2024-04-25T11:23:42.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.765+0000 7fb4826f2640 1 -- 172.21.15.69:0/167015546 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb47c1a1f30 con 0x7fb47c1097b0 2024-04-25T11:23:42.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.765+0000 7fb4797fa640 1 -- 172.21.15.69:0/167015546 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb474010460 con 0x7fb47c1097b0 2024-04-25T11:23:42.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.765+0000 7fb4826f2640 1 -- 172.21.15.69:0/167015546 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb47c1a2390 con 0x7fb47c1097b0 2024-04-25T11:23:42.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.766+0000 7fb4826f2640 1 -- 172.21.15.69:0/167015546 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb47c06a230 con 0x7fb47c1097b0 2024-04-25T11:23:42.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.767+0000 7fb4797fa640 1 -- 172.21.15.69:0/167015546 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fb474003970 con 0x7fb47c1097b0 2024-04-25T11:23:42.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.768+0000 7fb4797fa640 1 --2- 172.21.15.69:0/167015546 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fb4640afb70 0x7fb4640b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:42.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.768+0000 7fb47bfff640 1 --2- 172.21.15.69:0/167015546 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fb4640afb70 0x7fb4640b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:42.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.768+0000 7fb4797fa640 1 -- 172.21.15.69:0/167015546 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8277+0+0 (secure 0 0 0) 0x7fb4740dec00 con 0x7fb47c1097b0 2024-04-25T11:23:42.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.768+0000 7fb47bfff640 1 --2- 172.21.15.69:0/167015546 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fb4640afb70 0x7fb4640b2080 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb46c005ab0 tx=0x7fb46c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:42.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.771+0000 7fb4797fa640 1 -- 172.21.15.69:0/167015546 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb47c06a230 con 0x7fb47c1097b0 2024-04-25T11:23:42.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:42 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2048308962' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T11:23:42.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:42 smithi069 ceph-mon[29188]: osdmap e74: 12 total, 12 up, 12 in 2024-04-25T11:23:42.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:42 smithi069 ceph-mon[29188]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T11:23:42.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:42 smithi069 ceph-mon[29188]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T11:23:42.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:42 smithi069 ceph-mon[29188]: pgmap v279: 97 pgs: 7 creating+peering, 76 unknown, 14 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:42.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:42 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/3812468084' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-25T11:23:42.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:42.951+0000 7fb4826f2640 1 -- 172.21.15.69:0/167015546 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb47c0feee0 con 0x7fb47c1097b0 2024-04-25T11:23:42.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:42 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/2048308962' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T11:23:42.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:42 smithi152 ceph-mon[32701]: osdmap e74: 12 total, 12 up, 12 in 2024-04-25T11:23:42.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:42 smithi152 ceph-mon[32701]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T11:23:42.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:42 smithi152 ceph-mon[32701]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T11:23:42.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:42 smithi152 ceph-mon[32701]: pgmap v279: 97 pgs: 7 creating+peering, 76 unknown, 14 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:42.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:42 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/3812468084' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-25T11:23:43.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:42 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/2048308962' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T11:23:43.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:42 smithi146 ceph-mon[32821]: osdmap e74: 12 total, 12 up, 12 in 2024-04-25T11:23:43.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:42 smithi146 ceph-mon[32821]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T11:23:43.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:42 smithi146 ceph-mon[32821]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T11:23:43.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:42 smithi146 ceph-mon[32821]: pgmap v279: 97 pgs: 7 creating+peering, 76 unknown, 14 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:23:43.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:42 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/3812468084' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-25T11:23:43.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:43 smithi069 ceph-mon[29188]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T11:23:43.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:43 smithi069 ceph-mon[29188]: Cluster is now healthy 2024-04-25T11:23:43.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:43 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/3812468084' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-25T11:23:43.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:43 smithi069 ceph-mon[29188]: mds.? [v2:172.21.15.146:6838/2013283448,v1:172.21.15.146:6839/2013283448] up:active 2024-04-25T11:23:43.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:43 smithi069 ceph-mon[29188]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T11:23:43.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:43 smithi069 ceph-mon[29188]: osdmap e75: 12 total, 12 up, 12 in 2024-04-25T11:23:43.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:43 smithi069 ceph-mon[29188]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T11:23:43.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:43 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/167015546' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T11:23:43.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:43 smithi152 ceph-mon[32701]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T11:23:43.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:43 smithi152 ceph-mon[32701]: Cluster is now healthy 2024-04-25T11:23:43.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:43 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/3812468084' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-25T11:23:43.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:43 smithi152 ceph-mon[32701]: mds.? [v2:172.21.15.146:6838/2013283448,v1:172.21.15.146:6839/2013283448] up:active 2024-04-25T11:23:43.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:43 smithi152 ceph-mon[32701]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T11:23:43.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:43 smithi152 ceph-mon[32701]: osdmap e75: 12 total, 12 up, 12 in 2024-04-25T11:23:43.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:43 smithi152 ceph-mon[32701]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T11:23:43.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:43 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/167015546' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T11:23:44.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:43 smithi146 ceph-mon[32821]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T11:23:44.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:43 smithi146 ceph-mon[32821]: Cluster is now healthy 2024-04-25T11:23:44.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:43 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/3812468084' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-25T11:23:44.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:43 smithi146 ceph-mon[32821]: mds.? [v2:172.21.15.146:6838/2013283448,v1:172.21.15.146:6839/2013283448] up:active 2024-04-25T11:23:44.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:43 smithi146 ceph-mon[32821]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T11:23:44.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:43 smithi146 ceph-mon[32821]: osdmap e75: 12 total, 12 up, 12 in 2024-04-25T11:23:44.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:43 smithi146 ceph-mon[32821]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T11:23:44.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:43 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/167015546' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T11:23:44.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.650+0000 7fb4797fa640 1 -- 172.21.15.69:0/167015546 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v77) ==== 228+0+0 (secure 0 0 0) 0x7fb47c0feee0 con 0x7fb47c1097b0 2024-04-25T11:23:44.644 INFO:teuthology.orchestra.run.smithi069.stderr:pool 'cephfs_data_ec' created 2024-04-25T11:23:44.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.653+0000 7fb4826f2640 1 -- 172.21.15.69:0/167015546 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fb4640afb70 msgr2=0x7fb4640b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:44.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.653+0000 7fb4826f2640 1 --2- 172.21.15.69:0/167015546 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fb4640afb70 0x7fb4640b2080 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb46c005ab0 tx=0x7fb46c000bc0 comp rx=0 tx=0).stop 2024-04-25T11:23:44.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.653+0000 7fb4826f2640 1 -- 172.21.15.69:0/167015546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb47c1097b0 msgr2=0x7fb47c1a15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:44.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.653+0000 7fb4826f2640 1 --2- 172.21.15.69:0/167015546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb47c1097b0 0x7fb47c1a15d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb474009fd0 tx=0x7fb47400b260 comp rx=0 tx=0).stop 2024-04-25T11:23:44.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.653+0000 7fb4826f2640 1 -- 172.21.15.69:0/167015546 shutdown_connections 2024-04-25T11:23:44.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.653+0000 7fb4826f2640 1 --2- 172.21.15.69:0/167015546 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fb4640afb70 0x7fb4640b2080 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:44.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.653+0000 7fb4826f2640 1 --2- 172.21.15.69:0/167015546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb47c1097b0 0x7fb47c1a15d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:44.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.653+0000 7fb4826f2640 1 --2- 172.21.15.69:0/167015546 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb47c106c40 0x7fb47c19a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:44.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.653+0000 7fb4826f2640 1 --2- 172.21.15.69:0/167015546 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb47c101e60 0x7fb47c19a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:44.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.653+0000 7fb4826f2640 1 -- 172.21.15.69:0/167015546 >> 172.21.15.69:0/167015546 conn(0x7fb47c0fd370 msgr2=0x7fb47c10a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:44.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.653+0000 7fb4826f2640 1 -- 172.21.15.69:0/167015546 shutdown_connections 2024-04-25T11:23:44.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.653+0000 7fb4826f2640 1 -- 172.21.15.69:0/167015546 wait complete. 2024-04-25T11:23:44.666 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-25T11:23:44.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.788+0000 7f698fb0d640 1 -- 172.21.15.69:0/3059567582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6988104d40 msgr2=0x7f6988105150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:44.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.788+0000 7f698fb0d640 1 --2- 172.21.15.69:0/3059567582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6988104d40 0x7f6988105150 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6978009900 tx=0x7f697802a410 comp rx=0 tx=0).stop 2024-04-25T11:23:44.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.789+0000 7f698fb0d640 1 -- 172.21.15.69:0/3059567582 shutdown_connections 2024-04-25T11:23:44.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.789+0000 7f698fb0d640 1 --2- 172.21.15.69:0/3059567582 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6988109a40 0x7f698810be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:44.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.789+0000 7f698fb0d640 1 --2- 172.21.15.69:0/3059567582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6988104d40 0x7f6988105150 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:44.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.789+0000 7f698fb0d640 1 --2- 172.21.15.69:0/3059567582 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6988103b00 0x7f6988104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:44.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.789+0000 7f698fb0d640 1 -- 172.21.15.69:0/3059567582 >> 172.21.15.69:0/3059567582 conn(0x7f69880fecf0 msgr2=0x7f69881011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:44.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.789+0000 7f698fb0d640 1 -- 172.21.15.69:0/3059567582 shutdown_connections 2024-04-25T11:23:44.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.789+0000 7f698fb0d640 1 -- 172.21.15.69:0/3059567582 wait complete. 2024-04-25T11:23:44.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.789+0000 7f698fb0d640 1 Processor -- start 2024-04-25T11:23:44.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.789+0000 7f698fb0d640 1 -- start start 2024-04-25T11:23:44.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.789+0000 7f698fb0d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6988103b00 0x7f698819a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:44.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.789+0000 7f698fb0d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6988104d40 0x7f698819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:44.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.789+0000 7f698fb0d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6988109a40 0x7f69881a1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:44.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.790+0000 7f698fb0d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f698810f4d0 con 0x7f6988109a40 2024-04-25T11:23:44.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.790+0000 7f698fb0d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f698810f350 con 0x7f6988104d40 2024-04-25T11:23:44.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.790+0000 7f698fb0d640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f698810f650 con 0x7f6988103b00 2024-04-25T11:23:44.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.790+0000 7f698e0aa640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6988109a40 0x7f69881a1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:44.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.790+0000 7f698e0aa640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6988109a40 0x7f69881a1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48724/0 (socket says 172.21.15.69:48724) 2024-04-25T11:23:44.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.790+0000 7f698e0aa640 1 -- 172.21.15.69:0/2794126164 learned_addr learned my addr 172.21.15.69:0/2794126164 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:44.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.790+0000 7f698d0a8640 1 --2- 172.21.15.69:0/2794126164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6988104d40 0x7f698819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:44.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.790+0000 7f698d8a9640 1 --2- 172.21.15.69:0/2794126164 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6988103b00 0x7f698819a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:44.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.790+0000 7f698e0aa640 1 -- 172.21.15.69:0/2794126164 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6988103b00 msgr2=0x7f698819a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:44.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.790+0000 7f698e0aa640 1 --2- 172.21.15.69:0/2794126164 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6988103b00 0x7f698819a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:44.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.790+0000 7f698e0aa640 1 -- 172.21.15.69:0/2794126164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6988104d40 msgr2=0x7f698819a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:44.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.790+0000 7f698e0aa640 1 --2- 172.21.15.69:0/2794126164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6988104d40 0x7f698819a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:44.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.790+0000 7f698e0aa640 1 -- 172.21.15.69:0/2794126164 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69881a1c70 con 0x7f6988109a40 2024-04-25T11:23:44.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.790+0000 7f698d8a9640 1 --2- 172.21.15.69:0/2794126164 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6988103b00 0x7f698819a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:23:44.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.790+0000 7f698e0aa640 1 --2- 172.21.15.69:0/2794126164 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6988109a40 0x7f69881a1600 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f698000d750 tx=0x7f698000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:44.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.790+0000 7f697effd640 1 -- 172.21.15.69:0/2794126164 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6980017070 con 0x7f6988109a40 2024-04-25T11:23:44.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.790+0000 7f697effd640 1 -- 172.21.15.69:0/2794126164 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f69800054b0 con 0x7f6988109a40 2024-04-25T11:23:44.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.790+0000 7f697effd640 1 -- 172.21.15.69:0/2794126164 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6980004b90 con 0x7f6988109a40 2024-04-25T11:23:44.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.790+0000 7f698fb0d640 1 -- 172.21.15.69:0/2794126164 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69881a1f60 con 0x7f6988109a40 2024-04-25T11:23:44.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.790+0000 7f698fb0d640 1 -- 172.21.15.69:0/2794126164 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69881a2370 con 0x7f6988109a40 2024-04-25T11:23:44.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.791+0000 7f698fb0d640 1 -- 172.21.15.69:0/2794126164 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f698806afd0 con 0x7f6988109a40 2024-04-25T11:23:44.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.793+0000 7f697effd640 1 -- 172.21.15.69:0/2794126164 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f6980004d30 con 0x7f6988109a40 2024-04-25T11:23:44.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.793+0000 7f697effd640 1 --2- 172.21.15.69:0/2794126164 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f69740afaf0 0x7f69740b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:44.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.793+0000 7f698d8a9640 1 --2- 172.21.15.69:0/2794126164 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f69740afaf0 0x7f69740b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:44.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.794+0000 7f697effd640 1 -- 172.21.15.69:0/2794126164 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8770+0+0 (secure 0 0 0) 0x7f69800dec70 con 0x7f6988109a40 2024-04-25T11:23:44.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.794+0000 7f698d8a9640 1 --2- 172.21.15.69:0/2794126164 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f69740afaf0 0x7f69740b2000 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6988104700 tx=0x7f6984009400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:44.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.797+0000 7f697effd640 1 -- 172.21.15.69:0/2794126164 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f698806afd0 con 0x7f6988109a40 2024-04-25T11:23:44.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:44 smithi069 ceph-mon[29188]: osdmap e76: 12 total, 12 up, 12 in 2024-04-25T11:23:44.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:44 smithi069 ceph-mon[29188]: pgmap v282: 97 pgs: 7 creating+peering, 49 unknown, 41 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 1 op/s 2024-04-25T11:23:44.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:44.976+0000 7f698fb0d640 1 -- 172.21.15.69:0/2794126164 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f69881088a0 con 0x7f6988109a40 2024-04-25T11:23:44.980 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:44 smithi152 ceph-mon[32701]: osdmap e76: 12 total, 12 up, 12 in 2024-04-25T11:23:44.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:44 smithi152 ceph-mon[32701]: pgmap v282: 97 pgs: 7 creating+peering, 49 unknown, 41 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 1 op/s 2024-04-25T11:23:45.069 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:44 smithi146 ceph-mon[32821]: osdmap e76: 12 total, 12 up, 12 in 2024-04-25T11:23:45.070 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:44 smithi146 ceph-mon[32821]: pgmap v282: 97 pgs: 7 creating+peering, 49 unknown, 41 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 1 op/s 2024-04-25T11:23:45.349 INFO:journalctl@ceph.mds.b.smithi146.stdout:Apr 25 11:23:45 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-b[62892]: 2024-04-25T11:23:45.074+0000 7f44c9446700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:23:45.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.652+0000 7f697effd640 1 -- 172.21.15.69:0/2794126164 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v78) ==== 169+0+0 (secure 0 0 0) 0x7f69881088a0 con 0x7f6988109a40 2024-04-25T11:23:45.646 INFO:teuthology.orchestra.run.smithi069.stderr:set pool 4 allow_ec_overwrites to true 2024-04-25T11:23:45.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.655+0000 7f698fb0d640 1 -- 172.21.15.69:0/2794126164 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f69740afaf0 msgr2=0x7f69740b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:45.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.655+0000 7f698fb0d640 1 --2- 172.21.15.69:0/2794126164 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f69740afaf0 0x7f69740b2000 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6988104700 tx=0x7f6984009400 comp rx=0 tx=0).stop 2024-04-25T11:23:45.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.655+0000 7f698fb0d640 1 -- 172.21.15.69:0/2794126164 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6988109a40 msgr2=0x7f69881a1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:45.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.655+0000 7f698fb0d640 1 --2- 172.21.15.69:0/2794126164 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6988109a40 0x7f69881a1600 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f698000d750 tx=0x7f698000dc10 comp rx=0 tx=0).stop 2024-04-25T11:23:45.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.655+0000 7f698fb0d640 1 -- 172.21.15.69:0/2794126164 shutdown_connections 2024-04-25T11:23:45.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.655+0000 7f698fb0d640 1 --2- 172.21.15.69:0/2794126164 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f69740afaf0 0x7f69740b2000 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:45.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.655+0000 7f698fb0d640 1 --2- 172.21.15.69:0/2794126164 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6988109a40 0x7f69881a1600 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:45.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.655+0000 7f698fb0d640 1 --2- 172.21.15.69:0/2794126164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6988104d40 0x7f698819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:45.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.655+0000 7f698fb0d640 1 --2- 172.21.15.69:0/2794126164 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6988103b00 0x7f698819a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:45.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.655+0000 7f698fb0d640 1 -- 172.21.15.69:0/2794126164 >> 172.21.15.69:0/2794126164 conn(0x7f69880fecf0 msgr2=0x7f6988105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:45.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.655+0000 7f698fb0d640 1 -- 172.21.15.69:0/2794126164 shutdown_connections 2024-04-25T11:23:45.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.655+0000 7f698fb0d640 1 -- 172.21.15.69:0/2794126164 wait complete. 2024-04-25T11:23:45.671 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-25T11:23:45.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.786+0000 7f85a7625640 1 -- 172.21.15.69:0/492068636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85a0102a00 msgr2=0x7f85a0103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:45.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.786+0000 7f85a7625640 1 --2- 172.21.15.69:0/492068636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85a0102a00 0x7f85a0103700 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f859c009900 tx=0x7f859c02a410 comp rx=0 tx=0).stop 2024-04-25T11:23:45.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.786+0000 7f85a7625640 1 -- 172.21.15.69:0/492068636 shutdown_connections 2024-04-25T11:23:45.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.786+0000 7f85a7625640 1 --2- 172.21.15.69:0/492068636 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85a0108940 0x7f85a010ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:45.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.786+0000 7f85a7625640 1 --2- 172.21.15.69:0/492068636 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85a0103c40 0x7f85a0104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:45.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.786+0000 7f85a7625640 1 --2- 172.21.15.69:0/492068636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85a0102a00 0x7f85a0103700 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:45.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.786+0000 7f85a7625640 1 -- 172.21.15.69:0/492068636 >> 172.21.15.69:0/492068636 conn(0x7f85a00fdbf0 msgr2=0x7f85a01000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:45.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.786+0000 7f85a7625640 1 -- 172.21.15.69:0/492068636 shutdown_connections 2024-04-25T11:23:45.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.786+0000 7f85a7625640 1 -- 172.21.15.69:0/492068636 wait complete. 2024-04-25T11:23:45.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.787+0000 7f85a7625640 1 Processor -- start 2024-04-25T11:23:45.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.787+0000 7f85a7625640 1 -- start start 2024-04-25T11:23:45.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.787+0000 7f85a7625640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85a0102a00 0x7f85a019a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:45.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.787+0000 7f85a7625640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85a0103c40 0x7f85a019a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:45.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.787+0000 7f85a7625640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85a0108940 0x7f85a01a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:45.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.787+0000 7f85a7625640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f85a010f520 con 0x7f85a0108940 2024-04-25T11:23:45.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.787+0000 7f85a7625640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f85a010f3a0 con 0x7f85a0103c40 2024-04-25T11:23:45.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.787+0000 7f85a7625640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f85a010f6a0 con 0x7f85a0102a00 2024-04-25T11:23:45.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.787+0000 7f85a5bc2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85a0108940 0x7f85a01a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:45.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.787+0000 7f85a5bc2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85a0108940 0x7f85a01a1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48744/0 (socket says 172.21.15.69:48744) 2024-04-25T11:23:45.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.787+0000 7f85a5bc2640 1 -- 172.21.15.69:0/1579774619 learned_addr learned my addr 172.21.15.69:0/1579774619 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:45.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.787+0000 7f85a4bc0640 1 --2- 172.21.15.69:0/1579774619 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85a0103c40 0x7f85a019a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:45.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.788+0000 7f85a53c1640 1 --2- 172.21.15.69:0/1579774619 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85a0102a00 0x7f85a019a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:45.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.788+0000 7f85a5bc2640 1 -- 172.21.15.69:0/1579774619 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85a0102a00 msgr2=0x7f85a019a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:45.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.788+0000 7f85a5bc2640 1 --2- 172.21.15.69:0/1579774619 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85a0102a00 0x7f85a019a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:45.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.788+0000 7f85a5bc2640 1 -- 172.21.15.69:0/1579774619 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85a0103c40 msgr2=0x7f85a019a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:45.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.788+0000 7f85a5bc2640 1 --2- 172.21.15.69:0/1579774619 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85a0103c40 0x7f85a019a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:45.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.788+0000 7f85a5bc2640 1 -- 172.21.15.69:0/1579774619 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85a01a1d80 con 0x7f85a0108940 2024-04-25T11:23:45.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.788+0000 7f85a5bc2640 1 --2- 172.21.15.69:0/1579774619 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85a0108940 0x7f85a01a1710 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f859800b970 tx=0x7f859800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:45.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.788+0000 7f85967fc640 1 -- 172.21.15.69:0/1579774619 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8598019070 con 0x7f85a0108940 2024-04-25T11:23:45.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.788+0000 7f85967fc640 1 -- 172.21.15.69:0/1579774619 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8598014440 con 0x7f85a0108940 2024-04-25T11:23:45.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.788+0000 7f85967fc640 1 -- 172.21.15.69:0/1579774619 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85980075d0 con 0x7f85a0108940 2024-04-25T11:23:45.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.788+0000 7f85a7625640 1 -- 172.21.15.69:0/1579774619 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85a01a2070 con 0x7f85a0108940 2024-04-25T11:23:45.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.788+0000 7f85a7625640 1 -- 172.21.15.69:0/1579774619 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85a01a25f0 con 0x7f85a0108940 2024-04-25T11:23:45.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.789+0000 7f85a7625640 1 -- 172.21.15.69:0/1579774619 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85a0107750 con 0x7f85a0108940 2024-04-25T11:23:45.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.791+0000 7f85967fc640 1 -- 172.21.15.69:0/1579774619 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f859801c070 con 0x7f85a0108940 2024-04-25T11:23:45.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.792+0000 7f85967fc640 1 --2- 172.21.15.69:0/1579774619 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f858c0afaf0 0x7f858c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:45.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.792+0000 7f85a53c1640 1 --2- 172.21.15.69:0/1579774619 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f858c0afaf0 0x7f858c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:45.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.792+0000 7f85967fc640 1 -- 172.21.15.69:0/1579774619 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8770+0+0 (secure 0 0 0) 0x7f85980de900 con 0x7f85a0108940 2024-04-25T11:23:45.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.792+0000 7f85a53c1640 1 --2- 172.21.15.69:0/1579774619 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f858c0afaf0 0x7f858c0b2000 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f859c031460 tx=0x7f859c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:45.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:45.795+0000 7f85967fc640 1 -- 172.21.15.69:0/1579774619 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f85980a72c0 con 0x7f85a0108940 2024-04-25T11:23:45.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:45 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/167015546' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T11:23:45.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:45 smithi069 ceph-mon[29188]: osdmap e77: 12 total, 12 up, 12 in 2024-04-25T11:23:45.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:45 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2794126164' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-25T11:23:45.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:45 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/167015546' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T11:23:45.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:45 smithi152 ceph-mon[32701]: osdmap e77: 12 total, 12 up, 12 in 2024-04-25T11:23:45.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:45 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/2794126164' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-25T11:23:46.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.016+0000 7f85a7625640 1 -- 172.21.15.69:0/1579774619 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f85a002dcd0 con 0x7f85a0108940 2024-04-25T11:23:46.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:45 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/167015546' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T11:23:46.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:45 smithi146 ceph-mon[32821]: osdmap e77: 12 total, 12 up, 12 in 2024-04-25T11:23:46.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:45 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/2794126164' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-25T11:23:46.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.669+0000 7f85967fc640 1 -- 172.21.15.69:0/1579774619 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f85980af340 con 0x7f85a0108940 2024-04-25T11:23:46.663 INFO:teuthology.orchestra.run.smithi069.stderr:added data pool 4 to fsmap 2024-04-25T11:23:46.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.672+0000 7f85a7625640 1 -- 172.21.15.69:0/1579774619 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f858c0afaf0 msgr2=0x7f858c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:46.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.672+0000 7f85a7625640 1 --2- 172.21.15.69:0/1579774619 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f858c0afaf0 0x7f858c0b2000 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f859c031460 tx=0x7f859c000bc0 comp rx=0 tx=0).stop 2024-04-25T11:23:46.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.672+0000 7f85a7625640 1 -- 172.21.15.69:0/1579774619 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85a0108940 msgr2=0x7f85a01a1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:46.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.672+0000 7f85a7625640 1 --2- 172.21.15.69:0/1579774619 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85a0108940 0x7f85a01a1710 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f859800b970 tx=0x7f859800be30 comp rx=0 tx=0).stop 2024-04-25T11:23:46.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.672+0000 7f85a7625640 1 -- 172.21.15.69:0/1579774619 shutdown_connections 2024-04-25T11:23:46.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.672+0000 7f85a7625640 1 --2- 172.21.15.69:0/1579774619 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f858c0afaf0 0x7f858c0b2000 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:46.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.672+0000 7f85a7625640 1 --2- 172.21.15.69:0/1579774619 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85a0108940 0x7f85a01a1710 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:46.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.672+0000 7f85a7625640 1 --2- 172.21.15.69:0/1579774619 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85a0103c40 0x7f85a019a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:46.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.672+0000 7f85a7625640 1 --2- 172.21.15.69:0/1579774619 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85a0102a00 0x7f85a019a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:46.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.672+0000 7f85a7625640 1 -- 172.21.15.69:0/1579774619 >> 172.21.15.69:0/1579774619 conn(0x7f85a00fdbf0 msgr2=0x7f85a0104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:46.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.672+0000 7f85a7625640 1 -- 172.21.15.69:0/1579774619 shutdown_connections 2024-04-25T11:23:46.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.672+0000 7f85a7625640 1 -- 172.21.15.69:0/1579774619 wait complete. 2024-04-25T11:23:46.680 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:23:46.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.801+0000 7ff292334640 1 -- 172.21.15.69:0/514204443 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff28c103c00 msgr2=0x7ff28c104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:46.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.801+0000 7ff292334640 1 --2- 172.21.15.69:0/514204443 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff28c103c00 0x7ff28c104010 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff280009900 tx=0x7ff28002a410 comp rx=0 tx=0).stop 2024-04-25T11:23:46.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.801+0000 7ff292334640 1 -- 172.21.15.69:0/514204443 shutdown_connections 2024-04-25T11:23:46.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.801+0000 7ff292334640 1 --2- 172.21.15.69:0/514204443 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff28c108900 0x7ff28c10ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:46.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.801+0000 7ff292334640 1 --2- 172.21.15.69:0/514204443 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff28c103c00 0x7ff28c104010 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:46.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.801+0000 7ff292334640 1 --2- 172.21.15.69:0/514204443 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff28c1029c0 0x7ff28c1036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:46.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.801+0000 7ff292334640 1 -- 172.21.15.69:0/514204443 >> 172.21.15.69:0/514204443 conn(0x7ff28c0fdbd0 msgr2=0x7ff28c100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:46.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.801+0000 7ff292334640 1 -- 172.21.15.69:0/514204443 shutdown_connections 2024-04-25T11:23:46.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.801+0000 7ff292334640 1 -- 172.21.15.69:0/514204443 wait complete. 2024-04-25T11:23:46.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.801+0000 7ff292334640 1 Processor -- start 2024-04-25T11:23:46.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.801+0000 7ff292334640 1 -- start start 2024-04-25T11:23:46.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.802+0000 7ff292334640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff28c1029c0 0x7ff28c199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:46.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.802+0000 7ff292334640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff28c103c00 0x7ff28c199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:46.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.802+0000 7ff292334640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff28c108900 0x7ff28c1a04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:46.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.802+0000 7ff292334640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff28c10e390 con 0x7ff28c103c00 2024-04-25T11:23:46.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.802+0000 7ff292334640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff28c10e210 con 0x7ff28c108900 2024-04-25T11:23:46.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.802+0000 7ff292334640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff28c10e510 con 0x7ff28c1029c0 2024-04-25T11:23:46.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.802+0000 7ff28b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff28c103c00 0x7ff28c199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:46.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.802+0000 7ff28b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff28c103c00 0x7ff28c199560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48766/0 (socket says 172.21.15.69:48766) 2024-04-25T11:23:46.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.802+0000 7ff28b7fe640 1 -- 172.21.15.69:0/3513710922 learned_addr learned my addr 172.21.15.69:0/3513710922 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:46.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.802+0000 7ff2908d1640 1 --2- 172.21.15.69:0/3513710922 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff28c108900 0x7ff28c1a04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:46.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.802+0000 7ff28bfff640 1 --2- 172.21.15.69:0/3513710922 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff28c1029c0 0x7ff28c199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:46.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.802+0000 7ff28b7fe640 1 -- 172.21.15.69:0/3513710922 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff28c1029c0 msgr2=0x7ff28c199020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:46.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.802+0000 7ff28b7fe640 1 --2- 172.21.15.69:0/3513710922 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff28c1029c0 0x7ff28c199020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:46.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.802+0000 7ff28b7fe640 1 -- 172.21.15.69:0/3513710922 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff28c108900 msgr2=0x7ff28c1a04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:46.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.802+0000 7ff28b7fe640 1 --2- 172.21.15.69:0/3513710922 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff28c108900 0x7ff28c1a04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:46.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.802+0000 7ff28b7fe640 1 -- 172.21.15.69:0/3513710922 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff28c1a0b60 con 0x7ff28c103c00 2024-04-25T11:23:46.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.802+0000 7ff28b7fe640 1 --2- 172.21.15.69:0/3513710922 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff28c103c00 0x7ff28c199560 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff280009230 tx=0x7ff2800042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:46.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.802+0000 7ff2897fa640 1 -- 172.21.15.69:0/3513710922 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28003f030 con 0x7ff28c103c00 2024-04-25T11:23:46.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.802+0000 7ff292334640 1 -- 172.21.15.69:0/3513710922 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff28c1a0df0 con 0x7ff28c103c00 2024-04-25T11:23:46.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.802+0000 7ff2897fa640 1 -- 172.21.15.69:0/3513710922 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff280003c90 con 0x7ff28c103c00 2024-04-25T11:23:46.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.802+0000 7ff2897fa640 1 -- 172.21.15.69:0/3513710922 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff280035070 con 0x7ff28c103c00 2024-04-25T11:23:46.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.802+0000 7ff292334640 1 -- 172.21.15.69:0/3513710922 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff28c1a1250 con 0x7ff28c103c00 2024-04-25T11:23:46.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.803+0000 7ff292334640 1 -- 172.21.15.69:0/3513710922 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff28c107710 con 0x7ff28c103c00 2024-04-25T11:23:46.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.805+0000 7ff2897fa640 1 -- 172.21.15.69:0/3513710922 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7ff280042030 con 0x7ff28c103c00 2024-04-25T11:23:46.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.806+0000 7ff2897fa640 1 --2- 172.21.15.69:0/3513710922 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ff2740afaf0 0x7ff2740b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:46.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.806+0000 7ff28bfff640 1 --2- 172.21.15.69:0/3513710922 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ff2740afaf0 0x7ff2740b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:46.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.806+0000 7ff2897fa640 1 -- 172.21.15.69:0/3513710922 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8802+0+0 (secure 0 0 0) 0x7ff2800ff8e0 con 0x7ff28c103c00 2024-04-25T11:23:46.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.806+0000 7ff28bfff640 1 --2- 172.21.15.69:0/3513710922 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ff2740afaf0 0x7ff2740b2000 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff27c0058c0 tx=0x7ff27c00a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:46.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:46.809+0000 7ff2897fa640 1 -- 172.21.15.69:0/3513710922 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff2800c8280 con 0x7ff28c103c00 2024-04-25T11:23:46.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:46 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2794126164' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-25T11:23:46.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:46 smithi069 ceph-mon[29188]: osdmap e78: 12 total, 12 up, 12 in 2024-04-25T11:23:46.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:46 smithi069 ceph-mon[29188]: pgmap v285: 129 pgs: 7 creating+peering, 29 unknown, 93 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 6 op/s 2024-04-25T11:23:46.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:46 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/1579774619' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-25T11:23:46.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:46 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/2794126164' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-25T11:23:46.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:46 smithi152 ceph-mon[32701]: osdmap e78: 12 total, 12 up, 12 in 2024-04-25T11:23:46.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:46 smithi152 ceph-mon[32701]: pgmap v285: 129 pgs: 7 creating+peering, 29 unknown, 93 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 6 op/s 2024-04-25T11:23:46.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:46 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/1579774619' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-25T11:23:47.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.030+0000 7ff292334640 1 -- 172.21.15.69:0/3513710922 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff28c02db50 con 0x7ff28c103c00 2024-04-25T11:23:47.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.030+0000 7ff2897fa640 1 -- 172.21.15.69:0/3513710922 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11809 (secure 0 0 0) 0x7ff2800c7b80 con 0x7ff28c103c00 2024-04-25T11:23:47.025 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:23:47.025 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3399424094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3399424094},{"type":"v1","addr":"172.21.15.69:6835","nonce":3399424094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1463217771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1463217771},{"type":"v1","addr":"172.21.15.69:6837","nonce":1463217771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/966316001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":966316001},{"type":"v1","addr":"172.21.15.69:6839","nonce":966316001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/2060225181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":2060225181},{"type":"v1","addr":"172.21.15.69:6841","nonce":2060225181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/4197482473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":4197482473},{"type":"v1","addr":"172.21.15.152:6833","nonce":4197482473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/4267885323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4267885323},{"type":"v1","addr":"172.21.15.146:6833","nonce":4267885323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24439,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/3188843362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3188843362},{"type":"v1","addr":"172.21.15.152:6835","nonce":3188843362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/563274982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":563274982},{"type":"v1","addr":"172.21.15.152:6837","nonce":563274982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1876533699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1876533699},{"type":"v1","addr":"172.21.15.146:6835","nonce":1876533699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24475,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6839/3008615657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":3008615657},{"type":"v1","addr":"172.21.15.152:6839","nonce":3008615657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/2730317134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2730317134},{"type":"v1","addr":"172.21.15.146:6837","nonce":2730317134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:23:40.984930+0000","modified":"2024-04-25T11:23:46.017687+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24515},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6839/2013283448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":2013283448},{"type":"v1","addr":"172.21.15.146:6839","nonce":2013283448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24515,"qdb_cluster":[24515]},"id":1}]} 2024-04-25T11:23:47.025 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 17 2024-04-25T11:23:47.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.032+0000 7ff292334640 1 -- 172.21.15.69:0/3513710922 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ff2740afaf0 msgr2=0x7ff2740b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:47.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.032+0000 7ff292334640 1 --2- 172.21.15.69:0/3513710922 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ff2740afaf0 0x7ff2740b2000 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff27c0058c0 tx=0x7ff27c00a4c0 comp rx=0 tx=0).stop 2024-04-25T11:23:47.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.032+0000 7ff292334640 1 -- 172.21.15.69:0/3513710922 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff28c103c00 msgr2=0x7ff28c199560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:47.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.032+0000 7ff292334640 1 --2- 172.21.15.69:0/3513710922 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff28c103c00 0x7ff28c199560 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff280009230 tx=0x7ff2800042f0 comp rx=0 tx=0).stop 2024-04-25T11:23:47.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.032+0000 7ff292334640 1 -- 172.21.15.69:0/3513710922 shutdown_connections 2024-04-25T11:23:47.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.033+0000 7ff292334640 1 --2- 172.21.15.69:0/3513710922 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ff2740afaf0 0x7ff2740b2000 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:47.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.033+0000 7ff292334640 1 --2- 172.21.15.69:0/3513710922 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff28c108900 0x7ff28c1a04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:47.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.033+0000 7ff292334640 1 --2- 172.21.15.69:0/3513710922 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff28c103c00 0x7ff28c199560 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:47.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.033+0000 7ff292334640 1 --2- 172.21.15.69:0/3513710922 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff28c1029c0 0x7ff28c199020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:47.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.033+0000 7ff292334640 1 -- 172.21.15.69:0/3513710922 >> 172.21.15.69:0/3513710922 conn(0x7ff28c0fdbd0 msgr2=0x7ff28c1048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:47.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.033+0000 7ff292334640 1 -- 172.21.15.69:0/3513710922 shutdown_connections 2024-04-25T11:23:47.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.033+0000 7ff292334640 1 -- 172.21.15.69:0/3513710922 wait complete. 2024-04-25T11:23:47.038 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:23:47.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:46 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/2794126164' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-25T11:23:47.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:46 smithi146 ceph-mon[32821]: osdmap e78: 12 total, 12 up, 12 in 2024-04-25T11:23:47.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:46 smithi146 ceph-mon[32821]: pgmap v285: 129 pgs: 7 creating+peering, 29 unknown, 93 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 6 op/s 2024-04-25T11:23:47.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:46 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/1579774619' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-25T11:23:47.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.151+0000 7f6faae6e640 1 -- 172.21.15.69:0/1754658067 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fa4103c40 msgr2=0x7f6fa4104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:47.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.151+0000 7f6faae6e640 1 --2- 172.21.15.69:0/1754658067 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fa4103c40 0x7f6fa4104050 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c009900 tx=0x7f6f8c02a410 comp rx=0 tx=0).stop 2024-04-25T11:23:47.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.151+0000 7f6faae6e640 1 -- 172.21.15.69:0/1754658067 shutdown_connections 2024-04-25T11:23:47.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.151+0000 7f6faae6e640 1 --2- 172.21.15.69:0/1754658067 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fa4108940 0x7f6fa410ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:47.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.151+0000 7f6faae6e640 1 --2- 172.21.15.69:0/1754658067 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fa4103c40 0x7f6fa4104050 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:47.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.151+0000 7f6faae6e640 1 --2- 172.21.15.69:0/1754658067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6fa4102a00 0x7f6fa4103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:47.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.151+0000 7f6faae6e640 1 -- 172.21.15.69:0/1754658067 >> 172.21.15.69:0/1754658067 conn(0x7f6fa40fdbf0 msgr2=0x7f6fa41000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:47.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.152+0000 7f6faae6e640 1 -- 172.21.15.69:0/1754658067 shutdown_connections 2024-04-25T11:23:47.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.152+0000 7f6faae6e640 1 -- 172.21.15.69:0/1754658067 wait complete. 2024-04-25T11:23:47.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.152+0000 7f6faae6e640 1 Processor -- start 2024-04-25T11:23:47.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.152+0000 7f6faae6e640 1 -- start start 2024-04-25T11:23:47.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.152+0000 7f6faae6e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6fa4102a00 0x7f6fa4199050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:47.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.152+0000 7f6faae6e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fa4103c40 0x7f6fa4199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:47.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.152+0000 7f6faae6e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fa4108940 0x7f6fa41a0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:47.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.152+0000 7f6faae6e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa410e480 con 0x7f6fa4108940 2024-04-25T11:23:47.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.152+0000 7f6faae6e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa410e300 con 0x7f6fa4102a00 2024-04-25T11:23:47.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.152+0000 7f6faae6e640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa410e600 con 0x7f6fa4103c40 2024-04-25T11:23:47.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.152+0000 7f6fa940b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fa4108940 0x7f6fa41a0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:47.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.152+0000 7f6fa940b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fa4108940 0x7f6fa41a0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48796/0 (socket says 172.21.15.69:48796) 2024-04-25T11:23:47.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.152+0000 7f6fa940b640 1 -- 172.21.15.69:0/2747968591 learned_addr learned my addr 172.21.15.69:0/2747968591 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:47.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.153+0000 7f6fa8c0a640 1 --2- 172.21.15.69:0/2747968591 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6fa4102a00 0x7f6fa4199050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:47.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.153+0000 7f6fa3fff640 1 --2- 172.21.15.69:0/2747968591 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fa4103c40 0x7f6fa4199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:47.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.153+0000 7f6fa940b640 1 -- 172.21.15.69:0/2747968591 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fa4103c40 msgr2=0x7f6fa4199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:47.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.153+0000 7f6fa940b640 1 --2- 172.21.15.69:0/2747968591 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fa4103c40 0x7f6fa4199590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:47.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.153+0000 7f6fa940b640 1 -- 172.21.15.69:0/2747968591 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6fa4102a00 msgr2=0x7f6fa4199050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:47.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.153+0000 7f6fa940b640 1 --2- 172.21.15.69:0/2747968591 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6fa4102a00 0x7f6fa4199050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:47.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.153+0000 7f6fa940b640 1 -- 172.21.15.69:0/2747968591 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa41a0b90 con 0x7f6fa4108940 2024-04-25T11:23:47.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.153+0000 7f6fa940b640 1 --2- 172.21.15.69:0/2747968591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fa4108940 0x7f6fa41a0520 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6f94009fd0 tx=0x7f6f9400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:47.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.153+0000 7f6fa1ffb640 1 -- 172.21.15.69:0/2747968591 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f94011830 con 0x7f6fa4108940 2024-04-25T11:23:47.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.153+0000 7f6fa1ffb640 1 -- 172.21.15.69:0/2747968591 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6f94016070 con 0x7f6fa4108940 2024-04-25T11:23:47.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.153+0000 7f6faae6e640 1 -- 172.21.15.69:0/2747968591 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fa41a0e80 con 0x7f6fa4108940 2024-04-25T11:23:47.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.153+0000 7f6fa1ffb640 1 -- 172.21.15.69:0/2747968591 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f94010d20 con 0x7f6fa4108940 2024-04-25T11:23:47.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.153+0000 7f6faae6e640 1 -- 172.21.15.69:0/2747968591 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fa41a1360 con 0x7f6fa4108940 2024-04-25T11:23:47.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.154+0000 7f6faae6e640 1 -- 172.21.15.69:0/2747968591 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fa4107750 con 0x7f6fa4108940 2024-04-25T11:23:47.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.156+0000 7f6fa1ffb640 1 -- 172.21.15.69:0/2747968591 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f6f940119d0 con 0x7f6fa4108940 2024-04-25T11:23:47.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.156+0000 7f6fa1ffb640 1 --2- 172.21.15.69:0/2747968591 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f6f900afaf0 0x7f6f900b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:47.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.156+0000 7f6fa8c0a640 1 --2- 172.21.15.69:0/2747968591 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f6f900afaf0 0x7f6f900b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:47.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.156+0000 7f6fa1ffb640 1 -- 172.21.15.69:0/2747968591 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8802+0+0 (secure 0 0 0) 0x7f6f940deba0 con 0x7f6fa4108940 2024-04-25T11:23:47.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.157+0000 7f6fa8c0a640 1 --2- 172.21.15.69:0/2747968591 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f6f900afaf0 0x7f6f900b2000 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4103600 tx=0x7f6f9c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:47.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.160+0000 7f6fa1ffb640 1 -- 172.21.15.69:0/2747968591 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6f940a7540 con 0x7f6fa4108940 2024-04-25T11:23:47.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.337+0000 7f6faae6e640 1 -- 172.21.15.69:0/2747968591 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6fa402dd20 con 0x7f6fa4108940 2024-04-25T11:23:47.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.339+0000 7f6fa1ffb640 1 -- 172.21.15.69:0/2747968591 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+24143 (secure 0 0 0) 0x7f6f940a6e40 con 0x7f6fa4108940 2024-04-25T11:23:47.333 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:23:47.333 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":79,"fsid":"f7ed73dc-02f4-11ef-bc93-c7b262605968","created":"2024-04-25T11:15:37.965281+0000","modified":"2024-04-25T11:23:46.017458+0000","last_up_change":"2024-04-25T11:22:50.691869+0000","last_in_change":"2024-04-25T11:22:30.271959+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:18:57.781263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:23:38.685110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:23:39.943803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T11:23:43.650521+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6a0d179-4998-4754-bf82-2ea3dac54e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6803","nonce":3756025235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6805","nonce":3756025235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6809","nonce":3756025235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6807","nonce":3756025235}]},"public_addr":"172.21.15.69:6803/3756025235","cluster_addr":"172.21.15.69:6805/3756025235","heartbeat_back_addr":"172.21.15.69:6809/3756025235","heartbeat_front_addr":"172.21.15.69:6807/3756025235","state":["exists","up"]},{"osd":1,"uuid":"c0c66930-4233-493f-b2bb-99c473994b59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6801","nonce":3166305213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6803","nonce":3166305213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6807","nonce":3166305213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6805","nonce":3166305213}]},"public_addr":"172.21.15.146:6801/3166305213","cluster_addr":"172.21.15.146:6803/3166305213","heartbeat_back_addr":"172.21.15.146:6807/3166305213","heartbeat_front_addr":"172.21.15.146:6805/3166305213","state":["exists","up"]},{"osd":2,"uuid":"d4ef735b-9786-4cb9-83ef-7c3f0691a729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6801","nonce":1474319175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6803","nonce":1474319175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6807","nonce":1474319175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6805","nonce":1474319175}]},"public_addr":"172.21.15.152:6801/1474319175","cluster_addr":"172.21.15.152:6803/1474319175","heartbeat_back_addr":"172.21.15.152:6807/1474319175","heartbeat_front_addr":"172.21.15.152:6805/1474319175","state":["exists","up"]},{"osd":3,"uuid":"0ab85fb8-5bbb-4ffc-bddf-cae7644a64ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6811","nonce":1809071296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6813","nonce":1809071296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6817","nonce":1809071296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6815","nonce":1809071296}]},"public_addr":"172.21.15.69:6811/1809071296","cluster_addr":"172.21.15.69:6813/1809071296","heartbeat_back_addr":"172.21.15.69:6817/1809071296","heartbeat_front_addr":"172.21.15.69:6815/1809071296","state":["exists","up"]},{"osd":4,"uuid":"273e0592-bbbc-4188-8eeb-46a3879cf632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6809","nonce":1898520972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6811","nonce":1898520972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6815","nonce":1898520972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6813","nonce":1898520972}]},"public_addr":"172.21.15.146:6809/1898520972","cluster_addr":"172.21.15.146:6811/1898520972","heartbeat_back_addr":"172.21.15.146:6815/1898520972","heartbeat_front_addr":"172.21.15.146:6813/1898520972","state":["exists","up"]},{"osd":5,"uuid":"bbeae945-9fa2-48f9-a1d6-8bf18dcaef19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6809","nonce":1996555108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6811","nonce":1996555108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6815","nonce":1996555108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6813","nonce":1996555108}]},"public_addr":"172.21.15.152:6809/1996555108","cluster_addr":"172.21.15.152:6811/1996555108","heartbeat_back_addr":"172.21.15.152:6815/1996555108","heartbeat_front_addr":"172.21.15.152:6813/1996555108","state":["exists","up"]},{"osd":6,"uuid":"dbb61253-30bc-418e-bd75-118c80bbf931","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6819","nonce":491215762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6821","nonce":491215762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6825","nonce":491215762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6823","nonce":491215762}]},"public_addr":"172.21.15.69:6819/491215762","cluster_addr":"172.21.15.69:6821/491215762","heartbeat_back_addr":"172.21.15.69:6825/491215762","heartbeat_front_addr":"172.21.15.69:6823/491215762","state":["exists","up"]},{"osd":7,"uuid":"75f23854-e283-4e3e-94cf-7b4079d83876","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6817","nonce":2330294690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6819","nonce":2330294690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6823","nonce":2330294690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6821","nonce":2330294690}]},"public_addr":"172.21.15.146:6817/2330294690","cluster_addr":"172.21.15.146:6819/2330294690","heartbeat_back_addr":"172.21.15.146:6823/2330294690","heartbeat_front_addr":"172.21.15.146:6821/2330294690","state":["exists","up"]},{"osd":8,"uuid":"1aa1060c-0c14-4abd-be56-5ebfeb78759c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6817","nonce":950669381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6819","nonce":950669381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6823","nonce":950669381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6821","nonce":950669381}]},"public_addr":"172.21.15.152:6817/950669381","cluster_addr":"172.21.15.152:6819/950669381","heartbeat_back_addr":"172.21.15.152:6823/950669381","heartbeat_front_addr":"172.21.15.152:6821/950669381","state":["exists","up"]},{"osd":9,"uuid":"3f4bf800-4090-4138-9bb3-d001f13f450c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6827","nonce":2414186758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6829","nonce":2414186758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6833","nonce":2414186758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6831","nonce":2414186758}]},"public_addr":"172.21.15.69:6827/2414186758","cluster_addr":"172.21.15.69:6829/2414186758","heartbeat_back_addr":"172.21.15.69:6833/2414186758","heartbeat_front_addr":"172.21.15.69:6831/2414186758","state":["exists","up"]},{"osd":10,"uuid":"19e967e5-51ce-4506-80c9-0a95441baf99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6825","nonce":707256400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6827","nonce":707256400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6831","nonce":707256400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6829","nonce":707256400}]},"public_addr":"172.21.15.146:6825/707256400","cluster_addr":"172.21.15.146:6827/707256400","heartbeat_back_addr":"172.21.15.146:6831/707256400","heartbeat_front_addr":"172.21.15.146:6829/707256400","state":["exists","up"]},{"osd":11,"uuid":"7788c2de-6930-4808-be0f-f1fd1ae0cb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6825","nonce":2934336833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6827","nonce":2934336833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6831","nonce":2934336833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6829","nonce":2934336833}]},"public_addr":"172.21.15.152:6825/2934336833","cluster_addr":"172.21.15.152:6827/2934336833","heartbeat_back_addr":"172.21.15.152:6831/2934336833","heartbeat_front_addr":"172.21.15.152:6829/2934336833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:07.497448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:31.227494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:55.027978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:20.713437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:46.214704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:20:11.199983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:20:37.177070+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:02.951240+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:28.963680+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:56.095807+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:22:22.221744+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:22:48.929334+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/1372727546":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6801/1097124842":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6800/1508198215":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3103465990":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3540132960":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3899912190":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6800/1097124842":"2024-04-26T11:16:31.688159+0000","172.21.15.69:0/1199907202":"2024-04-26T11:16:04.860792+0000","172.21.15.69:6801/1508198215":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/2879102529":"2024-04-26T11:16:31.688159+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:23:47.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.341+0000 7f6faae6e640 1 -- 172.21.15.69:0/2747968591 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f6f900afaf0 msgr2=0x7f6f900b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:47.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.341+0000 7f6faae6e640 1 --2- 172.21.15.69:0/2747968591 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f6f900afaf0 0x7f6f900b2000 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4103600 tx=0x7f6f9c009500 comp rx=0 tx=0).stop 2024-04-25T11:23:47.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.341+0000 7f6faae6e640 1 -- 172.21.15.69:0/2747968591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fa4108940 msgr2=0x7f6fa41a0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:47.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.341+0000 7f6faae6e640 1 --2- 172.21.15.69:0/2747968591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fa4108940 0x7f6fa41a0520 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6f94009fd0 tx=0x7f6f9400b8e0 comp rx=0 tx=0).stop 2024-04-25T11:23:47.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.341+0000 7f6faae6e640 1 -- 172.21.15.69:0/2747968591 shutdown_connections 2024-04-25T11:23:47.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.341+0000 7f6faae6e640 1 --2- 172.21.15.69:0/2747968591 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f6f900afaf0 0x7f6f900b2000 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:47.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.341+0000 7f6faae6e640 1 --2- 172.21.15.69:0/2747968591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6fa4108940 0x7f6fa41a0520 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:47.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.341+0000 7f6faae6e640 1 --2- 172.21.15.69:0/2747968591 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fa4103c40 0x7f6fa4199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:47.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.341+0000 7f6faae6e640 1 --2- 172.21.15.69:0/2747968591 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6fa4102a00 0x7f6fa4199050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:47.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.341+0000 7f6faae6e640 1 -- 172.21.15.69:0/2747968591 >> 172.21.15.69:0/2747968591 conn(0x7f6fa40fdbf0 msgr2=0x7f6fa4104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:47.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.341+0000 7f6faae6e640 1 -- 172.21.15.69:0/2747968591 shutdown_connections 2024-04-25T11:23:47.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.341+0000 7f6faae6e640 1 -- 172.21.15.69:0/2747968591 wait complete. 2024-04-25T11:23:47.348 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:23:47.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.462+0000 7f23fe146640 1 -- 172.21.15.69:0/584067146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f23f81029c0 msgr2=0x7f23f81036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:47.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.462+0000 7f23fe146640 1 --2- 172.21.15.69:0/584067146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f23f81029c0 0x7f23f81036c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f23e8009900 tx=0x7f23e802a410 comp rx=0 tx=0).stop 2024-04-25T11:23:47.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.462+0000 7f23fe146640 1 -- 172.21.15.69:0/584067146 shutdown_connections 2024-04-25T11:23:47.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.462+0000 7f23fe146640 1 --2- 172.21.15.69:0/584067146 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23f8108900 0x7f23f810ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:47.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.462+0000 7f23fe146640 1 --2- 172.21.15.69:0/584067146 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23f8103c00 0x7f23f8104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:47.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.462+0000 7f23fe146640 1 --2- 172.21.15.69:0/584067146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f23f81029c0 0x7f23f81036c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:47.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.462+0000 7f23fe146640 1 -- 172.21.15.69:0/584067146 >> 172.21.15.69:0/584067146 conn(0x7f23f80fdbd0 msgr2=0x7f23f8100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:47.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.462+0000 7f23fe146640 1 -- 172.21.15.69:0/584067146 shutdown_connections 2024-04-25T11:23:47.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.462+0000 7f23fe146640 1 -- 172.21.15.69:0/584067146 wait complete. 2024-04-25T11:23:47.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.462+0000 7f23fe146640 1 Processor -- start 2024-04-25T11:23:47.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.463+0000 7f23fe146640 1 -- start start 2024-04-25T11:23:47.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.463+0000 7f23fe146640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23f81029c0 0x7f23f819a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:47.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.463+0000 7f23fe146640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23f8103c00 0x7f23f819a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:47.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.463+0000 7f23fe146640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f23f8108900 0x7f23f81a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:47.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.463+0000 7f23fe146640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f23f810f560 con 0x7f23f8108900 2024-04-25T11:23:47.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.463+0000 7f23fe146640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f23f810f3e0 con 0x7f23f81029c0 2024-04-25T11:23:47.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.463+0000 7f23fe146640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f23f810f6e0 con 0x7f23f8103c00 2024-04-25T11:23:47.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.463+0000 7f23f7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f23f8108900 0x7f23f81a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:47.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.463+0000 7f23f7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f23f8108900 0x7f23f81a1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48808/0 (socket says 172.21.15.69:48808) 2024-04-25T11:23:47.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.463+0000 7f23f7fff640 1 -- 172.21.15.69:0/3752998812 learned_addr learned my addr 172.21.15.69:0/3752998812 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:47.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.463+0000 7f23f77fe640 1 --2- 172.21.15.69:0/3752998812 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23f81029c0 0x7f23f819a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:47.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.463+0000 7f23effff640 1 --2- 172.21.15.69:0/3752998812 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23f8103c00 0x7f23f819a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:47.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.463+0000 7f23f7fff640 1 -- 172.21.15.69:0/3752998812 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23f8103c00 msgr2=0x7f23f819a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:47.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.463+0000 7f23f7fff640 1 --2- 172.21.15.69:0/3752998812 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23f8103c00 0x7f23f819a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:47.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.463+0000 7f23f7fff640 1 -- 172.21.15.69:0/3752998812 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23f81029c0 msgr2=0x7f23f819a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:47.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.463+0000 7f23f7fff640 1 --2- 172.21.15.69:0/3752998812 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23f81029c0 0x7f23f819a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:47.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.463+0000 7f23f7fff640 1 -- 172.21.15.69:0/3752998812 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23f81a1e00 con 0x7f23f8108900 2024-04-25T11:23:47.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.463+0000 7f23f7fff640 1 --2- 172.21.15.69:0/3752998812 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f23f8108900 0x7f23f81a1790 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f23f000b970 tx=0x7f23f000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:47.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.463+0000 7f23f57fa640 1 -- 172.21.15.69:0/3752998812 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f0019070 con 0x7f23f8108900 2024-04-25T11:23:47.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.464+0000 7f23f57fa640 1 -- 172.21.15.69:0/3752998812 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f23f0014440 con 0x7f23f8108900 2024-04-25T11:23:47.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.464+0000 7f23fe146640 1 -- 172.21.15.69:0/3752998812 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23f81a20f0 con 0x7f23f8108900 2024-04-25T11:23:47.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.464+0000 7f23f57fa640 1 -- 172.21.15.69:0/3752998812 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f00075d0 con 0x7f23f8108900 2024-04-25T11:23:47.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.464+0000 7f23fe146640 1 -- 172.21.15.69:0/3752998812 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23f81a2670 con 0x7f23f8108900 2024-04-25T11:23:47.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.465+0000 7f23fe146640 1 -- 172.21.15.69:0/3752998812 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23f8107710 con 0x7f23f8108900 2024-04-25T11:23:47.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.467+0000 7f23f57fa640 1 -- 172.21.15.69:0/3752998812 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f23f001c070 con 0x7f23f8108900 2024-04-25T11:23:47.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.467+0000 7f23f57fa640 1 --2- 172.21.15.69:0/3752998812 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f23e40afaf0 0x7f23e40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:47.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.468+0000 7f23f77fe640 1 --2- 172.21.15.69:0/3752998812 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f23e40afaf0 0x7f23e40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:47.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.468+0000 7f23f57fa640 1 -- 172.21.15.69:0/3752998812 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8802+0+0 (secure 0 0 0) 0x7f23f00de8d0 con 0x7f23f8108900 2024-04-25T11:23:47.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.468+0000 7f23f77fe640 1 --2- 172.21.15.69:0/3752998812 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f23e40afaf0 0x7f23e40b2000 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f23e8005760 tx=0x7f23e8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:47.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.471+0000 7f23f57fa640 1 -- 172.21.15.69:0/3752998812 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f23f00a6bc0 con 0x7f23f8108900 2024-04-25T11:23:47.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.648+0000 7f23fe146640 1 -- 172.21.15.69:0/3752998812 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f23f80637a0 con 0x7f23f8108900 2024-04-25T11:23:47.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.650+0000 7f23f57fa640 1 -- 172.21.15.69:0/3752998812 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+24143 (secure 0 0 0) 0x7f23f00af290 con 0x7f23f8108900 2024-04-25T11:23:47.644 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:23:47.644 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":79,"fsid":"f7ed73dc-02f4-11ef-bc93-c7b262605968","created":"2024-04-25T11:15:37.965281+0000","modified":"2024-04-25T11:23:46.017458+0000","last_up_change":"2024-04-25T11:22:50.691869+0000","last_in_change":"2024-04-25T11:22:30.271959+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:18:57.781263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:23:38.685110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:23:39.943803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T11:23:43.650521+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6a0d179-4998-4754-bf82-2ea3dac54e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6803","nonce":3756025235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6805","nonce":3756025235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6809","nonce":3756025235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6807","nonce":3756025235}]},"public_addr":"172.21.15.69:6803/3756025235","cluster_addr":"172.21.15.69:6805/3756025235","heartbeat_back_addr":"172.21.15.69:6809/3756025235","heartbeat_front_addr":"172.21.15.69:6807/3756025235","state":["exists","up"]},{"osd":1,"uuid":"c0c66930-4233-493f-b2bb-99c473994b59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6801","nonce":3166305213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6803","nonce":3166305213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6807","nonce":3166305213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6805","nonce":3166305213}]},"public_addr":"172.21.15.146:6801/3166305213","cluster_addr":"172.21.15.146:6803/3166305213","heartbeat_back_addr":"172.21.15.146:6807/3166305213","heartbeat_front_addr":"172.21.15.146:6805/3166305213","state":["exists","up"]},{"osd":2,"uuid":"d4ef735b-9786-4cb9-83ef-7c3f0691a729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6801","nonce":1474319175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6803","nonce":1474319175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6807","nonce":1474319175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6805","nonce":1474319175}]},"public_addr":"172.21.15.152:6801/1474319175","cluster_addr":"172.21.15.152:6803/1474319175","heartbeat_back_addr":"172.21.15.152:6807/1474319175","heartbeat_front_addr":"172.21.15.152:6805/1474319175","state":["exists","up"]},{"osd":3,"uuid":"0ab85fb8-5bbb-4ffc-bddf-cae7644a64ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6811","nonce":1809071296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6813","nonce":1809071296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6817","nonce":1809071296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6815","nonce":1809071296}]},"public_addr":"172.21.15.69:6811/1809071296","cluster_addr":"172.21.15.69:6813/1809071296","heartbeat_back_addr":"172.21.15.69:6817/1809071296","heartbeat_front_addr":"172.21.15.69:6815/1809071296","state":["exists","up"]},{"osd":4,"uuid":"273e0592-bbbc-4188-8eeb-46a3879cf632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6809","nonce":1898520972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6811","nonce":1898520972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6815","nonce":1898520972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6813","nonce":1898520972}]},"public_addr":"172.21.15.146:6809/1898520972","cluster_addr":"172.21.15.146:6811/1898520972","heartbeat_back_addr":"172.21.15.146:6815/1898520972","heartbeat_front_addr":"172.21.15.146:6813/1898520972","state":["exists","up"]},{"osd":5,"uuid":"bbeae945-9fa2-48f9-a1d6-8bf18dcaef19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6809","nonce":1996555108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6811","nonce":1996555108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6815","nonce":1996555108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6813","nonce":1996555108}]},"public_addr":"172.21.15.152:6809/1996555108","cluster_addr":"172.21.15.152:6811/1996555108","heartbeat_back_addr":"172.21.15.152:6815/1996555108","heartbeat_front_addr":"172.21.15.152:6813/1996555108","state":["exists","up"]},{"osd":6,"uuid":"dbb61253-30bc-418e-bd75-118c80bbf931","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6819","nonce":491215762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6821","nonce":491215762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6825","nonce":491215762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6823","nonce":491215762}]},"public_addr":"172.21.15.69:6819/491215762","cluster_addr":"172.21.15.69:6821/491215762","heartbeat_back_addr":"172.21.15.69:6825/491215762","heartbeat_front_addr":"172.21.15.69:6823/491215762","state":["exists","up"]},{"osd":7,"uuid":"75f23854-e283-4e3e-94cf-7b4079d83876","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6817","nonce":2330294690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6819","nonce":2330294690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6823","nonce":2330294690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6821","nonce":2330294690}]},"public_addr":"172.21.15.146:6817/2330294690","cluster_addr":"172.21.15.146:6819/2330294690","heartbeat_back_addr":"172.21.15.146:6823/2330294690","heartbeat_front_addr":"172.21.15.146:6821/2330294690","state":["exists","up"]},{"osd":8,"uuid":"1aa1060c-0c14-4abd-be56-5ebfeb78759c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6817","nonce":950669381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6819","nonce":950669381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6823","nonce":950669381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6821","nonce":950669381}]},"public_addr":"172.21.15.152:6817/950669381","cluster_addr":"172.21.15.152:6819/950669381","heartbeat_back_addr":"172.21.15.152:6823/950669381","heartbeat_front_addr":"172.21.15.152:6821/950669381","state":["exists","up"]},{"osd":9,"uuid":"3f4bf800-4090-4138-9bb3-d001f13f450c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6827","nonce":2414186758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6829","nonce":2414186758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6833","nonce":2414186758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6831","nonce":2414186758}]},"public_addr":"172.21.15.69:6827/2414186758","cluster_addr":"172.21.15.69:6829/2414186758","heartbeat_back_addr":"172.21.15.69:6833/2414186758","heartbeat_front_addr":"172.21.15.69:6831/2414186758","state":["exists","up"]},{"osd":10,"uuid":"19e967e5-51ce-4506-80c9-0a95441baf99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6825","nonce":707256400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6827","nonce":707256400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6831","nonce":707256400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6829","nonce":707256400}]},"public_addr":"172.21.15.146:6825/707256400","cluster_addr":"172.21.15.146:6827/707256400","heartbeat_back_addr":"172.21.15.146:6831/707256400","heartbeat_front_addr":"172.21.15.146:6829/707256400","state":["exists","up"]},{"osd":11,"uuid":"7788c2de-6930-4808-be0f-f1fd1ae0cb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6825","nonce":2934336833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6827","nonce":2934336833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6831","nonce":2934336833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6829","nonce":2934336833}]},"public_addr":"172.21.15.152:6825/2934336833","cluster_addr":"172.21.15.152:6827/2934336833","heartbeat_back_addr":"172.21.15.152:6831/2934336833","heartbeat_front_addr":"172.21.15.152:6829/2934336833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:07.497448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:31.227494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:55.027978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:20.713437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:46.214704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:20:11.199983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:20:37.177070+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:02.951240+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:28.963680+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:56.095807+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:22:22.221744+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:22:48.929334+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/1372727546":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6801/1097124842":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6800/1508198215":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3103465990":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3540132960":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3899912190":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6800/1097124842":"2024-04-26T11:16:31.688159+0000","172.21.15.69:0/1199907202":"2024-04-26T11:16:04.860792+0000","172.21.15.69:6801/1508198215":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/2879102529":"2024-04-26T11:16:31.688159+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:23:47.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.652+0000 7f23fe146640 1 -- 172.21.15.69:0/3752998812 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f23e40afaf0 msgr2=0x7f23e40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:47.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.652+0000 7f23fe146640 1 --2- 172.21.15.69:0/3752998812 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f23e40afaf0 0x7f23e40b2000 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f23e8005760 tx=0x7f23e8000bc0 comp rx=0 tx=0).stop 2024-04-25T11:23:47.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.652+0000 7f23fe146640 1 -- 172.21.15.69:0/3752998812 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f23f8108900 msgr2=0x7f23f81a1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:47.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.652+0000 7f23fe146640 1 --2- 172.21.15.69:0/3752998812 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f23f8108900 0x7f23f81a1790 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f23f000b970 tx=0x7f23f000be30 comp rx=0 tx=0).stop 2024-04-25T11:23:47.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.652+0000 7f23fe146640 1 -- 172.21.15.69:0/3752998812 shutdown_connections 2024-04-25T11:23:47.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.652+0000 7f23fe146640 1 --2- 172.21.15.69:0/3752998812 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f23e40afaf0 0x7f23e40b2000 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:47.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.652+0000 7f23fe146640 1 --2- 172.21.15.69:0/3752998812 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f23f8108900 0x7f23f81a1790 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:47.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.652+0000 7f23fe146640 1 --2- 172.21.15.69:0/3752998812 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23f8103c00 0x7f23f819a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:47.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.652+0000 7f23fe146640 1 --2- 172.21.15.69:0/3752998812 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23f81029c0 0x7f23f819a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:47.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.652+0000 7f23fe146640 1 -- 172.21.15.69:0/3752998812 >> 172.21.15.69:0/3752998812 conn(0x7f23f80fdbd0 msgr2=0x7f23f81048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:47.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.652+0000 7f23fe146640 1 -- 172.21.15.69:0/3752998812 shutdown_connections 2024-04-25T11:23:47.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.652+0000 7f23fe146640 1 -- 172.21.15.69:0/3752998812 wait complete. 2024-04-25T11:23:47.673 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:23:47.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.787+0000 7f5f1999d640 1 -- 172.21.15.69:0/2796840451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f141040a0 msgr2=0x7f5f141064c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:47.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.787+0000 7f5f1999d640 1 --2- 172.21.15.69:0/2796840451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f141040a0 0x7f5f141064c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5efc009900 tx=0x7f5efc02a410 comp rx=0 tx=0).stop 2024-04-25T11:23:47.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.787+0000 7f5f1999d640 1 -- 172.21.15.69:0/2796840451 shutdown_connections 2024-04-25T11:23:47.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.787+0000 7f5f1999d640 1 --2- 172.21.15.69:0/2796840451 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f141095c0 0x7f5f1410bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:47.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.787+0000 7f5f1999d640 1 --2- 172.21.15.69:0/2796840451 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f14106a90 0x7f5f14108f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:47.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.787+0000 7f5f1999d640 1 --2- 172.21.15.69:0/2796840451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f141040a0 0x7f5f141064c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:47.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.787+0000 7f5f1999d640 1 -- 172.21.15.69:0/2796840451 >> 172.21.15.69:0/2796840451 conn(0x7f5f140fda40 msgr2=0x7f5f140ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:47.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.787+0000 7f5f1999d640 1 -- 172.21.15.69:0/2796840451 shutdown_connections 2024-04-25T11:23:47.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.787+0000 7f5f1999d640 1 -- 172.21.15.69:0/2796840451 wait complete. 2024-04-25T11:23:47.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.787+0000 7f5f1999d640 1 Processor -- start 2024-04-25T11:23:47.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.787+0000 7f5f1999d640 1 -- start start 2024-04-25T11:23:47.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.787+0000 7f5f1999d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f141040a0 0x7f5f1419a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:47.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.787+0000 7f5f1999d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f14106a90 0x7f5f1419a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:47.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.787+0000 7f5f1999d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f141095c0 0x7f5f141a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:47.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.787+0000 7f5f1999d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1410f3b0 con 0x7f5f141040a0 2024-04-25T11:23:47.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.787+0000 7f5f1999d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1410f230 con 0x7f5f14106a90 2024-04-25T11:23:47.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.788+0000 7f5f1999d640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1410f530 con 0x7f5f141095c0 2024-04-25T11:23:47.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.788+0000 7f5f1899b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f141040a0 0x7f5f1419a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:47.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.788+0000 7f5f1899b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f141040a0 0x7f5f1419a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48824/0 (socket says 172.21.15.69:48824) 2024-04-25T11:23:47.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.788+0000 7f5f1899b640 1 -- 172.21.15.69:0/3213367751 learned_addr learned my addr 172.21.15.69:0/3213367751 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:47.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.788+0000 7f5f13fff640 1 --2- 172.21.15.69:0/3213367751 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f14106a90 0x7f5f1419a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:47.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.788+0000 7f5f1919c640 1 --2- 172.21.15.69:0/3213367751 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f141095c0 0x7f5f141a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:47.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.788+0000 7f5f1899b640 1 -- 172.21.15.69:0/3213367751 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f141095c0 msgr2=0x7f5f141a1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:47.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.788+0000 7f5f1899b640 1 --2- 172.21.15.69:0/3213367751 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f141095c0 0x7f5f141a1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:47.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.788+0000 7f5f1899b640 1 -- 172.21.15.69:0/3213367751 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f14106a90 msgr2=0x7f5f1419a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:47.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.788+0000 7f5f1899b640 1 --2- 172.21.15.69:0/3213367751 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f14106a90 0x7f5f1419a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:47.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.788+0000 7f5f1899b640 1 -- 172.21.15.69:0/3213367751 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f141a1be0 con 0x7f5f141040a0 2024-04-25T11:23:47.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.788+0000 7f5f1919c640 1 --2- 172.21.15.69:0/3213367751 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f141095c0 0x7f5f141a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:23:47.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.788+0000 7f5f13fff640 1 --2- 172.21.15.69:0/3213367751 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f14106a90 0x7f5f1419a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:23:47.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.788+0000 7f5f1899b640 1 --2- 172.21.15.69:0/3213367751 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f141040a0 0x7f5f1419a0a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5efc031460 tx=0x7f5efc0042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:47.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.788+0000 7f5f11ffb640 1 -- 172.21.15.69:0/3213367751 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5efc03f030 con 0x7f5f141040a0 2024-04-25T11:23:47.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.788+0000 7f5f11ffb640 1 -- 172.21.15.69:0/3213367751 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5efc0044d0 con 0x7f5f141040a0 2024-04-25T11:23:47.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.788+0000 7f5f1999d640 1 -- 172.21.15.69:0/3213367751 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f141a1e70 con 0x7f5f141040a0 2024-04-25T11:23:47.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.788+0000 7f5f11ffb640 1 -- 172.21.15.69:0/3213367751 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5efc035070 con 0x7f5f141040a0 2024-04-25T11:23:47.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.788+0000 7f5f1999d640 1 -- 172.21.15.69:0/3213367751 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f141a2300 con 0x7f5f141040a0 2024-04-25T11:23:47.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.790+0000 7f5f1999d640 1 -- 172.21.15.69:0/3213367751 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f140ff560 con 0x7f5f141040a0 2024-04-25T11:23:47.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.791+0000 7f5f11ffb640 1 -- 172.21.15.69:0/3213367751 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f5efc042030 con 0x7f5f141040a0 2024-04-25T11:23:47.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.791+0000 7f5f11ffb640 1 --2- 172.21.15.69:0/3213367751 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f5f000afae0 0x7f5f000b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:47.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.791+0000 7f5f13fff640 1 --2- 172.21.15.69:0/3213367751 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f5f000afae0 0x7f5f000b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:47.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.791+0000 7f5f11ffb640 1 -- 172.21.15.69:0/3213367751 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7f5efc0ff540 con 0x7f5f141040a0 2024-04-25T11:23:47.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.792+0000 7f5f13fff640 1 --2- 172.21.15.69:0/3213367751 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f5f000afae0 0x7f5f000b1ff0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5f1419b140 tx=0x7f5f040073e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:47.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:47.796+0000 7f5f11ffb640 1 -- 172.21.15.69:0/3213367751 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5efc0c7ee0 con 0x7f5f141040a0 2024-04-25T11:23:47.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:47 smithi069 ceph-mon[29188]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T11:23:47.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:47 smithi069 ceph-mon[29188]: osdmap e79: 12 total, 12 up, 12 in 2024-04-25T11:23:47.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:47 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/1579774619' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-25T11:23:47.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:47 smithi069 ceph-mon[29188]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T11:23:47.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:47 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/3513710922' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:47.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:47 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2747968591' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:23:47.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:47 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/3752998812' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:23:47.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:47 smithi152 ceph-mon[32701]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T11:23:47.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:47 smithi152 ceph-mon[32701]: osdmap e79: 12 total, 12 up, 12 in 2024-04-25T11:23:47.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:47 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/1579774619' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-25T11:23:47.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:47 smithi152 ceph-mon[32701]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T11:23:47.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:47 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/3513710922' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:47.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:47 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/2747968591' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:23:47.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:47 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/3752998812' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:23:48.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.014+0000 7f5f1999d640 1 -- 172.21.15.69:0/3213367751 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f140637a0 con 0x7f5f141040a0 2024-04-25T11:23:48.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.015+0000 7f5f11ffb640 1 -- 172.21.15.69:0/3213367751 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11809 (secure 0 0 0) 0x7f5efc0c77e0 con 0x7f5f141040a0 2024-04-25T11:23:48.009 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:23:48.009 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3399424094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3399424094},{"type":"v1","addr":"172.21.15.69:6835","nonce":3399424094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1463217771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1463217771},{"type":"v1","addr":"172.21.15.69:6837","nonce":1463217771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/966316001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":966316001},{"type":"v1","addr":"172.21.15.69:6839","nonce":966316001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/2060225181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":2060225181},{"type":"v1","addr":"172.21.15.69:6841","nonce":2060225181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/4197482473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":4197482473},{"type":"v1","addr":"172.21.15.152:6833","nonce":4197482473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/4267885323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4267885323},{"type":"v1","addr":"172.21.15.146:6833","nonce":4267885323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24439,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/3188843362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3188843362},{"type":"v1","addr":"172.21.15.152:6835","nonce":3188843362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/563274982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":563274982},{"type":"v1","addr":"172.21.15.152:6837","nonce":563274982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1876533699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1876533699},{"type":"v1","addr":"172.21.15.146:6835","nonce":1876533699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24475,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6839/3008615657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":3008615657},{"type":"v1","addr":"172.21.15.152:6839","nonce":3008615657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/2730317134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2730317134},{"type":"v1","addr":"172.21.15.146:6837","nonce":2730317134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:23:40.984930+0000","modified":"2024-04-25T11:23:46.017687+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24515},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6839/2013283448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":2013283448},{"type":"v1","addr":"172.21.15.146:6839","nonce":2013283448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24515,"qdb_cluster":[24515]},"id":1}]} 2024-04-25T11:23:48.009 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 17 2024-04-25T11:23:48.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.017+0000 7f5f1999d640 1 -- 172.21.15.69:0/3213367751 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f5f000afae0 msgr2=0x7f5f000b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:48.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.017+0000 7f5f1999d640 1 --2- 172.21.15.69:0/3213367751 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f5f000afae0 0x7f5f000b1ff0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5f1419b140 tx=0x7f5f040073e0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.017+0000 7f5f1999d640 1 -- 172.21.15.69:0/3213367751 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f141040a0 msgr2=0x7f5f1419a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:48.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.017+0000 7f5f1999d640 1 --2- 172.21.15.69:0/3213367751 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f141040a0 0x7f5f1419a0a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5efc031460 tx=0x7f5efc0042e0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.017+0000 7f5f1999d640 1 -- 172.21.15.69:0/3213367751 shutdown_connections 2024-04-25T11:23:48.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.017+0000 7f5f1999d640 1 --2- 172.21.15.69:0/3213367751 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f5f000afae0 0x7f5f000b1ff0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.017+0000 7f5f1999d640 1 --2- 172.21.15.69:0/3213367751 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f141095c0 0x7f5f141a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.017+0000 7f5f1999d640 1 --2- 172.21.15.69:0/3213367751 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f14106a90 0x7f5f1419a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.017+0000 7f5f1999d640 1 --2- 172.21.15.69:0/3213367751 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f141040a0 0x7f5f1419a0a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.017+0000 7f5f1999d640 1 -- 172.21.15.69:0/3213367751 >> 172.21.15.69:0/3213367751 conn(0x7f5f140fda40 msgr2=0x7f5f14106090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:48.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.017+0000 7f5f1999d640 1 -- 172.21.15.69:0/3213367751 shutdown_connections 2024-04-25T11:23:48.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.017+0000 7f5f1999d640 1 -- 172.21.15.69:0/3213367751 wait complete. 2024-04-25T11:23:48.023 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:23:48.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:47 smithi146 ceph-mon[32821]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T11:23:48.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:47 smithi146 ceph-mon[32821]: osdmap e79: 12 total, 12 up, 12 in 2024-04-25T11:23:48.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:47 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/1579774619' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-25T11:23:48.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:47 smithi146 ceph-mon[32821]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T11:23:48.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:47 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/3513710922' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:48.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:47 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/2747968591' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:23:48.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:47 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/3752998812' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:23:48.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.136+0000 7f128d642640 1 -- 172.21.15.69:0/2847661908 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1288108900 msgr2=0x7f128810ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:48.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.136+0000 7f128d642640 1 --2- 172.21.15.69:0/2847661908 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1288108900 0x7f128810ad20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1280009ca0 tx=0x7f128002a8c0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.136+0000 7f128d642640 1 -- 172.21.15.69:0/2847661908 shutdown_connections 2024-04-25T11:23:48.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.136+0000 7f128d642640 1 --2- 172.21.15.69:0/2847661908 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1288108900 0x7f128810ad20 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.136+0000 7f128d642640 1 --2- 172.21.15.69:0/2847661908 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1288103c00 0x7f1288104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.136+0000 7f128d642640 1 --2- 172.21.15.69:0/2847661908 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f12881029c0 0x7f12881036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.136+0000 7f128d642640 1 -- 172.21.15.69:0/2847661908 >> 172.21.15.69:0/2847661908 conn(0x7f12880fdbd0 msgr2=0x7f1288100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:48.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.136+0000 7f128d642640 1 -- 172.21.15.69:0/2847661908 shutdown_connections 2024-04-25T11:23:48.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.136+0000 7f128d642640 1 -- 172.21.15.69:0/2847661908 wait complete. 2024-04-25T11:23:48.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.137+0000 7f128d642640 1 Processor -- start 2024-04-25T11:23:48.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.137+0000 7f128d642640 1 -- start start 2024-04-25T11:23:48.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.137+0000 7f128d642640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f12881029c0 0x7f128819a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:48.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.137+0000 7f128d642640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1288103c00 0x7f128819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:48.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.137+0000 7f128d642640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1288108900 0x7f12881a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:48.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.137+0000 7f128d642640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f128806a830 con 0x7f12881029c0 2024-04-25T11:23:48.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.137+0000 7f128d642640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f128806a6b0 con 0x7f1288103c00 2024-04-25T11:23:48.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.137+0000 7f128d642640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f128806a9b0 con 0x7f1288108900 2024-04-25T11:23:48.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.137+0000 7f1286ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f12881029c0 0x7f128819a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:48.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.137+0000 7f1286ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f12881029c0 0x7f128819a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48844/0 (socket says 172.21.15.69:48844) 2024-04-25T11:23:48.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.137+0000 7f1286ffd640 1 -- 172.21.15.69:0/1018855968 learned_addr learned my addr 172.21.15.69:0/1018855968 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:48.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.137+0000 7f127ffff640 1 --2- 172.21.15.69:0/1018855968 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1288103c00 0x7f128819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:48.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.138+0000 7f12877fe640 1 --2- 172.21.15.69:0/1018855968 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1288108900 0x7f12881a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:48.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.138+0000 7f1286ffd640 1 -- 172.21.15.69:0/1018855968 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1288108900 msgr2=0x7f12881a15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:48.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.138+0000 7f1286ffd640 1 --2- 172.21.15.69:0/1018855968 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1288108900 0x7f12881a15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.138+0000 7f1286ffd640 1 -- 172.21.15.69:0/1018855968 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1288103c00 msgr2=0x7f128819a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:48.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.138+0000 7f1286ffd640 1 --2- 172.21.15.69:0/1018855968 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1288103c00 0x7f128819a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.138+0000 7f1286ffd640 1 -- 172.21.15.69:0/1018855968 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12881a1c50 con 0x7f12881029c0 2024-04-25T11:23:48.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.138+0000 7f1286ffd640 1 --2- 172.21.15.69:0/1018855968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f12881029c0 0x7f128819a180 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f127800d740 tx=0x7f127800db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:48.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.138+0000 7f1284ff9640 1 -- 172.21.15.69:0/1018855968 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1278017070 con 0x7f12881029c0 2024-04-25T11:23:48.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.138+0000 7f1284ff9640 1 -- 172.21.15.69:0/1018855968 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1278009cb0 con 0x7f12881029c0 2024-04-25T11:23:48.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.138+0000 7f128d642640 1 -- 172.21.15.69:0/1018855968 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12881a1f40 con 0x7f12881029c0 2024-04-25T11:23:48.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.138+0000 7f1284ff9640 1 -- 172.21.15.69:0/1018855968 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1278003bb0 con 0x7f12881029c0 2024-04-25T11:23:48.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.138+0000 7f128d642640 1 -- 172.21.15.69:0/1018855968 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12881a23a0 con 0x7f12881029c0 2024-04-25T11:23:48.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.139+0000 7f128d642640 1 -- 172.21.15.69:0/1018855968 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f128806afd0 con 0x7f12881029c0 2024-04-25T11:23:48.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.140+0000 7f1284ff9640 1 -- 172.21.15.69:0/1018855968 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f1278009820 con 0x7f12881029c0 2024-04-25T11:23:48.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.141+0000 7f1284ff9640 1 --2- 172.21.15.69:0/1018855968 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f12740afaf0 0x7f12740b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:48.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.141+0000 7f127ffff640 1 --2- 172.21.15.69:0/1018855968 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f12740afaf0 0x7f12740b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:48.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.141+0000 7f1284ff9640 1 -- 172.21.15.69:0/1018855968 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7f12780deba0 con 0x7f12881029c0 2024-04-25T11:23:48.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.142+0000 7f127ffff640 1 --2- 172.21.15.69:0/1018855968 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f12740afaf0 0x7f12740b2000 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f128819b220 tx=0x7f1270009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:48.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.145+0000 7f1284ff9640 1 -- 172.21.15.69:0/1018855968 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f128806afd0 con 0x7f12881029c0 2024-04-25T11:23:48.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.321+0000 7f128d642640 1 -- 172.21.15.69:0/1018855968 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f128802dcb0 con 0x7f12881029c0 2024-04-25T11:23:48.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.323+0000 7f1284ff9640 1 -- 172.21.15.69:0/1018855968 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+24130 (secure 0 0 0) 0x7f128802dcb0 con 0x7f12881029c0 2024-04-25T11:23:48.317 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:23:48.317 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":80,"fsid":"f7ed73dc-02f4-11ef-bc93-c7b262605968","created":"2024-04-25T11:15:37.965281+0000","modified":"2024-04-25T11:23:47.662955+0000","last_up_change":"2024-04-25T11:22:50.691869+0000","last_in_change":"2024-04-25T11:22:30.271959+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:18:57.781263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:23:38.685110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:23:39.943803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T11:23:43.650521+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6a0d179-4998-4754-bf82-2ea3dac54e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6803","nonce":3756025235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6805","nonce":3756025235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6809","nonce":3756025235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6807","nonce":3756025235}]},"public_addr":"172.21.15.69:6803/3756025235","cluster_addr":"172.21.15.69:6805/3756025235","heartbeat_back_addr":"172.21.15.69:6809/3756025235","heartbeat_front_addr":"172.21.15.69:6807/3756025235","state":["exists","up"]},{"osd":1,"uuid":"c0c66930-4233-493f-b2bb-99c473994b59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6801","nonce":3166305213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6803","nonce":3166305213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6807","nonce":3166305213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6805","nonce":3166305213}]},"public_addr":"172.21.15.146:6801/3166305213","cluster_addr":"172.21.15.146:6803/3166305213","heartbeat_back_addr":"172.21.15.146:6807/3166305213","heartbeat_front_addr":"172.21.15.146:6805/3166305213","state":["exists","up"]},{"osd":2,"uuid":"d4ef735b-9786-4cb9-83ef-7c3f0691a729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6801","nonce":1474319175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6803","nonce":1474319175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6807","nonce":1474319175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6805","nonce":1474319175}]},"public_addr":"172.21.15.152:6801/1474319175","cluster_addr":"172.21.15.152:6803/1474319175","heartbeat_back_addr":"172.21.15.152:6807/1474319175","heartbeat_front_addr":"172.21.15.152:6805/1474319175","state":["exists","up"]},{"osd":3,"uuid":"0ab85fb8-5bbb-4ffc-bddf-cae7644a64ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6811","nonce":1809071296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6813","nonce":1809071296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6817","nonce":1809071296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6815","nonce":1809071296}]},"public_addr":"172.21.15.69:6811/1809071296","cluster_addr":"172.21.15.69:6813/1809071296","heartbeat_back_addr":"172.21.15.69:6817/1809071296","heartbeat_front_addr":"172.21.15.69:6815/1809071296","state":["exists","up"]},{"osd":4,"uuid":"273e0592-bbbc-4188-8eeb-46a3879cf632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6809","nonce":1898520972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6811","nonce":1898520972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6815","nonce":1898520972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6813","nonce":1898520972}]},"public_addr":"172.21.15.146:6809/1898520972","cluster_addr":"172.21.15.146:6811/1898520972","heartbeat_back_addr":"172.21.15.146:6815/1898520972","heartbeat_front_addr":"172.21.15.146:6813/1898520972","state":["exists","up"]},{"osd":5,"uuid":"bbeae945-9fa2-48f9-a1d6-8bf18dcaef19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6809","nonce":1996555108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6811","nonce":1996555108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6815","nonce":1996555108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6813","nonce":1996555108}]},"public_addr":"172.21.15.152:6809/1996555108","cluster_addr":"172.21.15.152:6811/1996555108","heartbeat_back_addr":"172.21.15.152:6815/1996555108","heartbeat_front_addr":"172.21.15.152:6813/1996555108","state":["exists","up"]},{"osd":6,"uuid":"dbb61253-30bc-418e-bd75-118c80bbf931","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6819","nonce":491215762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6821","nonce":491215762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6825","nonce":491215762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6823","nonce":491215762}]},"public_addr":"172.21.15.69:6819/491215762","cluster_addr":"172.21.15.69:6821/491215762","heartbeat_back_addr":"172.21.15.69:6825/491215762","heartbeat_front_addr":"172.21.15.69:6823/491215762","state":["exists","up"]},{"osd":7,"uuid":"75f23854-e283-4e3e-94cf-7b4079d83876","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6817","nonce":2330294690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6819","nonce":2330294690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6823","nonce":2330294690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6821","nonce":2330294690}]},"public_addr":"172.21.15.146:6817/2330294690","cluster_addr":"172.21.15.146:6819/2330294690","heartbeat_back_addr":"172.21.15.146:6823/2330294690","heartbeat_front_addr":"172.21.15.146:6821/2330294690","state":["exists","up"]},{"osd":8,"uuid":"1aa1060c-0c14-4abd-be56-5ebfeb78759c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6817","nonce":950669381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6819","nonce":950669381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6823","nonce":950669381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6821","nonce":950669381}]},"public_addr":"172.21.15.152:6817/950669381","cluster_addr":"172.21.15.152:6819/950669381","heartbeat_back_addr":"172.21.15.152:6823/950669381","heartbeat_front_addr":"172.21.15.152:6821/950669381","state":["exists","up"]},{"osd":9,"uuid":"3f4bf800-4090-4138-9bb3-d001f13f450c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6827","nonce":2414186758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6829","nonce":2414186758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6833","nonce":2414186758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6831","nonce":2414186758}]},"public_addr":"172.21.15.69:6827/2414186758","cluster_addr":"172.21.15.69:6829/2414186758","heartbeat_back_addr":"172.21.15.69:6833/2414186758","heartbeat_front_addr":"172.21.15.69:6831/2414186758","state":["exists","up"]},{"osd":10,"uuid":"19e967e5-51ce-4506-80c9-0a95441baf99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6825","nonce":707256400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6827","nonce":707256400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6831","nonce":707256400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6829","nonce":707256400}]},"public_addr":"172.21.15.146:6825/707256400","cluster_addr":"172.21.15.146:6827/707256400","heartbeat_back_addr":"172.21.15.146:6831/707256400","heartbeat_front_addr":"172.21.15.146:6829/707256400","state":["exists","up"]},{"osd":11,"uuid":"7788c2de-6930-4808-be0f-f1fd1ae0cb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6825","nonce":2934336833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6827","nonce":2934336833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6831","nonce":2934336833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6829","nonce":2934336833}]},"public_addr":"172.21.15.152:6825/2934336833","cluster_addr":"172.21.15.152:6827/2934336833","heartbeat_back_addr":"172.21.15.152:6831/2934336833","heartbeat_front_addr":"172.21.15.152:6829/2934336833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:07.497448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:31.227494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:55.027978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:20.713437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:46.214704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:20:11.199983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:20:37.177070+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:02.951240+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:28.963680+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:56.095807+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:22:22.221744+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:22:48.929334+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/1372727546":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6801/1097124842":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6800/1508198215":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3103465990":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3540132960":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3899912190":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6800/1097124842":"2024-04-26T11:16:31.688159+0000","172.21.15.69:0/1199907202":"2024-04-26T11:16:04.860792+0000","172.21.15.69:6801/1508198215":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/2879102529":"2024-04-26T11:16:31.688159+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:23:48.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.325+0000 7f128d642640 1 -- 172.21.15.69:0/1018855968 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f12740afaf0 msgr2=0x7f12740b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:48.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.325+0000 7f128d642640 1 --2- 172.21.15.69:0/1018855968 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f12740afaf0 0x7f12740b2000 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f128819b220 tx=0x7f1270009110 comp rx=0 tx=0).stop 2024-04-25T11:23:48.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.325+0000 7f128d642640 1 -- 172.21.15.69:0/1018855968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f12881029c0 msgr2=0x7f128819a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:48.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.325+0000 7f128d642640 1 --2- 172.21.15.69:0/1018855968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f12881029c0 0x7f128819a180 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f127800d740 tx=0x7f127800db40 comp rx=0 tx=0).stop 2024-04-25T11:23:48.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.325+0000 7f128d642640 1 -- 172.21.15.69:0/1018855968 shutdown_connections 2024-04-25T11:23:48.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.325+0000 7f128d642640 1 --2- 172.21.15.69:0/1018855968 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f12740afaf0 0x7f12740b2000 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.325+0000 7f128d642640 1 --2- 172.21.15.69:0/1018855968 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1288108900 0x7f12881a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.325+0000 7f128d642640 1 --2- 172.21.15.69:0/1018855968 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1288103c00 0x7f128819a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.325+0000 7f128d642640 1 --2- 172.21.15.69:0/1018855968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f12881029c0 0x7f128819a180 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.325+0000 7f128d642640 1 -- 172.21.15.69:0/1018855968 >> 172.21.15.69:0/1018855968 conn(0x7f12880fdbd0 msgr2=0x7f12881048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:48.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.325+0000 7f128d642640 1 -- 172.21.15.69:0/1018855968 shutdown_connections 2024-04-25T11:23:48.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.325+0000 7f128d642640 1 -- 172.21.15.69:0/1018855968 wait complete. 2024-04-25T11:23:48.331 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:23:40.984930+0000', 'modified': '2024-04-25T11:23:46.017687+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24515}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24515': {'gid': 24515, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.146:6839/2013283448', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6838', 'nonce': 2013283448}, {'type': 'v1', 'addr': '172.21.15.146:6839', 'nonce': 2013283448}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24515, 'qdb_cluster': [24515]} 2024-04-25T11:23:48.331 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T11:23:48.331 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T11:23:48.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.446+0000 7fee7bf01640 1 -- 172.21.15.69:0/203112368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee741050b0 msgr2=0x7fee741054c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:48.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.446+0000 7fee7bf01640 1 --2- 172.21.15.69:0/203112368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee741050b0 0x7fee741054c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fee64009900 tx=0x7fee6402a3f0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.446+0000 7fee7bf01640 1 -- 172.21.15.69:0/203112368 shutdown_connections 2024-04-25T11:23:48.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.446+0000 7fee7bf01640 1 --2- 172.21.15.69:0/203112368 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee74102260 0x7fee7410bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.446+0000 7fee7bf01640 1 --2- 172.21.15.69:0/203112368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee74101760 0x7fee74101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.446+0000 7fee7bf01640 1 --2- 172.21.15.69:0/203112368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee741050b0 0x7fee741054c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.446+0000 7fee7bf01640 1 -- 172.21.15.69:0/203112368 >> 172.21.15.69:0/203112368 conn(0x7fee740fd380 msgr2=0x7fee740ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:48.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.446+0000 7fee7bf01640 1 -- 172.21.15.69:0/203112368 shutdown_connections 2024-04-25T11:23:48.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.446+0000 7fee7bf01640 1 -- 172.21.15.69:0/203112368 wait complete. 2024-04-25T11:23:48.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.447+0000 7fee7bf01640 1 Processor -- start 2024-04-25T11:23:48.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.447+0000 7fee7bf01640 1 -- start start 2024-04-25T11:23:48.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.447+0000 7fee7bf01640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee74101760 0x7fee7419a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:48.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.447+0000 7fee7bf01640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee74102260 0x7fee7419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:48.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.447+0000 7fee7bf01640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee741050b0 0x7fee741a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:48.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.447+0000 7fee7bf01640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fee7410f4b0 con 0x7fee74101760 2024-04-25T11:23:48.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.447+0000 7fee7bf01640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fee7410f330 con 0x7fee741050b0 2024-04-25T11:23:48.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.447+0000 7fee7bf01640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fee7410f630 con 0x7fee74102260 2024-04-25T11:23:48.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.447+0000 7fee79c9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee74101760 0x7fee7419a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:48.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.447+0000 7fee79c9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee74101760 0x7fee7419a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48874/0 (socket says 172.21.15.69:48874) 2024-04-25T11:23:48.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.447+0000 7fee79c9d640 1 -- 172.21.15.69:0/996407201 learned_addr learned my addr 172.21.15.69:0/996407201 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:48.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.447+0000 7fee7949c640 1 --2- 172.21.15.69:0/996407201 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee74102260 0x7fee7419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:48.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.447+0000 7fee7a49e640 1 --2- 172.21.15.69:0/996407201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee741050b0 0x7fee741a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:48.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.447+0000 7fee79c9d640 1 -- 172.21.15.69:0/996407201 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee74102260 msgr2=0x7fee7419a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:48.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.447+0000 7fee79c9d640 1 --2- 172.21.15.69:0/996407201 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee74102260 0x7fee7419a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.447+0000 7fee79c9d640 1 -- 172.21.15.69:0/996407201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee741050b0 msgr2=0x7fee741a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:48.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.447+0000 7fee79c9d640 1 --2- 172.21.15.69:0/996407201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee741050b0 0x7fee741a15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.447+0000 7fee79c9d640 1 -- 172.21.15.69:0/996407201 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee741a1b70 con 0x7fee74101760 2024-04-25T11:23:48.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.447+0000 7fee79c9d640 1 --2- 172.21.15.69:0/996407201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee74101760 0x7fee7419a0e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fee64005ab0 tx=0x7fee64004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:48.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.448+0000 7fee6affd640 1 -- 172.21.15.69:0/996407201 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6403f070 con 0x7fee74101760 2024-04-25T11:23:48.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.448+0000 7fee6affd640 1 -- 172.21.15.69:0/996407201 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee64003c80 con 0x7fee74101760 2024-04-25T11:23:48.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.448+0000 7fee7bf01640 1 -- 172.21.15.69:0/996407201 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee741a1e00 con 0x7fee74101760 2024-04-25T11:23:48.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.448+0000 7fee6affd640 1 -- 172.21.15.69:0/996407201 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee64035070 con 0x7fee74101760 2024-04-25T11:23:48.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.448+0000 7fee7bf01640 1 -- 172.21.15.69:0/996407201 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee741a2200 con 0x7fee74101760 2024-04-25T11:23:48.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.450+0000 7fee6affd640 1 -- 172.21.15.69:0/996407201 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fee6403b580 con 0x7fee74101760 2024-04-25T11:23:48.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.450+0000 7fee82db0740 1 Processor -- start 2024-04-25T11:23:48.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.451+0000 7fee82db0740 1 -- start start 2024-04-25T11:23:48.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.451+0000 7fee82db0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbdd845270 0x55bbddbe6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:48.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.451+0000 7fee82db0740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bbdd81c3b0 0x55bbddbe68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:48.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.451+0000 7fee82db0740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bbdd824590 0x55bbddbed7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:48.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.451+0000 7fee82db0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bbddb4b740 con 0x55bbdd845270 2024-04-25T11:23:48.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.451+0000 7fee82db0740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55bbddae73a0 con 0x55bbdd81c3b0 2024-04-25T11:23:48.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.451+0000 7fee82db0740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55bbddb47700 con 0x55bbdd824590 2024-04-25T11:23:48.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.451+0000 7fee7a49e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbdd845270 0x55bbddbe6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:48.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.451+0000 7fee7a49e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbdd845270 0x55bbddbe6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48878/0 (socket says 172.21.15.69:48878) 2024-04-25T11:23:48.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.451+0000 7fee7a49e640 1 -- 172.21.15.69:0/1206170170 learned_addr learned my addr 172.21.15.69:0/1206170170 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:48.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.451+0000 7fee6affd640 1 --2- 172.21.15.69:0/996407201 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fee600afaf0 0x7fee600b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:48.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.451+0000 7fee7949c640 1 --2- 172.21.15.69:0/1206170170 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bbdd824590 0x55bbddbed7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:48.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.451+0000 7fee79c9d640 1 --2- 172.21.15.69:0/1206170170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bbdd81c3b0 0x55bbddbe68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:48.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.451+0000 7fee7a49e640 1 -- 172.21.15.69:0/1206170170 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bbdd824590 msgr2=0x55bbddbed7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:48.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.451+0000 7fee7a49e640 1 --2- 172.21.15.69:0/1206170170 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bbdd824590 0x55bbddbed7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.451+0000 7fee7a49e640 1 -- 172.21.15.69:0/1206170170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bbdd81c3b0 msgr2=0x55bbddbe68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:48.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.451+0000 7fee7a49e640 1 --2- 172.21.15.69:0/1206170170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bbdd81c3b0 0x55bbddbe68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.451+0000 7fee7a49e640 1 -- 172.21.15.69:0/1206170170 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbddb03750 con 0x55bbdd845270 2024-04-25T11:23:48.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.451+0000 7fee79c9d640 1 --2- 172.21.15.69:0/1206170170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bbdd81c3b0 0x55bbddbe68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:23:48.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.451+0000 7fee6affd640 1 -- 172.21.15.69:0/996407201 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7fee640ff4e0 con 0x7fee74101760 2024-04-25T11:23:48.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.451+0000 7fee7a49e640 1 --2- 172.21.15.69:0/996407201 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fee600afaf0 0x7fee600b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:48.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.451+0000 7fee7a49e640 1 --2- 172.21.15.69:0/1206170170 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbdd845270 0x55bbddbe6370 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fee6c007fd0 tx=0x7fee6c00e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4385d5a701311b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:48.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.452+0000 7fee5ffff640 1 -- 172.21.15.69:0/1206170170 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6c022070 con 0x55bbdd845270 2024-04-25T11:23:48.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.452+0000 7fee5ffff640 1 -- 172.21.15.69:0/1206170170 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee6c011c30 con 0x55bbdd845270 2024-04-25T11:23:48.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.452+0000 7fee5ffff640 1 -- 172.21.15.69:0/1206170170 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6c01cda0 con 0x55bbdd845270 2024-04-25T11:23:48.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.452+0000 7fee82db0740 1 -- 172.21.15.69:0/1206170170 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbdd845270 msgr2=0x55bbddbe6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:48.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.452+0000 7fee82db0740 1 --2- 172.21.15.69:0/1206170170 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbdd845270 0x55bbddbe6370 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fee6c007fd0 tx=0x7fee6c00e7e0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.452+0000 7fee7a49e640 1 --2- 172.21.15.69:0/996407201 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fee600afaf0 0x7fee600b2000 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fee6c01f000 tx=0x7fee6c020000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:48.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.453+0000 7fee82db0740 1 -- 172.21.15.69:0/1206170170 shutdown_connections 2024-04-25T11:23:48.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.453+0000 7fee82db0740 1 --2- 172.21.15.69:0/1206170170 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bbdd824590 0x55bbddbed7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.453+0000 7fee82db0740 1 --2- 172.21.15.69:0/1206170170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bbdd81c3b0 0x55bbddbe68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.453+0000 7fee82db0740 1 --2- 172.21.15.69:0/1206170170 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbdd845270 0x55bbddbe6370 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.453+0000 7fee82db0740 1 -- 172.21.15.69:0/1206170170 >> 172.21.15.69:0/1206170170 conn(0x55bbdd8393c0 msgr2=0x55bbddbe1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:48.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.453+0000 7fee82db0740 1 -- 172.21.15.69:0/1206170170 shutdown_connections 2024-04-25T11:23:48.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.453+0000 7fee82db0740 1 -- 172.21.15.69:0/1206170170 wait complete. 2024-04-25T11:23:48.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.453+0000 7fee82db0740 1 -- start start 2024-04-25T11:23:48.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.453+0000 7fee82db0740 1 Processor -- start 2024-04-25T11:23:48.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.453+0000 7fee82db0740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bbdd81c3b0 0x55bbddbf02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:48.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.454+0000 7fee82db0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbdd824590 0x55bbddbf0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:48.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.454+0000 7fee82db0740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bbdd845270 0x55bbddbf7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:48.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.454+0000 7fee82db0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bbddbee240 con 0x55bbdd824590 2024-04-25T11:23:48.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.454+0000 7fee82db0740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55bbddad2fe0 con 0x55bbdd845270 2024-04-25T11:23:48.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.454+0000 7fee82db0740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55bbddbee3c0 con 0x55bbdd81c3b0 2024-04-25T11:23:48.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.454+0000 7fee82db0740 10 client.0 _ll_register_callbacks cb 0x55bbddae9150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:23:48.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.454+0000 7fee7949c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbdd824590 0x55bbddbf0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:48.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.454+0000 7fee7949c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbdd824590 0x55bbddbf0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48892/0 (socket says 172.21.15.69:48892) 2024-04-25T11:23:48.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.454+0000 7fee7949c640 1 -- 172.21.15.69:0/2412189119 learned_addr learned my addr 172.21.15.69:0/2412189119 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:48.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.454+0000 7fee7a49e640 1 --2- 172.21.15.69:0/2412189119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bbdd845270 0x55bbddbf7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:48.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.454+0000 7fee79c9d640 1 --2- 172.21.15.69:0/2412189119 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bbdd81c3b0 0x55bbddbf02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:48.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.454+0000 7fee7949c640 1 -- 172.21.15.69:0/2412189119 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bbdd81c3b0 msgr2=0x55bbddbf02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:48.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.454+0000 7fee7949c640 1 --2- 172.21.15.69:0/2412189119 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bbdd81c3b0 0x55bbddbf02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.454+0000 7fee7949c640 1 -- 172.21.15.69:0/2412189119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bbdd845270 msgr2=0x55bbddbf7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:48.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.454+0000 7fee7949c640 1 --2- 172.21.15.69:0/2412189119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bbdd845270 0x55bbddbf7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.454+0000 7fee7949c640 1 -- 172.21.15.69:0/2412189119 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbdd85d2d0 con 0x55bbdd824590 2024-04-25T11:23:48.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.454+0000 7fee7a49e640 1 --2- 172.21.15.69:0/2412189119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bbdd845270 0x55bbddbf7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:23:48.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.454+0000 7fee7949c640 1 --2- 172.21.15.69:0/2412189119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbdd824590 0x55bbddbf0810 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fee70012810 tx=0x7fee70012cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7271c4b54cc4deb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:48.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.454+0000 7fee5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-25T11:23:48.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.454+0000 7fee5d7fa640 1 -- 172.21.15.69:0/2412189119 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7001f060 con 0x55bbdd824590 2024-04-25T11:23:48.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.454+0000 7fee5d7fa640 1 -- 172.21.15.69:0/2412189119 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7fee70010dc0 con 0x55bbdd824590 2024-04-25T11:23:48.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.454+0000 7fee82db0740 1 -- 172.21.15.69:0/2412189119 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee6c011c30 con 0x55bbdd824590 2024-04-25T11:23:48.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.454+0000 7fee5d7fa640 1 -- 172.21.15.69:0/2412189119 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fee70026050 con 0x55bbdd824590 2024-04-25T11:23:48.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.454+0000 7fee82db0740 10 client.14751 fetch_fsmap learned FSMap version 17 2024-04-25T11:23:48.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.454+0000 7fee82db0740 1 -- 172.21.15.69:0/2412189119 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55bbdd9276b0 con 0x55bbdd824590 2024-04-25T11:23:48.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.455+0000 7fee5d7fa640 1 -- 172.21.15.69:0/2412189119 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee6c011c30 con 0x55bbdd824590 2024-04-25T11:23:48.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.455+0000 7fee5d7fa640 1 -- 172.21.15.69:0/2412189119 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7fee7000d3d0 con 0x55bbdd824590 2024-04-25T11:23:48.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.455+0000 7fee82db0740 10 client.14751 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T11:23:48.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.455+0000 7fee82db0740 10 client.14751 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T11:23:48.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.455+0000 7fee82db0740 20 client.14751 populate_metadata read hostname 'smithi069' 2024-04-25T11:23:48.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.455+0000 7fee82db0740 1 --2- 172.21.15.69:0/2412189119 >> [v2:172.21.15.146:6838/2013283448,v1:172.21.15.146:6839/2013283448] conn(0x55bbdd803e40 0x55bbddbfe060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:48.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.455+0000 7fee82db0740 4 client.14751 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:23:48.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.455+0000 7fee82db0740 1 -- 172.21.15.69:0/2412189119 --> [v2:172.21.15.146:6838/2013283448,v1:172.21.15.146:6839/2013283448] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbddb59350 con 0x55bbdd803e40 2024-04-25T11:23:48.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.455+0000 7fee79c9d640 1 --2- 172.21.15.69:0/2412189119 >> [v2:172.21.15.146:6838/2013283448,v1:172.21.15.146:6839/2013283448] conn(0x55bbdd803e40 0x55bbddbfe060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:48.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.456+0000 7fee5d7fa640 10 client.14751 ms_handle_connect on v2:172.21.15.146:6838/2013283448 2024-04-25T11:23:48.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.456+0000 7fee79c9d640 1 --2- 172.21.15.69:0/2412189119 >> [v2:172.21.15.146:6838/2013283448,v1:172.21.15.146:6839/2013283448] conn(0x55bbdd803e40 0x55bbddbfe060 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e6d75cf6f064d server_cookie=9f31321530dbe98d in_seq=0 out_seq=0 2024-04-25T11:23:48.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.456+0000 7fee5d7fa640 1 -- 172.21.15.69:0/2412189119 <== mds.0 v2:172.21.15.146:6838/2013283448 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bbddb59350 con 0x55bbdd803e40 2024-04-25T11:23:48.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.456+0000 7fee5d7fa640 10 client.14751 handle_command_reply: tid=0 2024-04-25T11:23:48.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.456+0000 7fee82db0740 1 client.14751 shutdown 2024-04-25T11:23:48.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.456+0000 7fee82db0740 1 -- 172.21.15.69:0/2412189119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbdd824590 msgr2=0x55bbddbf0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:48.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.456+0000 7fee82db0740 1 --2- 172.21.15.69:0/2412189119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbdd824590 0x55bbddbf0810 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fee70012810 tx=0x7fee70012cd0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.457+0000 7fee82db0740 1 -- 172.21.15.69:0/2412189119 shutdown_connections 2024-04-25T11:23:48.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.457+0000 7fee82db0740 1 --2- 172.21.15.69:0/2412189119 >> [v2:172.21.15.146:6838/2013283448,v1:172.21.15.146:6839/2013283448] conn(0x55bbdd803e40 0x55bbddbfe060 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.457+0000 7fee82db0740 1 --2- 172.21.15.69:0/2412189119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bbdd845270 0x55bbddbf7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.457+0000 7fee82db0740 1 --2- 172.21.15.69:0/2412189119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbdd824590 0x55bbddbf0810 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.457+0000 7fee82db0740 1 --2- 172.21.15.69:0/2412189119 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bbdd81c3b0 0x55bbddbf02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.457+0000 7fee5d7fa640 0 client.14751 ms_handle_reset on v2:172.21.15.146:6838/2013283448 2024-04-25T11:23:48.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.457+0000 7fee82db0740 1 -- 172.21.15.69:0/2412189119 >> 172.21.15.69:0/2412189119 conn(0x55bbdd8393c0 msgr2=0x55bbddbe0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:48.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.457+0000 7fee82db0740 1 -- 172.21.15.69:0/2412189119 shutdown_connections 2024-04-25T11:23:48.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.457+0000 7fee82db0740 1 -- 172.21.15.69:0/2412189119 wait complete. 2024-04-25T11:23:48.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.457+0000 7fee82db0740 20 client.14751 trim_cache size 0 max 16384 2024-04-25T11:23:48.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.475+0000 7fee82db0740 1 Processor -- start 2024-04-25T11:23:48.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.475+0000 7fee82db0740 1 -- start start 2024-04-25T11:23:48.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.475+0000 7fee82db0740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bbdd81c3b0 0x55bbddbead30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:48.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.475+0000 7fee82db0740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bbdd824590 0x55bbddbe9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:48.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.475+0000 7fee82db0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbdd845270 0x55bbddbe1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:48.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.475+0000 7fee82db0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bbddbfc3a0 con 0x55bbdd845270 2024-04-25T11:23:48.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.475+0000 7fee82db0740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55bbddbfca70 con 0x55bbdd81c3b0 2024-04-25T11:23:48.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.475+0000 7fee82db0740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55bbddbfd140 con 0x55bbdd824590 2024-04-25T11:23:48.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.475+0000 7fee7a49e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbdd845270 0x55bbddbe1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:48.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.475+0000 7fee7a49e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbdd845270 0x55bbddbe1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48906/0 (socket says 172.21.15.69:48906) 2024-04-25T11:23:48.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.475+0000 7fee7a49e640 1 -- 172.21.15.69:0/3407161025 learned_addr learned my addr 172.21.15.69:0/3407161025 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:48.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.475+0000 7fee79c9d640 1 --2- 172.21.15.69:0/3407161025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bbdd81c3b0 0x55bbddbead30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:48.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.476+0000 7fee7a49e640 1 -- 172.21.15.69:0/3407161025 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bbdd824590 msgr2=0x55bbddbe9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:48.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.476+0000 7fee7a49e640 1 --2- 172.21.15.69:0/3407161025 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bbdd824590 0x55bbddbe9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.476+0000 7fee7a49e640 1 -- 172.21.15.69:0/3407161025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bbdd81c3b0 msgr2=0x55bbddbead30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:48.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.476+0000 7fee7a49e640 1 --2- 172.21.15.69:0/3407161025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bbdd81c3b0 0x55bbddbead30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.476+0000 7fee7a49e640 1 -- 172.21.15.69:0/3407161025 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbddbe7570 con 0x55bbdd845270 2024-04-25T11:23:48.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.476+0000 7fee7a49e640 1 --2- 172.21.15.69:0/3407161025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbdd845270 0x55bbddbe1dc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fee6c007510 tx=0x7fee6c012450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80fb220e4b0751e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:48.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.476+0000 7fee5f7fe640 1 -- 172.21.15.69:0/3407161025 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6c035070 con 0x55bbdd845270 2024-04-25T11:23:48.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.476+0000 7fee5f7fe640 1 -- 172.21.15.69:0/3407161025 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee6c007ad0 con 0x55bbdd845270 2024-04-25T11:23:48.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.476+0000 7fee5f7fe640 1 -- 172.21.15.69:0/3407161025 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6c022040 con 0x55bbdd845270 2024-04-25T11:23:48.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.477+0000 7fee82db0740 1 -- 172.21.15.69:0/3407161025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbdd845270 msgr2=0x55bbddbe1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:48.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.477+0000 7fee82db0740 1 --2- 172.21.15.69:0/3407161025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbdd845270 0x55bbddbe1dc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fee6c007510 tx=0x7fee6c012450 comp rx=0 tx=0).stop 2024-04-25T11:23:48.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.477+0000 7fee82db0740 1 -- 172.21.15.69:0/3407161025 shutdown_connections 2024-04-25T11:23:48.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.477+0000 7fee82db0740 1 --2- 172.21.15.69:0/3407161025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbdd845270 0x55bbddbe1dc0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.477+0000 7fee82db0740 1 --2- 172.21.15.69:0/3407161025 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bbdd824590 0x55bbddbe9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.477+0000 7fee82db0740 1 --2- 172.21.15.69:0/3407161025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bbdd81c3b0 0x55bbddbead30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.477+0000 7fee82db0740 1 -- 172.21.15.69:0/3407161025 >> 172.21.15.69:0/3407161025 conn(0x55bbdd8393c0 msgr2=0x55bbddbe16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:48.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.477+0000 7fee82db0740 1 -- 172.21.15.69:0/3407161025 shutdown_connections 2024-04-25T11:23:48.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.477+0000 7fee82db0740 1 -- 172.21.15.69:0/3407161025 wait complete. 2024-04-25T11:23:48.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.477+0000 7fee82db0740 1 -- start start 2024-04-25T11:23:48.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.477+0000 7fee82db0740 1 Processor -- start 2024-04-25T11:23:48.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.477+0000 7fee82db0740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bbdd81c3b0 0x55bbddc63150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:48.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.477+0000 7fee82db0740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bbdd824590 0x55bbddc636d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:48.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.477+0000 7fee82db0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbdd845270 0x55bbddc69f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:48.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.477+0000 7fee82db0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bbddbfdee0 con 0x55bbdd845270 2024-04-25T11:23:48.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.477+0000 7fee82db0740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55bbddbfd810 con 0x55bbdd824590 2024-04-25T11:23:48.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.477+0000 7fee82db0740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55bbddbfe5b0 con 0x55bbdd81c3b0 2024-04-25T11:23:48.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.477+0000 7fee82db0740 10 client.0 _ll_register_callbacks cb 0x7fee7000a6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:23:48.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.478+0000 7fee7a49e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbdd845270 0x55bbddc69f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:48.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.478+0000 7fee7a49e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbdd845270 0x55bbddc69f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48910/0 (socket says 172.21.15.69:48910) 2024-04-25T11:23:48.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.478+0000 7fee7a49e640 1 -- 172.21.15.69:0/234855514 learned_addr learned my addr 172.21.15.69:0/234855514 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:48.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.478+0000 7fee7949c640 1 --2- 172.21.15.69:0/234855514 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bbdd824590 0x55bbddc636d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:48.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.478+0000 7fee79c9d640 1 --2- 172.21.15.69:0/234855514 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bbdd81c3b0 0x55bbddc63150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:48.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.478+0000 7fee7a49e640 1 -- 172.21.15.69:0/234855514 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bbdd81c3b0 msgr2=0x55bbddc63150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:48.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.478+0000 7fee7a49e640 1 --2- 172.21.15.69:0/234855514 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bbdd81c3b0 0x55bbddc63150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.478+0000 7fee7a49e640 1 -- 172.21.15.69:0/234855514 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bbdd824590 msgr2=0x55bbddc636d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:48.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.478+0000 7fee7a49e640 1 --2- 172.21.15.69:0/234855514 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bbdd824590 0x55bbddc636d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.478+0000 7fee7a49e640 1 -- 172.21.15.69:0/234855514 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbddc6a5a0 con 0x55bbdd845270 2024-04-25T11:23:48.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.478+0000 7fee7a49e640 1 --2- 172.21.15.69:0/234855514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbdd845270 0x55bbddc69f90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fee6c007660 tx=0x7fee6c01c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6564502ba43f8a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:48.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.478+0000 7fee5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-25T11:23:48.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.478+0000 7fee5d7fa640 1 -- 172.21.15.69:0/234855514 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6c03e060 con 0x55bbdd845270 2024-04-25T11:23:48.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.478+0000 7fee5d7fa640 1 -- 172.21.15.69:0/234855514 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7fee6c01cb20 con 0x55bbdd845270 2024-04-25T11:23:48.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.478+0000 7fee82db0740 1 -- 172.21.15.69:0/234855514 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee6c007ad0 con 0x55bbdd845270 2024-04-25T11:23:48.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.478+0000 7fee5d7fa640 1 -- 172.21.15.69:0/234855514 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fee6c045050 con 0x55bbdd845270 2024-04-25T11:23:48.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.478+0000 7fee82db0740 10 client.14757 fetch_fsmap learned FSMap version 17 2024-04-25T11:23:48.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.478+0000 7fee82db0740 1 -- 172.21.15.69:0/234855514 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55bbddc6b7c0 con 0x55bbdd845270 2024-04-25T11:23:48.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.478+0000 7fee5d7fa640 1 -- 172.21.15.69:0/234855514 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee6c007ad0 con 0x55bbdd845270 2024-04-25T11:23:48.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.479+0000 7fee5d7fa640 1 -- 172.21.15.69:0/234855514 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7fee6c02f070 con 0x55bbdd845270 2024-04-25T11:23:48.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.479+0000 7fee82db0740 10 client.14757 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T11:23:48.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.479+0000 7fee82db0740 10 client.14757 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T11:23:48.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.479+0000 7fee82db0740 20 client.14757 populate_metadata read hostname 'smithi069' 2024-04-25T11:23:48.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.479+0000 7fee82db0740 1 --2- 172.21.15.69:0/234855514 >> [v2:172.21.15.146:6838/2013283448,v1:172.21.15.146:6839/2013283448] conn(0x55bbdd7d9280 0x55bbddc6e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:48.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.479+0000 7fee82db0740 4 client.14757 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:23:48.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.479+0000 7fee82db0740 1 -- 172.21.15.69:0/234855514 --> [v2:172.21.15.146:6838/2013283448,v1:172.21.15.146:6839/2013283448] -- command(tid 0: {"prefix": "status"}) -- 0x55bbddb531c0 con 0x55bbdd7d9280 2024-04-25T11:23:48.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.479+0000 7fee79c9d640 1 --2- 172.21.15.69:0/234855514 >> [v2:172.21.15.146:6838/2013283448,v1:172.21.15.146:6839/2013283448] conn(0x55bbdd7d9280 0x55bbddc6e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:48.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.479+0000 7fee5d7fa640 10 client.14757 ms_handle_connect on v2:172.21.15.146:6838/2013283448 2024-04-25T11:23:48.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.479+0000 7fee79c9d640 1 --2- 172.21.15.69:0/234855514 >> [v2:172.21.15.146:6838/2013283448,v1:172.21.15.146:6839/2013283448] conn(0x55bbdd7d9280 0x55bbddc6e9a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=664a2a9b053f52f9 server_cookie=26849810fc2552b9 in_seq=0 out_seq=0 2024-04-25T11:23:48.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.480+0000 7fee5d7fa640 1 -- 172.21.15.69:0/234855514 <== mds.0 v2:172.21.15.146:6838/2013283448 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55bbddb531c0 con 0x55bbdd7d9280 2024-04-25T11:23:48.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.480+0000 7fee5d7fa640 10 client.14757 handle_command_reply: tid=0 2024-04-25T11:23:48.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.480+0000 7fee82db0740 1 client.14757 shutdown 2024-04-25T11:23:48.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.480+0000 7fee82db0740 1 -- 172.21.15.69:0/234855514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbdd845270 msgr2=0x55bbddc69f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:48.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.480+0000 7fee82db0740 1 --2- 172.21.15.69:0/234855514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbdd845270 0x55bbddc69f90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fee6c007660 tx=0x7fee6c01c720 comp rx=0 tx=0).stop 2024-04-25T11:23:48.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.480+0000 7fee82db0740 1 -- 172.21.15.69:0/234855514 shutdown_connections 2024-04-25T11:23:48.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.480+0000 7fee82db0740 1 --2- 172.21.15.69:0/234855514 >> [v2:172.21.15.146:6838/2013283448,v1:172.21.15.146:6839/2013283448] conn(0x55bbdd7d9280 0x55bbddc6e9a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.480+0000 7fee82db0740 1 --2- 172.21.15.69:0/234855514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bbdd845270 0x55bbddc69f90 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.480+0000 7fee82db0740 1 --2- 172.21.15.69:0/234855514 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bbdd824590 0x55bbddc636d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.480+0000 7fee5d7fa640 0 client.14757 ms_handle_reset on v2:172.21.15.146:6838/2013283448 2024-04-25T11:23:48.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.480+0000 7fee82db0740 1 --2- 172.21.15.69:0/234855514 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bbdd81c3b0 0x55bbddc63150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.480+0000 7fee82db0740 1 -- 172.21.15.69:0/234855514 >> 172.21.15.69:0/234855514 conn(0x55bbdd8393c0 msgr2=0x55bbddbe0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:48.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.480+0000 7fee82db0740 1 -- 172.21.15.69:0/234855514 shutdown_connections 2024-04-25T11:23:48.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.480+0000 7fee82db0740 1 -- 172.21.15.69:0/234855514 wait complete. 2024-04-25T11:23:48.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.480+0000 7fee82db0740 20 client.14757 trim_cache size 0 max 16384 2024-04-25T11:23:48.481 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-25T11:23:48.481 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "f7ed73dc-02f4-11ef-bc93-c7b262605968", 2024-04-25T11:23:48.481 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 0, 2024-04-25T11:23:48.481 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 24515, 2024-04-25T11:23:48.481 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-04-25T11:23:48.482 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-04-25T11:23:48.482 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-04-25T11:23:48.482 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 6.8610464569999996, 2024-04-25T11:23:48.482 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 17, 2024-04-25T11:23:48.482 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 80, 2024-04-25T11:23:48.482 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 79, 2024-04-25T11:23:48.482 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 13.802127199999999 2024-04-25T11:23:48.482 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-25T11:23:48.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.480+0000 7fee7bf01640 1 -- 172.21.15.69:0/996407201 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fee600afaf0 msgr2=0x7fee600b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:48.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.480+0000 7fee7bf01640 1 --2- 172.21.15.69:0/996407201 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fee600afaf0 0x7fee600b2000 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fee6c01f000 tx=0x7fee6c020000 comp rx=0 tx=0).stop 2024-04-25T11:23:48.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.480+0000 7fee7bf01640 1 -- 172.21.15.69:0/996407201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee74101760 msgr2=0x7fee7419a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:48.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.480+0000 7fee7bf01640 1 --2- 172.21.15.69:0/996407201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee74101760 0x7fee7419a0e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fee64005ab0 tx=0x7fee64004270 comp rx=0 tx=0).stop 2024-04-25T11:23:48.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.480+0000 7fee7bf01640 1 -- 172.21.15.69:0/996407201 shutdown_connections 2024-04-25T11:23:48.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.480+0000 7fee7bf01640 1 --2- 172.21.15.69:0/996407201 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fee600afaf0 0x7fee600b2000 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.480+0000 7fee7bf01640 1 --2- 172.21.15.69:0/996407201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee741050b0 0x7fee741a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.480+0000 7fee7bf01640 1 --2- 172.21.15.69:0/996407201 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee74102260 0x7fee7419a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.480+0000 7fee7bf01640 1 --2- 172.21.15.69:0/996407201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee74101760 0x7fee7419a0e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:48.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.480+0000 7fee7bf01640 1 -- 172.21.15.69:0/996407201 >> 172.21.15.69:0/996407201 conn(0x7fee740fd380 msgr2=0x7fee74105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:48.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.481+0000 7fee7bf01640 1 -- 172.21.15.69:0/996407201 shutdown_connections 2024-04-25T11:23:48.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:48.481+0000 7fee7bf01640 1 -- 172.21.15.69:0/996407201 wait complete. 2024-04-25T11:23:48.486 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T11:23:48.486 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T11:23:48.486 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-25T11:23:48.724 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:23:48.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:48 smithi069 ceph-mon[29188]: osdmap e80: 12 total, 12 up, 12 in 2024-04-25T11:23:48.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:48 smithi069 ceph-mon[29188]: pgmap v288: 129 pgs: 1 creating+activating, 3 creating+peering, 25 unknown, 100 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 9 op/s 2024-04-25T11:23:48.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:48 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/3213367751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:48.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:48 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/1018855968' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:23:48.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:48 smithi152 ceph-mon[32701]: osdmap e80: 12 total, 12 up, 12 in 2024-04-25T11:23:48.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:48 smithi152 ceph-mon[32701]: pgmap v288: 129 pgs: 1 creating+activating, 3 creating+peering, 25 unknown, 100 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 9 op/s 2024-04-25T11:23:48.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:48 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/3213367751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:48.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:48 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/1018855968' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:23:49.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:48 smithi146 ceph-mon[32821]: osdmap e80: 12 total, 12 up, 12 in 2024-04-25T11:23:49.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:48 smithi146 ceph-mon[32821]: pgmap v288: 129 pgs: 1 creating+activating, 3 creating+peering, 25 unknown, 100 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 9 op/s 2024-04-25T11:23:49.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:48 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/3213367751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:23:49.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:48 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/1018855968' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:23:49.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:49 smithi069 ceph-mon[29188]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T11:23:49.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:49 smithi069 ceph-mon[29188]: Cluster is now healthy 2024-04-25T11:23:49.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:49 smithi152 ceph-mon[32701]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T11:23:49.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:49 smithi152 ceph-mon[32701]: Cluster is now healthy 2024-04-25T11:23:50.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:49 smithi146 ceph-mon[32821]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T11:23:50.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:49 smithi146 ceph-mon[32821]: Cluster is now healthy 2024-04-25T11:23:50.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.560+0000 7f989b173700 1 -- 172.21.15.69:0/294780337 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9894105d00 msgr2=0x7f989410e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:50.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.560+0000 7f989b173700 1 --2- 172.21.15.69:0/294780337 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9894105d00 0x7f989410e250 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9880009990 tx=0x7f9880009ca0 comp rx=0 tx=0).stop 2024-04-25T11:23:50.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.560+0000 7f989b173700 1 -- 172.21.15.69:0/294780337 shutdown_connections 2024-04-25T11:23:50.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.560+0000 7f989b173700 1 --2- 172.21.15.69:0/294780337 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f989410e7e0 0x7f9894110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:50.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.560+0000 7f989b173700 1 --2- 172.21.15.69:0/294780337 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98941053b0 0x7f98941057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:50.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.560+0000 7f989b173700 1 --2- 172.21.15.69:0/294780337 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9894105d00 0x7f989410e250 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:50.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.560+0000 7f989b173700 1 -- 172.21.15.69:0/294780337 >> 172.21.15.69:0/294780337 conn(0x7f98940fec90 msgr2=0x7f98941010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:50.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.560+0000 7f989b173700 1 -- 172.21.15.69:0/294780337 shutdown_connections 2024-04-25T11:23:50.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.560+0000 7f989b173700 1 -- 172.21.15.69:0/294780337 wait complete. 2024-04-25T11:23:50.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.561+0000 7f989b173700 1 Processor -- start 2024-04-25T11:23:50.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.561+0000 7f989b173700 1 -- start start 2024-04-25T11:23:50.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.561+0000 7f989b173700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98941053b0 0x7f989419ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:50.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.561+0000 7f989b173700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9894105d00 0x7f989419b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:50.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.561+0000 7f989b173700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f989410e7e0 0x7f989419f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:50.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.561+0000 7f989b173700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f98941142b0 con 0x7f989410e7e0 2024-04-25T11:23:50.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.561+0000 7f989b173700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9894114130 con 0x7f98941053b0 2024-04-25T11:23:50.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.561+0000 7f989b173700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9894114430 con 0x7f9894105d00 2024-04-25T11:23:50.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.561+0000 7f9899710700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f989410e7e0 0x7f989419f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:50.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.561+0000 7f9899710700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f989410e7e0 0x7f989419f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48924/0 (socket says 172.21.15.69:48924) 2024-04-25T11:23:50.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.561+0000 7f9899710700 1 -- 172.21.15.69:0/1487725758 learned_addr learned my addr 172.21.15.69:0/1487725758 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:50.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.562+0000 7f9893fff700 1 --2- 172.21.15.69:0/1487725758 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9894105d00 0x7f989419b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:50.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.562+0000 7f9898f0f700 1 --2- 172.21.15.69:0/1487725758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98941053b0 0x7f989419ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:50.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.562+0000 7f9899710700 1 -- 172.21.15.69:0/1487725758 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9894105d00 msgr2=0x7f989419b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:50.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.562+0000 7f9899710700 1 --2- 172.21.15.69:0/1487725758 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9894105d00 0x7f989419b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:50.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.562+0000 7f9899710700 1 -- 172.21.15.69:0/1487725758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98941053b0 msgr2=0x7f989419ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:50.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.562+0000 7f9899710700 1 --2- 172.21.15.69:0/1487725758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98941053b0 0x7f989419ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:50.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.562+0000 7f9899710700 1 -- 172.21.15.69:0/1487725758 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f989419fc30 con 0x7f989410e7e0 2024-04-25T11:23:50.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.562+0000 7f9893fff700 1 --2- 172.21.15.69:0/1487725758 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9894105d00 0x7f989419b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:23:50.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.562+0000 7f9898f0f700 1 --2- 172.21.15.69:0/1487725758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98941053b0 0x7f989419ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:23:50.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.562+0000 7f9899710700 1 --2- 172.21.15.69:0/1487725758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f989410e7e0 0x7f989419f590 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f988c00b970 tx=0x7f988c00bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:50.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.562+0000 7f9891ffb700 1 -- 172.21.15.69:0/1487725758 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c019070 con 0x7f989410e7e0 2024-04-25T11:23:50.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.562+0000 7f9891ffb700 1 -- 172.21.15.69:0/1487725758 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f988c00ce40 con 0x7f989410e7e0 2024-04-25T11:23:50.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.562+0000 7f9891ffb700 1 -- 172.21.15.69:0/1487725758 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c007370 con 0x7f989410e7e0 2024-04-25T11:23:50.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.562+0000 7f989b173700 1 -- 172.21.15.69:0/1487725758 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f989419ff20 con 0x7f989410e7e0 2024-04-25T11:23:50.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.562+0000 7f989b173700 1 -- 172.21.15.69:0/1487725758 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98941a7840 con 0x7f989410e7e0 2024-04-25T11:23:50.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.563+0000 7f989b173700 1 -- 172.21.15.69:0/1487725758 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9894051190 con 0x7f989410e7e0 2024-04-25T11:23:50.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.565+0000 7f9891ffb700 1 -- 172.21.15.69:0/1487725758 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f988c01c070 con 0x7f989410e7e0 2024-04-25T11:23:50.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.565+0000 7f9891ffb700 1 --2- 172.21.15.69:0/1487725758 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f98840afb80 0x7f98840b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:50.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.565+0000 7f9898f0f700 1 --2- 172.21.15.69:0/1487725758 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f98840afb80 0x7f98840b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:50.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.566+0000 7f9891ffb700 1 -- 172.21.15.69:0/1487725758 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7f988c0debb0 con 0x7f989410e7e0 2024-04-25T11:23:50.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.566+0000 7f9898f0f700 1 --2- 172.21.15.69:0/1487725758 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f98840afb80 0x7f98840b2090 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f98880099d0 tx=0x7f9888008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:50.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.569+0000 7f9891ffb700 1 -- 172.21.15.69:0/1487725758 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f988c023070 con 0x7f989410e7e0 2024-04-25T11:23:50.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.814+0000 7f989b173700 1 -- 172.21.15.69:0/1487725758 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f989402e510 con 0x7f989410e7e0 2024-04-25T11:23:50.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.815+0000 7f9891ffb700 1 -- 172.21.15.69:0/1487725758 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f988c0a7550 con 0x7f989410e7e0 2024-04-25T11:23:50.809 INFO:teuthology.orchestra.run.smithi069.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T11:23:50.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.817+0000 7f989b173700 1 -- 172.21.15.69:0/1487725758 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f98840afb80 msgr2=0x7f98840b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:50.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.817+0000 7f989b173700 1 --2- 172.21.15.69:0/1487725758 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f98840afb80 0x7f98840b2090 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f98880099d0 tx=0x7f9888008040 comp rx=0 tx=0).stop 2024-04-25T11:23:50.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.817+0000 7f989b173700 1 -- 172.21.15.69:0/1487725758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f989410e7e0 msgr2=0x7f989419f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:50.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.817+0000 7f989b173700 1 --2- 172.21.15.69:0/1487725758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f989410e7e0 0x7f989419f590 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f988c00b970 tx=0x7f988c00bd30 comp rx=0 tx=0).stop 2024-04-25T11:23:50.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.817+0000 7f989b173700 1 -- 172.21.15.69:0/1487725758 shutdown_connections 2024-04-25T11:23:50.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.817+0000 7f989b173700 1 --2- 172.21.15.69:0/1487725758 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f98840afb80 0x7f98840b2090 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:50.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.817+0000 7f989b173700 1 --2- 172.21.15.69:0/1487725758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f989410e7e0 0x7f989419f590 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:50.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.817+0000 7f989b173700 1 --2- 172.21.15.69:0/1487725758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f98941053b0 0x7f989419ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:50.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.817+0000 7f989b173700 1 --2- 172.21.15.69:0/1487725758 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9894105d00 0x7f989419b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:50.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.817+0000 7f989b173700 1 -- 172.21.15.69:0/1487725758 >> 172.21.15.69:0/1487725758 conn(0x7f98940fec90 msgr2=0x7f98940ff480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:50.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.817+0000 7f989b173700 1 -- 172.21.15.69:0/1487725758 shutdown_connections 2024-04-25T11:23:50.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:50.817+0000 7f989b173700 1 -- 172.21.15.69:0/1487725758 wait complete. 2024-04-25T11:23:50.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:50 smithi069 ceph-mon[29188]: pgmap v289: 129 pgs: 1 creating+activating, 3 creating+peering, 125 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 199 B/s wr, 3 op/s 2024-04-25T11:23:50.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:50 smithi152 ceph-mon[32701]: pgmap v289: 129 pgs: 1 creating+activating, 3 creating+peering, 125 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 199 B/s wr, 3 op/s 2024-04-25T11:23:51.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:50 smithi146 ceph-mon[32821]: pgmap v289: 129 pgs: 1 creating+activating, 3 creating+peering, 125 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 199 B/s wr, 3 op/s 2024-04-25T11:23:51.512 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T11:23:51.512 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:23:51.512 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi069.front.sepia.ceph.com 2024-04-25T11:23:51.512 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-25T11:23:51.512 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T11:23:51.513 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T11:23:51.513 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T11:23:51.513 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:23:51.513 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T11:23:51.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:23:51.513 DEBUG:teuthology.orchestra.run.smithi069:> ip addr 2024-04-25T11:23:51.530 INFO:teuthology.orchestra.run.smithi069.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:23:51.530 INFO:teuthology.orchestra.run.smithi069.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:23:51.530 INFO:teuthology.orchestra.run.smithi069.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:23:51.530 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:23:51.530 INFO:teuthology.orchestra.run.smithi069.stdout: inet6 ::1/128 scope host 2024-04-25T11:23:51.530 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:23:51.530 INFO:teuthology.orchestra.run.smithi069.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:23:51.530 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:d8:d9:c2 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:23:51.530 INFO:teuthology.orchestra.run.smithi069.stdout: altname enp7s0f0 2024-04-25T11:23:51.530 INFO:teuthology.orchestra.run.smithi069.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:23:51.530 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:d8:d9:c3 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:23:51.531 INFO:teuthology.orchestra.run.smithi069.stdout: altname enp7s0f1 2024-04-25T11:23:51.531 INFO:teuthology.orchestra.run.smithi069.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:23:51.531 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:8f:d7:36 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:23:51.531 INFO:teuthology.orchestra.run.smithi069.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:23:51.531 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:8f:d7:37 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:23:51.531 INFO:teuthology.orchestra.run.smithi069.stdout: inet 172.21.15.69/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T11:23:51.531 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft 42212sec preferred_lft 42212sec 2024-04-25T11:23:51.531 INFO:teuthology.orchestra.run.smithi069.stdout: inet6 fe80::4637:f1e7:b6c7:4071/64 scope link noprefixroute 2024-04-25T11:23:51.531 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:23:51.532 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T11:23:51.532 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:23:51.532 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-04-25T11:23:51.532 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link add name ceph-brx type bridge 2024-04-25T11:23:51.532 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip addr flush dev ceph-brx 2024-04-25T11:23:51.532 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link set ceph-brx up 2024-04-25T11:23:51.532 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T11:23:51.532 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-04-25T11:23:51.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:23:51.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:23:51.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:23:51.713 DEBUG:teuthology.orchestra.run.smithi069:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:23:51.735 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:51 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/1487725758' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T11:23:51.736 INFO:teuthology.orchestra.run.smithi069.stdout:1 2024-04-25T11:23:51.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:23:51.737 DEBUG:teuthology.orchestra.run.smithi069:> route 2024-04-25T11:23:51.822 INFO:teuthology.orchestra.run.smithi069.stdout:Kernel IP routing table 2024-04-25T11:23:51.822 INFO:teuthology.orchestra.run.smithi069.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:23:51.822 INFO:teuthology.orchestra.run.smithi069.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T11:23:51.822 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T11:23:51.822 INFO:teuthology.orchestra.run.smithi069.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:23:51.822 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:23:51.822 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-04-25T11:23:51.823 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T11:23:51.823 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T11:23:51.823 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T11:23:51.823 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-04-25T11:23:51.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:23:51.981 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:51 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/1487725758' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T11:23:52.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:23:52.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:23:52.081 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list 2024-04-25T11:23:52.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:23:52.097 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list-id 2024-04-25T11:23:52.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:51 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/1487725758' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T11:23:52.152 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:23:52.152 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-04-25T11:23:52.152 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T11:23:52.152 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-25T11:23:52.152 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-04-25T11:23:52.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:23:52.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:23:52.267 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-25T11:23:52.267 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:23:52.267 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-04-25T11:23:52.267 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-25T11:23:52.267 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T11:23:52.267 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-25T11:23:52.267 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-25T11:23:52.267 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-25T11:23:52.267 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-04-25T11:23:52.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:23:52.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:23:52.413 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:23:52.413 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-04-25T11:23:52.413 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link set brx.0 up 2024-04-25T11:23:52.413 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T11:23:52.413 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-04-25T11:23:52.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:23:52.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:23:52.487 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-25T11:23:52.487 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:23:52.488 DEBUG:teuthology.orchestra.run.smithi069:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-25T11:23:52.502 INFO:teuthology.orchestra.run.smithi069.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T11:23:52.502 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:23:52.502 DEBUG:teuthology.orchestra.run.smithi069:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-25T11:23:52.557 DEBUG:teuthology.orchestra.run.smithi069:> sudo modprobe fuse 2024-04-25T11:23:52.621 DEBUG:teuthology.orchestra.run.smithi069:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T11:23:52.677 INFO:teuthology.orchestra.run.smithi069.stdout:/proc 2024-04-25T11:23:52.677 INFO:teuthology.orchestra.run.smithi069.stdout:/sys 2024-04-25T11:23:52.677 INFO:teuthology.orchestra.run.smithi069.stdout:/dev 2024-04-25T11:23:52.677 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/security 2024-04-25T11:23:52.677 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/shm 2024-04-25T11:23:52.677 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/pts 2024-04-25T11:23:52.678 INFO:teuthology.orchestra.run.smithi069.stdout:/run 2024-04-25T11:23:52.678 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/cgroup 2024-04-25T11:23:52.678 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/pstore 2024-04-25T11:23:52.678 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/bpf 2024-04-25T11:23:52.678 INFO:teuthology.orchestra.run.smithi069.stdout:/ 2024-04-25T11:23:52.678 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/selinux 2024-04-25T11:23:52.678 INFO:teuthology.orchestra.run.smithi069.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T11:23:52.678 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/hugepages 2024-04-25T11:23:52.678 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/mqueue 2024-04-25T11:23:52.678 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/debug 2024-04-25T11:23:52.678 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/tracing 2024-04-25T11:23:52.678 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T11:23:52.678 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/fuse/connections 2024-04-25T11:23:52.679 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/config 2024-04-25T11:23:52.679 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T11:23:52.679 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T11:23:52.679 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T11:23:52.679 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T11:23:52.679 INFO:teuthology.orchestra.run.smithi069.stdout:/run/user/1000 2024-04-25T11:23:52.679 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/ceph 2024-04-25T11:23:52.679 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay 2024-04-25T11:23:52.679 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/9f4a46c23d4d86068aa88b4ed1c37628a2eac7a16343feefaeee7fe7577a197a/merged 2024-04-25T11:23:52.679 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/0e9a9a4a9937a1dfd02a7cf62b3058484401334d2dcad2fe35369ee846c671d2/merged 2024-04-25T11:23:52.679 INFO:teuthology.orchestra.run.smithi069.stdout:/run/user/0 2024-04-25T11:23:52.679 INFO:teuthology.orchestra.run.smithi069.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T11:23:52.679 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/2ec9d973deed8ffc94944b1e27e1cb8425417481ba8a54de9056a925de4ebbf6/merged 2024-04-25T11:23:52.679 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/47acf86d54d28602a952d5304647222bc771de57ca486ef4467a1678184c79cc/merged 2024-04-25T11:23:52.680 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/2b9ea3a24270210d404542f52a91f5ff1d1cd981381372eb54c8bc7fba4180be/merged 2024-04-25T11:23:52.680 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/ec425cdf63598fd0b9c9d832b419e769899261ec1b4fc1169beb34d51205c41e/merged 2024-04-25T11:23:52.680 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/1655ce1e6d74b3eb8cd3939568945e34e5f2bec725a5cc6b69244f04a4a8a219/merged 2024-04-25T11:23:52.680 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/6a86092111ee15e201da99074eddf4f6103d966a47d8a3794d659f1ebcc06710/merged 2024-04-25T11:23:52.680 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/5e34351696aeceb6d98047f196043703aec77e26a7575cf28fc317b0b6da0847/merged 2024-04-25T11:23:52.680 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/2b4fca38630f372edbce438286af127f856a1cfc9f37b0f9cfa2542baee1f413/merged 2024-04-25T11:23:52.680 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns 2024-04-25T11:23:52.680 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T11:23:52.680 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T11:23:52.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:23:52.680 DEBUG:teuthology.orchestra.run.smithi069:> ls /sys/fs/fuse/connections 2024-04-25T11:23:52.733 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T11:23:52.733 DEBUG:teuthology.orchestra.run.smithi069:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-25T11:23:52.775 DEBUG:teuthology.orchestra.run.smithi069:> sudo modprobe fuse 2024-04-25T11:23:52.798 DEBUG:teuthology.orchestra.run.smithi069:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T11:23:52.857 INFO:teuthology.orchestra.run.smithi069.stdout:/proc 2024-04-25T11:23:52.857 INFO:teuthology.orchestra.run.smithi069.stdout:/sys 2024-04-25T11:23:52.857 INFO:teuthology.orchestra.run.smithi069.stdout:/dev 2024-04-25T11:23:52.857 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/security 2024-04-25T11:23:52.857 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/shm 2024-04-25T11:23:52.857 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/pts 2024-04-25T11:23:52.857 INFO:teuthology.orchestra.run.smithi069.stdout:/run 2024-04-25T11:23:52.857 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/cgroup 2024-04-25T11:23:52.858 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/pstore 2024-04-25T11:23:52.858 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/bpf 2024-04-25T11:23:52.858 INFO:teuthology.orchestra.run.smithi069.stdout:/ 2024-04-25T11:23:52.858 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/selinux 2024-04-25T11:23:52.858 INFO:teuthology.orchestra.run.smithi069.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T11:23:52.858 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/hugepages 2024-04-25T11:23:52.858 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/mqueue 2024-04-25T11:23:52.858 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/debug 2024-04-25T11:23:52.858 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/tracing 2024-04-25T11:23:52.858 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T11:23:52.858 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/fuse/connections 2024-04-25T11:23:52.858 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/config 2024-04-25T11:23:52.858 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T11:23:52.859 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T11:23:52.859 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T11:23:52.859 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T11:23:52.859 INFO:teuthology.orchestra.run.smithi069.stdout:/run/user/1000 2024-04-25T11:23:52.859 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/ceph 2024-04-25T11:23:52.859 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay 2024-04-25T11:23:52.859 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/9f4a46c23d4d86068aa88b4ed1c37628a2eac7a16343feefaeee7fe7577a197a/merged 2024-04-25T11:23:52.859 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/0e9a9a4a9937a1dfd02a7cf62b3058484401334d2dcad2fe35369ee846c671d2/merged 2024-04-25T11:23:52.859 INFO:teuthology.orchestra.run.smithi069.stdout:/run/user/0 2024-04-25T11:23:52.859 INFO:teuthology.orchestra.run.smithi069.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T11:23:52.859 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/2ec9d973deed8ffc94944b1e27e1cb8425417481ba8a54de9056a925de4ebbf6/merged 2024-04-25T11:23:52.859 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/47acf86d54d28602a952d5304647222bc771de57ca486ef4467a1678184c79cc/merged 2024-04-25T11:23:52.859 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/2b9ea3a24270210d404542f52a91f5ff1d1cd981381372eb54c8bc7fba4180be/merged 2024-04-25T11:23:52.859 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/ec425cdf63598fd0b9c9d832b419e769899261ec1b4fc1169beb34d51205c41e/merged 2024-04-25T11:23:52.860 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/1655ce1e6d74b3eb8cd3939568945e34e5f2bec725a5cc6b69244f04a4a8a219/merged 2024-04-25T11:23:52.860 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/6a86092111ee15e201da99074eddf4f6103d966a47d8a3794d659f1ebcc06710/merged 2024-04-25T11:23:52.860 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/5e34351696aeceb6d98047f196043703aec77e26a7575cf28fc317b0b6da0847/merged 2024-04-25T11:23:52.860 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/2b4fca38630f372edbce438286af127f856a1cfc9f37b0f9cfa2542baee1f413/merged 2024-04-25T11:23:52.860 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns 2024-04-25T11:23:52.860 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T11:23:52.860 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T11:23:52.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:23:52.860 DEBUG:teuthology.orchestra.run.smithi069:> ls /sys/fs/fuse/connections 2024-04-25T11:23:52.872 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi069.stderr:2024-04-25T11:23:52.877+0000 7f8024066480 -1 init, newargv = 0x7f8008056f20 newargc=15 2024-04-25T11:23:52.872 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi069.stderr:ceph-fuse[73034]: starting ceph client 2024-04-25T11:23:52.879 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi069.stderr:ceph-fuse[73034]: starting fuse 2024-04-25T11:23:52.886 INFO:teuthology.orchestra.run.smithi069.stdout:115 2024-04-25T11:23:52.887 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [115] 2024-04-25T11:23:52.887 DEBUG:teuthology.orchestra.run.smithi069:> sudo stdin-killer -- python3 -c ' 2024-04-25T11:23:52.887 DEBUG:teuthology.orchestra.run.smithi069:> import glob 2024-04-25T11:23:52.887 DEBUG:teuthology.orchestra.run.smithi069:> import re 2024-04-25T11:23:52.887 DEBUG:teuthology.orchestra.run.smithi069:> import os 2024-04-25T11:23:52.887 DEBUG:teuthology.orchestra.run.smithi069:> import subprocess 2024-04-25T11:23:52.887 DEBUG:teuthology.orchestra.run.smithi069:> 2024-04-25T11:23:52.887 DEBUG:teuthology.orchestra.run.smithi069:> def _find_admin_socket(client_name): 2024-04-25T11:23:52.887 DEBUG:teuthology.orchestra.run.smithi069:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-25T11:23:52.887 DEBUG:teuthology.orchestra.run.smithi069:> files = glob.glob(asok_path) 2024-04-25T11:23:52.887 DEBUG:teuthology.orchestra.run.smithi069:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-25T11:23:52.887 DEBUG:teuthology.orchestra.run.smithi069:> 2024-04-25T11:23:52.888 DEBUG:teuthology.orchestra.run.smithi069:> # Given a non-glob path, it better be there 2024-04-25T11:23:52.888 DEBUG:teuthology.orchestra.run.smithi069:> if "*" not in asok_path: 2024-04-25T11:23:52.888 DEBUG:teuthology.orchestra.run.smithi069:> assert(len(files) == 1) 2024-04-25T11:23:52.888 DEBUG:teuthology.orchestra.run.smithi069:> return files[0] 2024-04-25T11:23:52.888 DEBUG:teuthology.orchestra.run.smithi069:> 2024-04-25T11:23:52.888 DEBUG:teuthology.orchestra.run.smithi069:> for f in files: 2024-04-25T11:23:52.888 DEBUG:teuthology.orchestra.run.smithi069:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T11:23:52.888 DEBUG:teuthology.orchestra.run.smithi069:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T11:23:52.888 DEBUG:teuthology.orchestra.run.smithi069:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T11:23:52.888 DEBUG:teuthology.orchestra.run.smithi069:> contents = proc_f.read() 2024-04-25T11:23:52.888 DEBUG:teuthology.orchestra.run.smithi069:> if mountpoint in contents: 2024-04-25T11:23:52.888 DEBUG:teuthology.orchestra.run.smithi069:> return f 2024-04-25T11:23:52.888 DEBUG:teuthology.orchestra.run.smithi069:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T11:23:52.888 DEBUG:teuthology.orchestra.run.smithi069:> 2024-04-25T11:23:52.888 DEBUG:teuthology.orchestra.run.smithi069:> print(_find_admin_socket("client.admin")) 2024-04-25T11:23:52.888 DEBUG:teuthology.orchestra.run.smithi069:> ' 2024-04-25T11:23:52.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:52 smithi069 ceph-mon[29188]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 167 B/s wr, 3 op/s 2024-04-25T11:23:52.980 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:52 smithi152 ceph-mon[32701]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 167 B/s wr, 3 op/s 2024-04-25T11:23:53.003 INFO:teuthology.orchestra.run.smithi069.stdout:/var/run/ceph/ceph-client.admin.73034.asok 2024-04-25T11:23:53.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:23:53.013 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.73034.asok 2024-04-25T11:23:53.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:23:53.014 DEBUG:teuthology.orchestra.run.smithi069:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.73034.asok status 2024-04-25T11:23:53.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:52 smithi146 ceph-mon[32821]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 167 B/s wr, 3 op/s 2024-04-25T11:23:53.155 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-25T11:23:53.155 INFO:teuthology.orchestra.run.smithi069.stdout: "metadata": { 2024-04-25T11:23:53.155 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T11:23:53.155 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T11:23:53.155 INFO:teuthology.orchestra.run.smithi069.stdout: "entity_id": "admin", 2024-04-25T11:23:53.156 INFO:teuthology.orchestra.run.smithi069.stdout: "hostname": "smithi069", 2024-04-25T11:23:53.156 INFO:teuthology.orchestra.run.smithi069.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-25T11:23:53.156 INFO:teuthology.orchestra.run.smithi069.stdout: "pid": "73034", 2024-04-25T11:23:53.156 INFO:teuthology.orchestra.run.smithi069.stdout: "root": "/" 2024-04-25T11:23:53.156 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:23:53.156 INFO:teuthology.orchestra.run.smithi069.stdout: "dentry_count": 0, 2024-04-25T11:23:53.156 INFO:teuthology.orchestra.run.smithi069.stdout: "dentry_pinned_count": 0, 2024-04-25T11:23:53.156 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 14769, 2024-04-25T11:23:53.156 INFO:teuthology.orchestra.run.smithi069.stdout: "inst": { 2024-04-25T11:23:53.156 INFO:teuthology.orchestra.run.smithi069.stdout: "name": { 2024-04-25T11:23:53.156 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "client", 2024-04-25T11:23:53.156 INFO:teuthology.orchestra.run.smithi069.stdout: "num": 14769 2024-04-25T11:23:53.156 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:23:53.156 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": { 2024-04-25T11:23:53.156 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v1", 2024-04-25T11:23:53.157 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "192.168.0.1:0", 2024-04-25T11:23:53.157 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 934448373 2024-04-25T11:23:53.157 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:23:53.157 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:23:53.157 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": { 2024-04-25T11:23:53.157 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v1", 2024-04-25T11:23:53.157 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "192.168.0.1:0", 2024-04-25T11:23:53.157 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 934448373 2024-04-25T11:23:53.157 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:23:53.157 INFO:teuthology.orchestra.run.smithi069.stdout: "inst_str": "client.14769 192.168.0.1:0/934448373", 2024-04-25T11:23:53.157 INFO:teuthology.orchestra.run.smithi069.stdout: "addr_str": "192.168.0.1:0/934448373", 2024-04-25T11:23:53.157 INFO:teuthology.orchestra.run.smithi069.stdout: "inode_count": 1, 2024-04-25T11:23:53.157 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_epoch": 17, 2024-04-25T11:23:53.157 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_epoch": 80, 2024-04-25T11:23:53.157 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_epoch_barrier": 0, 2024-04-25T11:23:53.158 INFO:teuthology.orchestra.run.smithi069.stdout: "blocklisted": false, 2024-04-25T11:23:53.158 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs" 2024-04-25T11:23:53.158 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-25T11:23:53.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:23:53.164 DEBUG:teuthology.orchestra.run.smithi069:> stat --file-system '--printf=%T 2024-04-25T11:23:53.164 DEBUG:teuthology.orchestra.run.smithi069:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-25T11:23:53.179 INFO:teuthology.orchestra.run.smithi069.stdout:fuseblk 2024-04-25T11:23:53.179 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-25T11:23:53.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:23:53.179 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-25T11:23:53.244 DEBUG:teuthology.orchestra.run.smithi069:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-25T11:23:53.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:23:53.372 INFO:teuthology.orchestra.run.smithi069.stdout:# file: . 2024-04-25T11:23:53.372 INFO:teuthology.orchestra.run.smithi069.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-25T11:23:53.372 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:23:53.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:23:53.378 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T11:23:53.378 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:23:53.394 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:23:53.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.548+0000 7fad46a4c640 1 -- 172.21.15.69:0/2330272244 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad40103c00 msgr2=0x7fad40104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:53.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.548+0000 7fad46a4c640 1 --2- 172.21.15.69:0/2330272244 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad40103c00 0x7fad40104010 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fad30009900 tx=0x7fad3002a410 comp rx=0 tx=0).stop 2024-04-25T11:23:53.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.548+0000 7fad46a4c640 1 -- 172.21.15.69:0/2330272244 shutdown_connections 2024-04-25T11:23:53.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.548+0000 7fad46a4c640 1 --2- 172.21.15.69:0/2330272244 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad40108900 0x7fad4010ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:53.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.548+0000 7fad46a4c640 1 --2- 172.21.15.69:0/2330272244 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad40103c00 0x7fad40104010 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:53.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.548+0000 7fad46a4c640 1 --2- 172.21.15.69:0/2330272244 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad401029c0 0x7fad401036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:53.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.548+0000 7fad46a4c640 1 -- 172.21.15.69:0/2330272244 >> 172.21.15.69:0/2330272244 conn(0x7fad400fdbd0 msgr2=0x7fad40100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:53.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.548+0000 7fad46a4c640 1 -- 172.21.15.69:0/2330272244 shutdown_connections 2024-04-25T11:23:53.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.548+0000 7fad46a4c640 1 -- 172.21.15.69:0/2330272244 wait complete. 2024-04-25T11:23:53.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.548+0000 7fad46a4c640 1 Processor -- start 2024-04-25T11:23:53.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.548+0000 7fad46a4c640 1 -- start start 2024-04-25T11:23:53.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.549+0000 7fad46a4c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad401029c0 0x7fad4019a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:53.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.549+0000 7fad46a4c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad40103c00 0x7fad4019a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:53.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.549+0000 7fad46a4c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad40108900 0x7fad401a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:53.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.549+0000 7fad46a4c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fad4010f500 con 0x7fad40103c00 2024-04-25T11:23:53.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.549+0000 7fad46a4c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fad4010f380 con 0x7fad40108900 2024-04-25T11:23:53.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.549+0000 7fad46a4c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fad4010f680 con 0x7fad401029c0 2024-04-25T11:23:53.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.549+0000 7fad3f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad40103c00 0x7fad4019a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:53.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.549+0000 7fad3f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad40103c00 0x7fad4019a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58260/0 (socket says 172.21.15.69:58260) 2024-04-25T11:23:53.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.549+0000 7fad3f7fe640 1 -- 172.21.15.69:0/2472373394 learned_addr learned my addr 172.21.15.69:0/2472373394 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:53.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.549+0000 7fad3f7fe640 1 -- 172.21.15.69:0/2472373394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad401029c0 msgr2=0x7fad4019a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:53.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.549+0000 7fad3f7fe640 1 --2- 172.21.15.69:0/2472373394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad401029c0 0x7fad4019a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:53.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.549+0000 7fad3f7fe640 1 -- 172.21.15.69:0/2472373394 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad40108900 msgr2=0x7fad401a1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:53.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.549+0000 7fad44fe9640 1 --2- 172.21.15.69:0/2472373394 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad40108900 0x7fad401a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:53.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.549+0000 7fad3f7fe640 1 --2- 172.21.15.69:0/2472373394 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad40108900 0x7fad401a1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:53.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.549+0000 7fad3f7fe640 1 -- 172.21.15.69:0/2472373394 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad401a1cd0 con 0x7fad40103c00 2024-04-25T11:23:53.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.549+0000 7fad44fe9640 1 --2- 172.21.15.69:0/2472373394 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad40108900 0x7fad401a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:23:53.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.549+0000 7fad3f7fe640 1 --2- 172.21.15.69:0/2472373394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad40103c00 0x7fad4019a6d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fad30009230 tx=0x7fad300042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:53.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.549+0000 7fad3d7fa640 1 -- 172.21.15.69:0/2472373394 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad3003f030 con 0x7fad40103c00 2024-04-25T11:23:53.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.550+0000 7fad3d7fa640 1 -- 172.21.15.69:0/2472373394 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad30003c90 con 0x7fad40103c00 2024-04-25T11:23:53.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.550+0000 7fad46a4c640 1 -- 172.21.15.69:0/2472373394 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad401a1f60 con 0x7fad40103c00 2024-04-25T11:23:53.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.550+0000 7fad3d7fa640 1 -- 172.21.15.69:0/2472373394 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad30035070 con 0x7fad40103c00 2024-04-25T11:23:53.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.550+0000 7fad46a4c640 1 -- 172.21.15.69:0/2472373394 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad401a2440 con 0x7fad40103c00 2024-04-25T11:23:53.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.551+0000 7fad46a4c640 1 -- 172.21.15.69:0/2472373394 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad40107710 con 0x7fad40103c00 2024-04-25T11:23:53.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.552+0000 7fad3d7fa640 1 -- 172.21.15.69:0/2472373394 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fad30042030 con 0x7fad40103c00 2024-04-25T11:23:53.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.553+0000 7fad3d7fa640 1 --2- 172.21.15.69:0/2472373394 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fad2c0afb70 0x7fad2c0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:53.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.553+0000 7fad3ffff640 1 --2- 172.21.15.69:0/2472373394 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fad2c0afb70 0x7fad2c0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:53.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.553+0000 7fad3d7fa640 1 -- 172.21.15.69:0/2472373394 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7fad300ff580 con 0x7fad40103c00 2024-04-25T11:23:53.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.553+0000 7fad3ffff640 1 --2- 172.21.15.69:0/2472373394 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fad2c0afb70 0x7fad2c0b2080 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fad28005fd0 tx=0x7fad2800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:53.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.558+0000 7fad3d7fa640 1 -- 172.21.15.69:0/2472373394 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fad300c7ec0 con 0x7fad40103c00 2024-04-25T11:23:53.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.741+0000 7fad46a4c640 1 -- 172.21.15.69:0/2472373394 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad4002dd70 con 0x7fad40103c00 2024-04-25T11:23:53.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.742+0000 7fad3d7fa640 1 -- 172.21.15.69:0/2472373394 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+24130 (secure 0 0 0) 0x7fad300c77c0 con 0x7fad40103c00 2024-04-25T11:23:53.738 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:23:53.738 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":80,"fsid":"f7ed73dc-02f4-11ef-bc93-c7b262605968","created":"2024-04-25T11:15:37.965281+0000","modified":"2024-04-25T11:23:47.662955+0000","last_up_change":"2024-04-25T11:22:50.691869+0000","last_in_change":"2024-04-25T11:22:30.271959+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:18:57.781263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:23:38.685110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:23:39.943803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T11:23:43.650521+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6a0d179-4998-4754-bf82-2ea3dac54e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6803","nonce":3756025235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6805","nonce":3756025235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6809","nonce":3756025235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6807","nonce":3756025235}]},"public_addr":"172.21.15.69:6803/3756025235","cluster_addr":"172.21.15.69:6805/3756025235","heartbeat_back_addr":"172.21.15.69:6809/3756025235","heartbeat_front_addr":"172.21.15.69:6807/3756025235","state":["exists","up"]},{"osd":1,"uuid":"c0c66930-4233-493f-b2bb-99c473994b59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6801","nonce":3166305213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6803","nonce":3166305213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6807","nonce":3166305213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6805","nonce":3166305213}]},"public_addr":"172.21.15.146:6801/3166305213","cluster_addr":"172.21.15.146:6803/3166305213","heartbeat_back_addr":"172.21.15.146:6807/3166305213","heartbeat_front_addr":"172.21.15.146:6805/3166305213","state":["exists","up"]},{"osd":2,"uuid":"d4ef735b-9786-4cb9-83ef-7c3f0691a729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6801","nonce":1474319175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6803","nonce":1474319175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6807","nonce":1474319175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6805","nonce":1474319175}]},"public_addr":"172.21.15.152:6801/1474319175","cluster_addr":"172.21.15.152:6803/1474319175","heartbeat_back_addr":"172.21.15.152:6807/1474319175","heartbeat_front_addr":"172.21.15.152:6805/1474319175","state":["exists","up"]},{"osd":3,"uuid":"0ab85fb8-5bbb-4ffc-bddf-cae7644a64ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6811","nonce":1809071296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6813","nonce":1809071296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6817","nonce":1809071296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6815","nonce":1809071296}]},"public_addr":"172.21.15.69:6811/1809071296","cluster_addr":"172.21.15.69:6813/1809071296","heartbeat_back_addr":"172.21.15.69:6817/1809071296","heartbeat_front_addr":"172.21.15.69:6815/1809071296","state":["exists","up"]},{"osd":4,"uuid":"273e0592-bbbc-4188-8eeb-46a3879cf632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6809","nonce":1898520972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6811","nonce":1898520972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6815","nonce":1898520972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6813","nonce":1898520972}]},"public_addr":"172.21.15.146:6809/1898520972","cluster_addr":"172.21.15.146:6811/1898520972","heartbeat_back_addr":"172.21.15.146:6815/1898520972","heartbeat_front_addr":"172.21.15.146:6813/1898520972","state":["exists","up"]},{"osd":5,"uuid":"bbeae945-9fa2-48f9-a1d6-8bf18dcaef19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6809","nonce":1996555108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6811","nonce":1996555108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6815","nonce":1996555108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6813","nonce":1996555108}]},"public_addr":"172.21.15.152:6809/1996555108","cluster_addr":"172.21.15.152:6811/1996555108","heartbeat_back_addr":"172.21.15.152:6815/1996555108","heartbeat_front_addr":"172.21.15.152:6813/1996555108","state":["exists","up"]},{"osd":6,"uuid":"dbb61253-30bc-418e-bd75-118c80bbf931","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6819","nonce":491215762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6821","nonce":491215762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6825","nonce":491215762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6823","nonce":491215762}]},"public_addr":"172.21.15.69:6819/491215762","cluster_addr":"172.21.15.69:6821/491215762","heartbeat_back_addr":"172.21.15.69:6825/491215762","heartbeat_front_addr":"172.21.15.69:6823/491215762","state":["exists","up"]},{"osd":7,"uuid":"75f23854-e283-4e3e-94cf-7b4079d83876","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6817","nonce":2330294690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6819","nonce":2330294690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6823","nonce":2330294690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6821","nonce":2330294690}]},"public_addr":"172.21.15.146:6817/2330294690","cluster_addr":"172.21.15.146:6819/2330294690","heartbeat_back_addr":"172.21.15.146:6823/2330294690","heartbeat_front_addr":"172.21.15.146:6821/2330294690","state":["exists","up"]},{"osd":8,"uuid":"1aa1060c-0c14-4abd-be56-5ebfeb78759c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6817","nonce":950669381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6819","nonce":950669381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6823","nonce":950669381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6821","nonce":950669381}]},"public_addr":"172.21.15.152:6817/950669381","cluster_addr":"172.21.15.152:6819/950669381","heartbeat_back_addr":"172.21.15.152:6823/950669381","heartbeat_front_addr":"172.21.15.152:6821/950669381","state":["exists","up"]},{"osd":9,"uuid":"3f4bf800-4090-4138-9bb3-d001f13f450c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6827","nonce":2414186758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6829","nonce":2414186758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6833","nonce":2414186758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6831","nonce":2414186758}]},"public_addr":"172.21.15.69:6827/2414186758","cluster_addr":"172.21.15.69:6829/2414186758","heartbeat_back_addr":"172.21.15.69:6833/2414186758","heartbeat_front_addr":"172.21.15.69:6831/2414186758","state":["exists","up"]},{"osd":10,"uuid":"19e967e5-51ce-4506-80c9-0a95441baf99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6825","nonce":707256400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6827","nonce":707256400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6831","nonce":707256400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6829","nonce":707256400}]},"public_addr":"172.21.15.146:6825/707256400","cluster_addr":"172.21.15.146:6827/707256400","heartbeat_back_addr":"172.21.15.146:6831/707256400","heartbeat_front_addr":"172.21.15.146:6829/707256400","state":["exists","up"]},{"osd":11,"uuid":"7788c2de-6930-4808-be0f-f1fd1ae0cb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6825","nonce":2934336833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6827","nonce":2934336833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6831","nonce":2934336833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6829","nonce":2934336833}]},"public_addr":"172.21.15.152:6825/2934336833","cluster_addr":"172.21.15.152:6827/2934336833","heartbeat_back_addr":"172.21.15.152:6831/2934336833","heartbeat_front_addr":"172.21.15.152:6829/2934336833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:07.497448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:31.227494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:55.027978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:20.713437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:46.214704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:20:11.199983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:20:37.177070+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:02.951240+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:28.963680+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:56.095807+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:22:22.221744+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:22:48.929334+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/1372727546":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6801/1097124842":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6800/1508198215":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3103465990":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3540132960":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3899912190":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6800/1097124842":"2024-04-26T11:16:31.688159+0000","172.21.15.69:0/1199907202":"2024-04-26T11:16:04.860792+0000","172.21.15.69:6801/1508198215":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/2879102529":"2024-04-26T11:16:31.688159+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:23:53.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.744+0000 7fad46a4c640 1 -- 172.21.15.69:0/2472373394 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fad2c0afb70 msgr2=0x7fad2c0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:53.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.744+0000 7fad46a4c640 1 --2- 172.21.15.69:0/2472373394 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fad2c0afb70 0x7fad2c0b2080 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fad28005fd0 tx=0x7fad2800b410 comp rx=0 tx=0).stop 2024-04-25T11:23:53.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.744+0000 7fad46a4c640 1 -- 172.21.15.69:0/2472373394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad40103c00 msgr2=0x7fad4019a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:53.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.744+0000 7fad46a4c640 1 --2- 172.21.15.69:0/2472373394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad40103c00 0x7fad4019a6d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fad30009230 tx=0x7fad300042f0 comp rx=0 tx=0).stop 2024-04-25T11:23:53.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.744+0000 7fad46a4c640 1 -- 172.21.15.69:0/2472373394 shutdown_connections 2024-04-25T11:23:53.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.744+0000 7fad46a4c640 1 --2- 172.21.15.69:0/2472373394 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fad2c0afb70 0x7fad2c0b2080 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:53.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.744+0000 7fad46a4c640 1 --2- 172.21.15.69:0/2472373394 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad40108900 0x7fad401a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:53.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.744+0000 7fad46a4c640 1 --2- 172.21.15.69:0/2472373394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad40103c00 0x7fad4019a6d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:53.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.744+0000 7fad46a4c640 1 --2- 172.21.15.69:0/2472373394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad401029c0 0x7fad4019a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:53.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.744+0000 7fad46a4c640 1 -- 172.21.15.69:0/2472373394 >> 172.21.15.69:0/2472373394 conn(0x7fad400fdbd0 msgr2=0x7fad401048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:53.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.744+0000 7fad46a4c640 1 -- 172.21.15.69:0/2472373394 shutdown_connections 2024-04-25T11:23:53.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.744+0000 7fad46a4c640 1 -- 172.21.15.69:0/2472373394 wait complete. 2024-04-25T11:23:53.751 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:18:57.781263+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:23:38.685110+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:23:39.943803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T11:23:43.650521+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T11:23:53.752 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:23:53.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.871+0000 7efc3631f640 1 -- 172.21.15.69:0/1148407958 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc30102a00 msgr2=0x7efc30103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:53.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.871+0000 7efc3631f640 1 --2- 172.21.15.69:0/1148407958 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc30102a00 0x7efc30103700 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7efc20009900 tx=0x7efc2002a410 comp rx=0 tx=0).stop 2024-04-25T11:23:53.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.871+0000 7efc3631f640 1 -- 172.21.15.69:0/1148407958 shutdown_connections 2024-04-25T11:23:53.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.871+0000 7efc3631f640 1 --2- 172.21.15.69:0/1148407958 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc30108940 0x7efc3010ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:53.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.871+0000 7efc3631f640 1 --2- 172.21.15.69:0/1148407958 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc30103c40 0x7efc30104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:53.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.871+0000 7efc3631f640 1 --2- 172.21.15.69:0/1148407958 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc30102a00 0x7efc30103700 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:53.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.871+0000 7efc3631f640 1 -- 172.21.15.69:0/1148407958 >> 172.21.15.69:0/1148407958 conn(0x7efc300fdbf0 msgr2=0x7efc301000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:53.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.871+0000 7efc3631f640 1 -- 172.21.15.69:0/1148407958 shutdown_connections 2024-04-25T11:23:53.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.871+0000 7efc3631f640 1 -- 172.21.15.69:0/1148407958 wait complete. 2024-04-25T11:23:53.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.872+0000 7efc3631f640 1 Processor -- start 2024-04-25T11:23:53.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.872+0000 7efc3631f640 1 -- start start 2024-04-25T11:23:53.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.872+0000 7efc3631f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc30102a00 0x7efc3019a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:53.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.872+0000 7efc3631f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc30103c40 0x7efc3019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:53.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.872+0000 7efc3631f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc30108940 0x7efc301a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:53.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.872+0000 7efc3631f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efc3010f4e0 con 0x7efc30108940 2024-04-25T11:23:53.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.872+0000 7efc3631f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7efc3010f360 con 0x7efc30102a00 2024-04-25T11:23:53.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.872+0000 7efc3631f640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7efc3010f660 con 0x7efc30103c40 2024-04-25T11:23:53.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.873+0000 7efc348bc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc30108940 0x7efc301a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:53.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.873+0000 7efc2ffff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc30102a00 0x7efc3019a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:53.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.873+0000 7efc2f7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc30103c40 0x7efc3019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:53.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.873+0000 7efc348bc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc30108940 0x7efc301a1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58286/0 (socket says 172.21.15.69:58286) 2024-04-25T11:23:53.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.873+0000 7efc348bc640 1 -- 172.21.15.69:0/629343706 learned_addr learned my addr 172.21.15.69:0/629343706 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:53.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.873+0000 7efc348bc640 1 -- 172.21.15.69:0/629343706 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc30103c40 msgr2=0x7efc3019a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:53.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.873+0000 7efc348bc640 1 --2- 172.21.15.69:0/629343706 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc30103c40 0x7efc3019a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:53.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.873+0000 7efc348bc640 1 -- 172.21.15.69:0/629343706 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc30102a00 msgr2=0x7efc3019a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:53.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.873+0000 7efc348bc640 1 --2- 172.21.15.69:0/629343706 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc30102a00 0x7efc3019a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:53.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.873+0000 7efc348bc640 1 -- 172.21.15.69:0/629343706 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc301a1ca0 con 0x7efc30108940 2024-04-25T11:23:53.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.873+0000 7efc348bc640 1 --2- 172.21.15.69:0/629343706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc30108940 0x7efc301a1630 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efc2800cad0 tx=0x7efc2800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:53.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.873+0000 7efc2d7fa640 1 -- 172.21.15.69:0/629343706 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2800ae20 con 0x7efc30108940 2024-04-25T11:23:53.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.873+0000 7efc2d7fa640 1 -- 172.21.15.69:0/629343706 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efc28005020 con 0x7efc30108940 2024-04-25T11:23:53.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.873+0000 7efc3631f640 1 -- 172.21.15.69:0/629343706 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc301a1f90 con 0x7efc30108940 2024-04-25T11:23:53.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.873+0000 7efc2d7fa640 1 -- 172.21.15.69:0/629343706 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2800f6b0 con 0x7efc30108940 2024-04-25T11:23:53.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.873+0000 7efc3631f640 1 -- 172.21.15.69:0/629343706 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc301a23f0 con 0x7efc30108940 2024-04-25T11:23:53.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.874+0000 7efc3631f640 1 -- 172.21.15.69:0/629343706 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc30107750 con 0x7efc30108940 2024-04-25T11:23:53.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.876+0000 7efc2d7fa640 1 -- 172.21.15.69:0/629343706 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7efc280051d0 con 0x7efc30108940 2024-04-25T11:23:53.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.876+0000 7efc2d7fa640 1 --2- 172.21.15.69:0/629343706 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7efc180afaf0 0x7efc180b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:53.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.876+0000 7efc2ffff640 1 --2- 172.21.15.69:0/629343706 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7efc180afaf0 0x7efc180b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:53.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.877+0000 7efc2d7fa640 1 -- 172.21.15.69:0/629343706 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7efc280df0e0 con 0x7efc30108940 2024-04-25T11:23:53.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.877+0000 7efc2ffff640 1 --2- 172.21.15.69:0/629343706 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7efc180afaf0 0x7efc180b2000 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efc20005760 tx=0x7efc20000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:53.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:53.880+0000 7efc2d7fa640 1 -- 172.21.15.69:0/629343706 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efc28014030 con 0x7efc30108940 2024-04-25T11:23:54.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.057+0000 7efc3631f640 1 -- 172.21.15.69:0/629343706 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efc3002dda0 con 0x7efc30108940 2024-04-25T11:23:54.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.058+0000 7efc2d7fa640 1 -- 172.21.15.69:0/629343706 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) ==== 93+0+10 (secure 0 0 0) 0x7efc280a7430 con 0x7efc30108940 2024-04-25T11:23:54.053 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-04-25T11:23:54.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.059+0000 7efc3631f640 1 -- 172.21.15.69:0/629343706 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7efc180afaf0 msgr2=0x7efc180b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:54.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.059+0000 7efc3631f640 1 --2- 172.21.15.69:0/629343706 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7efc180afaf0 0x7efc180b2000 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efc20005760 tx=0x7efc20000bc0 comp rx=0 tx=0).stop 2024-04-25T11:23:54.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.059+0000 7efc3631f640 1 -- 172.21.15.69:0/629343706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc30108940 msgr2=0x7efc301a1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:54.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.059+0000 7efc3631f640 1 --2- 172.21.15.69:0/629343706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc30108940 0x7efc301a1630 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efc2800cad0 tx=0x7efc2800cf90 comp rx=0 tx=0).stop 2024-04-25T11:23:54.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.060+0000 7efc3631f640 1 -- 172.21.15.69:0/629343706 shutdown_connections 2024-04-25T11:23:54.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.060+0000 7efc3631f640 1 --2- 172.21.15.69:0/629343706 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7efc180afaf0 0x7efc180b2000 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:54.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.060+0000 7efc3631f640 1 --2- 172.21.15.69:0/629343706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc30108940 0x7efc301a1630 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:54.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.060+0000 7efc3631f640 1 --2- 172.21.15.69:0/629343706 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc30103c40 0x7efc3019a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:54.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.060+0000 7efc3631f640 1 --2- 172.21.15.69:0/629343706 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efc30102a00 0x7efc3019a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:54.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.060+0000 7efc3631f640 1 -- 172.21.15.69:0/629343706 >> 172.21.15.69:0/629343706 conn(0x7efc300fdbf0 msgr2=0x7efc30104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:54.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.060+0000 7efc3631f640 1 -- 172.21.15.69:0/629343706 shutdown_connections 2024-04-25T11:23:54.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.060+0000 7efc3631f640 1 -- 172.21.15.69:0/629343706 wait complete. 2024-04-25T11:23:54.064 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:53 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2472373394' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:23:54.066 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:23:54.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:53 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/2472373394' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:23:54.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.182+0000 7fae472e0640 1 -- 172.21.15.69:0/1553587830 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fae401098a0 msgr2=0x7fae4010bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:54.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.182+0000 7fae472e0640 1 --2- 172.21.15.69:0/1553587830 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fae401098a0 0x7fae4010bcc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fae38009ca0 tx=0x7fae3802a8c0 comp rx=0 tx=0).stop 2024-04-25T11:23:54.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.183+0000 7fae472e0640 1 -- 172.21.15.69:0/1553587830 shutdown_connections 2024-04-25T11:23:54.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.183+0000 7fae472e0640 1 --2- 172.21.15.69:0/1553587830 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fae401098a0 0x7fae4010bcc0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:54.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.183+0000 7fae472e0640 1 --2- 172.21.15.69:0/1553587830 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae40106f40 0x7fae40109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:54.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.183+0000 7fae472e0640 1 --2- 172.21.15.69:0/1553587830 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae401045e0 0x7fae40106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:54.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.183+0000 7fae472e0640 1 -- 172.21.15.69:0/1553587830 >> 172.21.15.69:0/1553587830 conn(0x7fae400fdbc0 msgr2=0x7fae40100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:54.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.183+0000 7fae472e0640 1 -- 172.21.15.69:0/1553587830 shutdown_connections 2024-04-25T11:23:54.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.183+0000 7fae472e0640 1 -- 172.21.15.69:0/1553587830 wait complete. 2024-04-25T11:23:54.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.183+0000 7fae472e0640 1 Processor -- start 2024-04-25T11:23:54.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.183+0000 7fae472e0640 1 -- start start 2024-04-25T11:23:54.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.183+0000 7fae472e0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fae401045e0 0x7fae4019a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:54.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.183+0000 7fae472e0640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae40106f40 0x7fae4019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:54.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.183+0000 7fae472e0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae401098a0 0x7fae401a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:54.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.183+0000 7fae472e0640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fae4010f570 con 0x7fae401045e0 2024-04-25T11:23:54.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.183+0000 7fae472e0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fae4010f3f0 con 0x7fae401098a0 2024-04-25T11:23:54.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.183+0000 7fae472e0640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fae4010f6f0 con 0x7fae40106f40 2024-04-25T11:23:54.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.184+0000 7fae4507c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fae401045e0 0x7fae4019a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:54.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.184+0000 7fae4507c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fae401045e0 0x7fae4019a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58308/0 (socket says 172.21.15.69:58308) 2024-04-25T11:23:54.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.184+0000 7fae4507c640 1 -- 172.21.15.69:0/2817070295 learned_addr learned my addr 172.21.15.69:0/2817070295 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:54.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.184+0000 7fae4487b640 1 --2- 172.21.15.69:0/2817070295 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae40106f40 0x7fae4019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:54.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.184+0000 7fae4587d640 1 --2- 172.21.15.69:0/2817070295 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae401098a0 0x7fae401a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:54.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.184+0000 7fae4507c640 1 -- 172.21.15.69:0/2817070295 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae40106f40 msgr2=0x7fae4019a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:54.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.184+0000 7fae4507c640 1 --2- 172.21.15.69:0/2817070295 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae40106f40 0x7fae4019a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:54.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.184+0000 7fae4507c640 1 -- 172.21.15.69:0/2817070295 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae401098a0 msgr2=0x7fae401a1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:54.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.184+0000 7fae4507c640 1 --2- 172.21.15.69:0/2817070295 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae401098a0 0x7fae401a1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:54.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.184+0000 7fae4507c640 1 -- 172.21.15.69:0/2817070295 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae401a1cb0 con 0x7fae401045e0 2024-04-25T11:23:54.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.184+0000 7fae4587d640 1 --2- 172.21.15.69:0/2817070295 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae401098a0 0x7fae401a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:23:54.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.184+0000 7fae4507c640 1 --2- 172.21.15.69:0/2817070295 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fae401045e0 0x7fae4019a170 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fae3c00c970 tx=0x7fae3c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:54.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.184+0000 7fae367fc640 1 -- 172.21.15.69:0/2817070295 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3c007bb0 con 0x7fae401045e0 2024-04-25T11:23:54.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.184+0000 7fae367fc640 1 -- 172.21.15.69:0/2817070295 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae3c007d50 con 0x7fae401045e0 2024-04-25T11:23:54.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.184+0000 7fae472e0640 1 -- 172.21.15.69:0/2817070295 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae401a1fa0 con 0x7fae401045e0 2024-04-25T11:23:54.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.184+0000 7fae367fc640 1 -- 172.21.15.69:0/2817070295 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3c004d00 con 0x7fae401045e0 2024-04-25T11:23:54.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.184+0000 7fae472e0640 1 -- 172.21.15.69:0/2817070295 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae401a2400 con 0x7fae401045e0 2024-04-25T11:23:54.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.185+0000 7fae472e0640 1 -- 172.21.15.69:0/2817070295 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae4006a190 con 0x7fae401045e0 2024-04-25T11:23:54.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.186+0000 7fae367fc640 1 -- 172.21.15.69:0/2817070295 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fae3c004ec0 con 0x7fae401045e0 2024-04-25T11:23:54.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.187+0000 7fae367fc640 1 --2- 172.21.15.69:0/2817070295 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fae2c0afaf0 0x7fae2c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:54.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.187+0000 7fae4487b640 1 --2- 172.21.15.69:0/2817070295 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fae2c0afaf0 0x7fae2c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:54.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.187+0000 7fae367fc640 1 -- 172.21.15.69:0/2817070295 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7fae3c0deef0 con 0x7fae401045e0 2024-04-25T11:23:54.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.188+0000 7fae4487b640 1 --2- 172.21.15.69:0/2817070295 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fae2c0afaf0 0x7fae2c0b2000 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fae4019b210 tx=0x7fae30009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:54.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.191+0000 7fae367fc640 1 -- 172.21.15.69:0/2817070295 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fae4006a190 con 0x7fae401045e0 2024-04-25T11:23:54.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:53 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/2472373394' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:23:54.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.368+0000 7fae472e0640 1 -- 172.21.15.69:0/2817070295 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fae4002dbd0 con 0x7fae401045e0 2024-04-25T11:23:54.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.369+0000 7fae367fc640 1 -- 172.21.15.69:0/2817070295 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) ==== 104+0+11 (secure 0 0 0) 0x7fae3c014030 con 0x7fae401045e0 2024-04-25T11:23:54.364 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-25T11:23:54.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.371+0000 7fae472e0640 1 -- 172.21.15.69:0/2817070295 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fae2c0afaf0 msgr2=0x7fae2c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:54.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.371+0000 7fae472e0640 1 --2- 172.21.15.69:0/2817070295 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fae2c0afaf0 0x7fae2c0b2000 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fae4019b210 tx=0x7fae30009110 comp rx=0 tx=0).stop 2024-04-25T11:23:54.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.371+0000 7fae472e0640 1 -- 172.21.15.69:0/2817070295 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fae401045e0 msgr2=0x7fae4019a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:54.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.371+0000 7fae472e0640 1 --2- 172.21.15.69:0/2817070295 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fae401045e0 0x7fae4019a170 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fae3c00c970 tx=0x7fae3c00cd70 comp rx=0 tx=0).stop 2024-04-25T11:23:54.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.371+0000 7fae472e0640 1 -- 172.21.15.69:0/2817070295 shutdown_connections 2024-04-25T11:23:54.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.371+0000 7fae472e0640 1 --2- 172.21.15.69:0/2817070295 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fae2c0afaf0 0x7fae2c0b2000 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:54.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.371+0000 7fae472e0640 1 --2- 172.21.15.69:0/2817070295 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fae401098a0 0x7fae401a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:54.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.371+0000 7fae472e0640 1 --2- 172.21.15.69:0/2817070295 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae40106f40 0x7fae4019a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:54.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.371+0000 7fae472e0640 1 --2- 172.21.15.69:0/2817070295 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fae401045e0 0x7fae4019a170 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:54.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.371+0000 7fae472e0640 1 -- 172.21.15.69:0/2817070295 >> 172.21.15.69:0/2817070295 conn(0x7fae400fdbc0 msgr2=0x7fae401050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:54.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.371+0000 7fae472e0640 1 -- 172.21.15.69:0/2817070295 shutdown_connections 2024-04-25T11:23:54.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.371+0000 7fae472e0640 1 -- 172.21.15.69:0/2817070295 wait complete. 2024-04-25T11:23:54.377 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:23:54.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.492+0000 7f4480cee640 1 -- 172.21.15.69:0/4265666079 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f447c102260 msgr2=0x7f447c10bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:54.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.492+0000 7f4480cee640 1 --2- 172.21.15.69:0/4265666079 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f447c102260 0x7f447c10bcc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4474009ca0 tx=0x7f447402a8c0 comp rx=0 tx=0).stop 2024-04-25T11:23:54.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.492+0000 7f4480cee640 1 -- 172.21.15.69:0/4265666079 shutdown_connections 2024-04-25T11:23:54.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.492+0000 7f4480cee640 1 --2- 172.21.15.69:0/4265666079 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f447c102260 0x7f447c10bcc0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:54.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.492+0000 7f4480cee640 1 --2- 172.21.15.69:0/4265666079 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f447c101760 0x7f447c101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:54.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.492+0000 7f4480cee640 1 --2- 172.21.15.69:0/4265666079 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f447c1050b0 0x7f447c1054c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:54.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.492+0000 7f4480cee640 1 -- 172.21.15.69:0/4265666079 >> 172.21.15.69:0/4265666079 conn(0x7f447c0fd380 msgr2=0x7f447c0ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:54.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.492+0000 7f4480cee640 1 -- 172.21.15.69:0/4265666079 shutdown_connections 2024-04-25T11:23:54.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.492+0000 7f4480cee640 1 -- 172.21.15.69:0/4265666079 wait complete. 2024-04-25T11:23:54.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.493+0000 7f4480cee640 1 Processor -- start 2024-04-25T11:23:54.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.493+0000 7f4480cee640 1 -- start start 2024-04-25T11:23:54.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.493+0000 7f4480cee640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f447c101760 0x7f447c19a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:54.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.493+0000 7f4480cee640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f447c102260 0x7f447c19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:54.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.493+0000 7f4480cee640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f447c1050b0 0x7f447c1a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:54.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.493+0000 7f4480cee640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f447c10f520 con 0x7f447c101760 2024-04-25T11:23:54.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.493+0000 7f4480cee640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f447c10f3a0 con 0x7f447c102260 2024-04-25T11:23:54.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.493+0000 7f4480cee640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f447c10f6a0 con 0x7f447c1050b0 2024-04-25T11:23:54.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.494+0000 7f447a59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f447c101760 0x7f447c19a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:54.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.494+0000 7f447a59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f447c101760 0x7f447c19a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58338/0 (socket says 172.21.15.69:58338) 2024-04-25T11:23:54.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.494+0000 7f447a59c640 1 -- 172.21.15.69:0/2001358552 learned_addr learned my addr 172.21.15.69:0/2001358552 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:54.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.494+0000 7f4479d9b640 1 --2- 172.21.15.69:0/2001358552 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f447c102260 0x7f447c19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:54.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.494+0000 7f447ad9d640 1 --2- 172.21.15.69:0/2001358552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f447c1050b0 0x7f447c1a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:54.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.494+0000 7f447a59c640 1 -- 172.21.15.69:0/2001358552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f447c1050b0 msgr2=0x7f447c1a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:54.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.494+0000 7f447a59c640 1 --2- 172.21.15.69:0/2001358552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f447c1050b0 0x7f447c1a1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:54.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.494+0000 7f447a59c640 1 -- 172.21.15.69:0/2001358552 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f447c102260 msgr2=0x7f447c19a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:54.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.494+0000 7f447a59c640 1 --2- 172.21.15.69:0/2001358552 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f447c102260 0x7f447c19a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:54.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.494+0000 7f447a59c640 1 -- 172.21.15.69:0/2001358552 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f447c1a1c90 con 0x7f447c101760 2024-04-25T11:23:54.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.494+0000 7f447a59c640 1 --2- 172.21.15.69:0/2001358552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f447c101760 0x7f447c19a150 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f446c00d740 tx=0x7f446c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:54.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.494+0000 7f446b7fe640 1 -- 172.21.15.69:0/2001358552 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446c017070 con 0x7f447c101760 2024-04-25T11:23:54.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.494+0000 7f4480cee640 1 -- 172.21.15.69:0/2001358552 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f447c1a1f80 con 0x7f447c101760 2024-04-25T11:23:54.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.494+0000 7f446b7fe640 1 -- 172.21.15.69:0/2001358552 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f446c00de50 con 0x7f447c101760 2024-04-25T11:23:54.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.494+0000 7f446b7fe640 1 -- 172.21.15.69:0/2001358552 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446c003bb0 con 0x7f447c101760 2024-04-25T11:23:54.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.494+0000 7f4480cee640 1 -- 172.21.15.69:0/2001358552 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f447c1a2440 con 0x7f447c101760 2024-04-25T11:23:54.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.495+0000 7f4480cee640 1 -- 172.21.15.69:0/2001358552 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f447c10aa50 con 0x7f447c101760 2024-04-25T11:23:54.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.497+0000 7f446b7fe640 1 -- 172.21.15.69:0/2001358552 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f446c003d50 con 0x7f447c101760 2024-04-25T11:23:54.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.497+0000 7f446b7fe640 1 --2- 172.21.15.69:0/2001358552 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f44640afaf0 0x7f44640b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:54.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.497+0000 7f4479d9b640 1 --2- 172.21.15.69:0/2001358552 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f44640afaf0 0x7f44640b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:54.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.497+0000 7f446b7fe640 1 -- 172.21.15.69:0/2001358552 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7f446c0deb50 con 0x7f447c101760 2024-04-25T11:23:54.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.498+0000 7f4479d9b640 1 --2- 172.21.15.69:0/2001358552 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f44640afaf0 0x7f44640b2000 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f44700058c0 tx=0x7f447000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:54.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.501+0000 7f446b7fe640 1 -- 172.21.15.69:0/2001358552 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f446c0a6ea0 con 0x7f447c101760 2024-04-25T11:23:54.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.677+0000 7f4480cee640 1 -- 172.21.15.69:0/2001358552 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f447c02dda0 con 0x7f447c101760 2024-04-25T11:23:54.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.678+0000 7f446b7fe640 1 -- 172.21.15.69:0/2001358552 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) ==== 100+0+11 (secure 0 0 0) 0x7f446c0ab1a0 con 0x7f447c101760 2024-04-25T11:23:54.673 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-04-25T11:23:54.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.680+0000 7f4480cee640 1 -- 172.21.15.69:0/2001358552 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f44640afaf0 msgr2=0x7f44640b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:54.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.680+0000 7f4480cee640 1 --2- 172.21.15.69:0/2001358552 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f44640afaf0 0x7f44640b2000 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f44700058c0 tx=0x7f447000a2d0 comp rx=0 tx=0).stop 2024-04-25T11:23:54.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.680+0000 7f4480cee640 1 -- 172.21.15.69:0/2001358552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f447c101760 msgr2=0x7f447c19a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:54.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.680+0000 7f4480cee640 1 --2- 172.21.15.69:0/2001358552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f447c101760 0x7f447c19a150 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f446c00d740 tx=0x7f446c00dc00 comp rx=0 tx=0).stop 2024-04-25T11:23:54.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.680+0000 7f4480cee640 1 -- 172.21.15.69:0/2001358552 shutdown_connections 2024-04-25T11:23:54.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.680+0000 7f4480cee640 1 --2- 172.21.15.69:0/2001358552 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f44640afaf0 0x7f44640b2000 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:54.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.680+0000 7f4480cee640 1 --2- 172.21.15.69:0/2001358552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f447c1050b0 0x7f447c1a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:54.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.680+0000 7f4480cee640 1 --2- 172.21.15.69:0/2001358552 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f447c102260 0x7f447c19a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:54.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.680+0000 7f4480cee640 1 --2- 172.21.15.69:0/2001358552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f447c101760 0x7f447c19a150 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:54.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.680+0000 7f4480cee640 1 -- 172.21.15.69:0/2001358552 >> 172.21.15.69:0/2001358552 conn(0x7f447c0fd380 msgr2=0x7f447c107c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:54.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.680+0000 7f4480cee640 1 -- 172.21.15.69:0/2001358552 shutdown_connections 2024-04-25T11:23:54.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.680+0000 7f4480cee640 1 -- 172.21.15.69:0/2001358552 wait complete. 2024-04-25T11:23:54.686 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T11:23:54.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.802+0000 7f2832e20640 1 -- 172.21.15.69:0/1730375261 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f282c109a30 msgr2=0x7f282c10be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:54.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.802+0000 7f2832e20640 1 --2- 172.21.15.69:0/1730375261 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f282c109a30 0x7f282c10be50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2824009ca0 tx=0x7f282402a8c0 comp rx=0 tx=0).stop 2024-04-25T11:23:54.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.802+0000 7f2832e20640 1 -- 172.21.15.69:0/1730375261 shutdown_connections 2024-04-25T11:23:54.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.802+0000 7f2832e20640 1 --2- 172.21.15.69:0/1730375261 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f282c109a30 0x7f282c10be50 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:54.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.802+0000 7f2832e20640 1 --2- 172.21.15.69:0/1730375261 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f282c1070d0 0x7f282c1094f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:54.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.802+0000 7f2832e20640 1 --2- 172.21.15.69:0/1730375261 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f282c101f20 0x7f282c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:54.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.802+0000 7f2832e20640 1 -- 172.21.15.69:0/1730375261 >> 172.21.15.69:0/1730375261 conn(0x7f282c0fd380 msgr2=0x7f282c0ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:54.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.802+0000 7f2832e20640 1 -- 172.21.15.69:0/1730375261 shutdown_connections 2024-04-25T11:23:54.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.802+0000 7f2832e20640 1 -- 172.21.15.69:0/1730375261 wait complete. 2024-04-25T11:23:54.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.803+0000 7f2832e20640 1 Processor -- start 2024-04-25T11:23:54.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.803+0000 7f2832e20640 1 -- start start 2024-04-25T11:23:54.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.803+0000 7f2832e20640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f282c101f20 0x7f282c19a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:54.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.803+0000 7f2832e20640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f282c1070d0 0x7f282c19a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:54.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.803+0000 7f2832e20640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f282c109a30 0x7f282c1a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:54.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.803+0000 7f2832e20640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f282c10f5d0 con 0x7f282c109a30 2024-04-25T11:23:54.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.803+0000 7f2832e20640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f282c10f450 con 0x7f282c101f20 2024-04-25T11:23:54.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.803+0000 7f2832e20640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f282c10f750 con 0x7f282c1070d0 2024-04-25T11:23:54.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.803+0000 7f28313bd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f282c109a30 0x7f282c1a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:54.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.803+0000 7f28313bd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f282c109a30 0x7f282c1a1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58352/0 (socket says 172.21.15.69:58352) 2024-04-25T11:23:54.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.803+0000 7f28313bd640 1 -- 172.21.15.69:0/1778531150 learned_addr learned my addr 172.21.15.69:0/1778531150 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:54.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.803+0000 7f282bfff640 1 --2- 172.21.15.69:0/1778531150 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f282c1070d0 0x7f282c19a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:54.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.803+0000 7f2830bbc640 1 --2- 172.21.15.69:0/1778531150 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f282c101f20 0x7f282c19a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:54.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.804+0000 7f28313bd640 1 -- 172.21.15.69:0/1778531150 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f282c1070d0 msgr2=0x7f282c19a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:54.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.804+0000 7f28313bd640 1 --2- 172.21.15.69:0/1778531150 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f282c1070d0 0x7f282c19a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:54.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.804+0000 7f28313bd640 1 -- 172.21.15.69:0/1778531150 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f282c101f20 msgr2=0x7f282c19a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:54.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.804+0000 7f28313bd640 1 --2- 172.21.15.69:0/1778531150 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f282c101f20 0x7f282c19a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:54.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.804+0000 7f28313bd640 1 -- 172.21.15.69:0/1778531150 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f282c1a1cc0 con 0x7f282c109a30 2024-04-25T11:23:54.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.804+0000 7f282bfff640 1 --2- 172.21.15.69:0/1778531150 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f282c1070d0 0x7f282c19a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:23:54.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.804+0000 7f28313bd640 1 --2- 172.21.15.69:0/1778531150 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f282c109a30 0x7f282c1a1650 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2824031040 tx=0x7f2824003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:54.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.804+0000 7f2829ffb640 1 -- 172.21.15.69:0/1778531150 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2824004040 con 0x7f282c109a30 2024-04-25T11:23:54.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.804+0000 7f2829ffb640 1 -- 172.21.15.69:0/1778531150 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2824004670 con 0x7f282c109a30 2024-04-25T11:23:54.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.804+0000 7f2832e20640 1 -- 172.21.15.69:0/1778531150 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f282c1a1f50 con 0x7f282c109a30 2024-04-25T11:23:54.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.804+0000 7f2829ffb640 1 -- 172.21.15.69:0/1778531150 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2824030d50 con 0x7f282c109a30 2024-04-25T11:23:54.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.804+0000 7f2832e20640 1 -- 172.21.15.69:0/1778531150 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f282c1a23b0 con 0x7f282c109a30 2024-04-25T11:23:54.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.805+0000 7f2832e20640 1 -- 172.21.15.69:0/1778531150 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f282c06a190 con 0x7f282c109a30 2024-04-25T11:23:54.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.807+0000 7f2829ffb640 1 -- 172.21.15.69:0/1778531150 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f28240305c0 con 0x7f282c109a30 2024-04-25T11:23:54.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.808+0000 7f2829ffb640 1 --2- 172.21.15.69:0/1778531150 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f281c0afaf0 0x7f281c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:54.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.808+0000 7f2830bbc640 1 --2- 172.21.15.69:0/1778531150 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f281c0afaf0 0x7f281c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:54.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.808+0000 7f2829ffb640 1 -- 172.21.15.69:0/1778531150 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7f2824100620 con 0x7f282c109a30 2024-04-25T11:23:54.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.808+0000 7f2830bbc640 1 --2- 172.21.15.69:0/1778531150 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f281c0afaf0 0x7f281c0b2000 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2820005fd0 tx=0x7f2820005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:54.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.812+0000 7f2829ffb640 1 -- 172.21.15.69:0/1778531150 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f282c06a190 con 0x7f282c109a30 2024-04-25T11:23:54.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:54 smithi069 ceph-mon[29188]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 2 op/s 2024-04-25T11:23:54.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:54 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/629343706' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:23:54.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:54 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2817070295' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:23:54.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:54 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2001358552' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:23:54.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.991+0000 7f2832e20640 1 -- 172.21.15.69:0/1778531150 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f282c02dbd0 con 0x7f282c109a30 2024-04-25T11:23:54.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.991+0000 7f2829ffb640 1 -- 172.21.15.69:0/1778531150 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v80) ==== 103+0+11 (secure 0 0 0) 0x7f28240c8fc0 con 0x7f282c109a30 2024-04-25T11:23:54.986 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-25T11:23:54.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.993+0000 7f2832e20640 1 -- 172.21.15.69:0/1778531150 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f281c0afaf0 msgr2=0x7f281c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:54.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.993+0000 7f2832e20640 1 --2- 172.21.15.69:0/1778531150 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f281c0afaf0 0x7f281c0b2000 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2820005fd0 tx=0x7f2820005850 comp rx=0 tx=0).stop 2024-04-25T11:23:54.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.993+0000 7f2832e20640 1 -- 172.21.15.69:0/1778531150 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f282c109a30 msgr2=0x7f282c1a1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:54.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.993+0000 7f2832e20640 1 --2- 172.21.15.69:0/1778531150 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f282c109a30 0x7f282c1a1650 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2824031040 tx=0x7f2824003ba0 comp rx=0 tx=0).stop 2024-04-25T11:23:54.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.993+0000 7f2832e20640 1 -- 172.21.15.69:0/1778531150 shutdown_connections 2024-04-25T11:23:54.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.993+0000 7f2832e20640 1 --2- 172.21.15.69:0/1778531150 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f281c0afaf0 0x7f281c0b2000 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:54.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.993+0000 7f2832e20640 1 --2- 172.21.15.69:0/1778531150 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f282c109a30 0x7f282c1a1650 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:54.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.993+0000 7f2832e20640 1 --2- 172.21.15.69:0/1778531150 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f282c1070d0 0x7f282c19a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:54.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.993+0000 7f2832e20640 1 --2- 172.21.15.69:0/1778531150 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f282c101f20 0x7f282c19a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:54.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.993+0000 7f2832e20640 1 -- 172.21.15.69:0/1778531150 >> 172.21.15.69:0/1778531150 conn(0x7f282c0fd380 msgr2=0x7f282c1055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:54.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.993+0000 7f2832e20640 1 -- 172.21.15.69:0/1778531150 shutdown_connections 2024-04-25T11:23:54.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:54.993+0000 7f2832e20640 1 -- 172.21.15.69:0/1778531150 wait complete. 2024-04-25T11:23:54.999 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:23:55.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:54 smithi146 ceph-mon[32821]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 2 op/s 2024-04-25T11:23:55.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:54 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/629343706' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:23:55.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:54 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/2817070295' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:23:55.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:54 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/2001358552' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:23:55.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.114+0000 7f883d9f3640 1 -- 172.21.15.69:0/2873967804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8838108920 msgr2=0x7f883810ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:55.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.114+0000 7f883d9f3640 1 --2- 172.21.15.69:0/2873967804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8838108920 0x7f883810ad40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8830009900 tx=0x7f883002a410 comp rx=0 tx=0).stop 2024-04-25T11:23:55.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.114+0000 7f883d9f3640 1 -- 172.21.15.69:0/2873967804 shutdown_connections 2024-04-25T11:23:55.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.114+0000 7f883d9f3640 1 --2- 172.21.15.69:0/2873967804 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8838108920 0x7f883810ad40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:55.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.114+0000 7f883d9f3640 1 --2- 172.21.15.69:0/2873967804 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8838103c20 0x7f8838104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:55.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.114+0000 7f883d9f3640 1 --2- 172.21.15.69:0/2873967804 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f88381029e0 0x7f88381036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:55.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.114+0000 7f883d9f3640 1 -- 172.21.15.69:0/2873967804 >> 172.21.15.69:0/2873967804 conn(0x7f88380fdbd0 msgr2=0x7f8838100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:55.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.114+0000 7f883d9f3640 1 -- 172.21.15.69:0/2873967804 shutdown_connections 2024-04-25T11:23:55.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.114+0000 7f883d9f3640 1 -- 172.21.15.69:0/2873967804 wait complete. 2024-04-25T11:23:55.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.114+0000 7f883d9f3640 1 Processor -- start 2024-04-25T11:23:55.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.115+0000 7f883d9f3640 1 -- start start 2024-04-25T11:23:55.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.115+0000 7f883d9f3640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88381029e0 0x7f8838199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:55.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.115+0000 7f883d9f3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8838103c20 0x7f88381998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:55.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.115+0000 7f883d9f3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8838108920 0x7f88381a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:55.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.115+0000 7f883d9f3640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f883810e3c0 con 0x7f8838103c20 2024-04-25T11:23:55.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.115+0000 7f883d9f3640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f883810e240 con 0x7f8838108920 2024-04-25T11:23:55.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.115+0000 7f883d9f3640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f883810e540 con 0x7f88381029e0 2024-04-25T11:23:55.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.115+0000 7f88367fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8838103c20 0x7f88381998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:55.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.115+0000 7f88367fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8838103c20 0x7f88381998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58356/0 (socket says 172.21.15.69:58356) 2024-04-25T11:23:55.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.115+0000 7f88367fc640 1 -- 172.21.15.69:0/3195797133 learned_addr learned my addr 172.21.15.69:0/3195797133 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:55.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.115+0000 7f8836ffd640 1 --2- 172.21.15.69:0/3195797133 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88381029e0 0x7f8838199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:55.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.115+0000 7f88367fc640 1 -- 172.21.15.69:0/3195797133 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88381029e0 msgr2=0x7f8838199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:55.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.115+0000 7f88377fe640 1 --2- 172.21.15.69:0/3195797133 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8838108920 0x7f88381a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:55.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.115+0000 7f88367fc640 1 --2- 172.21.15.69:0/3195797133 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88381029e0 0x7f8838199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:55.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.115+0000 7f88367fc640 1 -- 172.21.15.69:0/3195797133 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8838108920 msgr2=0x7f88381a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:55.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.115+0000 7f88367fc640 1 --2- 172.21.15.69:0/3195797133 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8838108920 0x7f88381a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:55.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.115+0000 7f88367fc640 1 -- 172.21.15.69:0/3195797133 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88381a0e50 con 0x7f8838103c20 2024-04-25T11:23:55.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.115+0000 7f88377fe640 1 --2- 172.21.15.69:0/3195797133 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8838108920 0x7f88381a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:23:55.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.115+0000 7f88367fc640 1 --2- 172.21.15.69:0/3195797133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8838103c20 0x7f88381998c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f882c00d770 tx=0x7f882c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:55.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.116+0000 7f883c9f1640 1 -- 172.21.15.69:0/3195797133 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882c017070 con 0x7f8838103c20 2024-04-25T11:23:55.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.116+0000 7f883c9f1640 1 -- 172.21.15.69:0/3195797133 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f882c0054b0 con 0x7f8838103c20 2024-04-25T11:23:55.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.116+0000 7f883d9f3640 1 -- 172.21.15.69:0/3195797133 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88381a1140 con 0x7f8838103c20 2024-04-25T11:23:55.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.116+0000 7f883c9f1640 1 -- 172.21.15.69:0/3195797133 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882c004c60 con 0x7f8838103c20 2024-04-25T11:23:55.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.116+0000 7f883d9f3640 1 -- 172.21.15.69:0/3195797133 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88381a1640 con 0x7f8838103c20 2024-04-25T11:23:55.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.117+0000 7f883d9f3640 1 -- 172.21.15.69:0/3195797133 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8838107730 con 0x7f8838103c20 2024-04-25T11:23:55.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.118+0000 7f883c9f1640 1 -- 172.21.15.69:0/3195797133 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f882c004e00 con 0x7f8838103c20 2024-04-25T11:23:55.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.119+0000 7f883c9f1640 1 --2- 172.21.15.69:0/3195797133 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f88200afaf0 0x7f88200b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:55.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.119+0000 7f8836ffd640 1 --2- 172.21.15.69:0/3195797133 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f88200afaf0 0x7f88200b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:55.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.119+0000 7f883c9f1640 1 -- 172.21.15.69:0/3195797133 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7f882c0dedd0 con 0x7f8838103c20 2024-04-25T11:23:55.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.119+0000 7f8836ffd640 1 --2- 172.21.15.69:0/3195797133 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f88200afaf0 0x7f88200b2000 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f88381035e0 tx=0x7f8828009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:55.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.123+0000 7f883c9f1640 1 -- 172.21.15.69:0/3195797133 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f882c0a7770 con 0x7f8838103c20 2024-04-25T11:23:55.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:54 smithi152 ceph-mon[32701]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 2 op/s 2024-04-25T11:23:55.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:54 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/629343706' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:23:55.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:54 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/2817070295' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:23:55.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:54 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/2001358552' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:23:55.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.311+0000 7f883d9f3640 1 -- 172.21.15.69:0/3195797133 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f883806c2a0 con 0x7f8838103c20 2024-04-25T11:23:55.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.311+0000 7f883c9f1640 1 -- 172.21.15.69:0/3195797133 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v80) ==== 81+0+592 (secure 0 0 0) 0x7f882c0a7070 con 0x7f8838103c20 2024-04-25T11:23:55.307 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/1372727546 2024-04-26T11:16:31.688159+0000 2024-04-25T11:23:55.307 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6801/1097124842 2024-04-26T11:16:31.688159+0000 2024-04-25T11:23:55.307 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6800/1508198215 2024-04-26T11:16:04.860792+0000 2024-04-25T11:23:55.307 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/3103465990 2024-04-26T11:16:04.860792+0000 2024-04-25T11:23:55.307 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/3540132960 2024-04-26T11:16:04.860792+0000 2024-04-25T11:23:55.307 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/3899912190 2024-04-26T11:16:31.688159+0000 2024-04-25T11:23:55.307 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6800/1097124842 2024-04-26T11:16:31.688159+0000 2024-04-25T11:23:55.307 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/1199907202 2024-04-26T11:16:04.860792+0000 2024-04-25T11:23:55.307 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6801/1508198215 2024-04-26T11:16:04.860792+0000 2024-04-25T11:23:55.307 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/2879102529 2024-04-26T11:16:31.688159+0000 2024-04-25T11:23:55.308 INFO:teuthology.orchestra.run.smithi069.stderr:listed 10 entries 2024-04-25T11:23:55.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.313+0000 7f883d9f3640 1 -- 172.21.15.69:0/3195797133 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f88200afaf0 msgr2=0x7f88200b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:55.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.313+0000 7f883d9f3640 1 --2- 172.21.15.69:0/3195797133 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f88200afaf0 0x7f88200b2000 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f88381035e0 tx=0x7f8828009870 comp rx=0 tx=0).stop 2024-04-25T11:23:55.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.313+0000 7f883d9f3640 1 -- 172.21.15.69:0/3195797133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8838103c20 msgr2=0x7f88381998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:55.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.313+0000 7f883d9f3640 1 --2- 172.21.15.69:0/3195797133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8838103c20 0x7f88381998c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f882c00d770 tx=0x7f882c00dc30 comp rx=0 tx=0).stop 2024-04-25T11:23:55.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.314+0000 7f883d9f3640 1 -- 172.21.15.69:0/3195797133 shutdown_connections 2024-04-25T11:23:55.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.314+0000 7f883d9f3640 1 --2- 172.21.15.69:0/3195797133 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f88200afaf0 0x7f88200b2000 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:55.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.314+0000 7f883d9f3640 1 --2- 172.21.15.69:0/3195797133 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8838108920 0x7f88381a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:55.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.314+0000 7f883d9f3640 1 --2- 172.21.15.69:0/3195797133 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8838103c20 0x7f88381998c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:55.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.314+0000 7f883d9f3640 1 --2- 172.21.15.69:0/3195797133 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88381029e0 0x7f8838199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:55.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.314+0000 7f883d9f3640 1 -- 172.21.15.69:0/3195797133 >> 172.21.15.69:0/3195797133 conn(0x7f88380fdbd0 msgr2=0x7f8838104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:55.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.314+0000 7f883d9f3640 1 -- 172.21.15.69:0/3195797133 shutdown_connections 2024-04-25T11:23:55.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.314+0000 7f883d9f3640 1 -- 172.21.15.69:0/3195797133 wait complete. 2024-04-25T11:23:55.320 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T11:23:55.320 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:23:55.336 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:23:55.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.488+0000 7f79a563c640 1 -- 172.21.15.69:0/40682747 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79a0109a50 msgr2=0x7f79a010be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:55.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.488+0000 7f79a563c640 1 --2- 172.21.15.69:0/40682747 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79a0109a50 0x7f79a010be70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7998009ca0 tx=0x7f799802a8c0 comp rx=0 tx=0).stop 2024-04-25T11:23:55.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.488+0000 7f79a563c640 1 -- 172.21.15.69:0/40682747 shutdown_connections 2024-04-25T11:23:55.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.488+0000 7f79a563c640 1 --2- 172.21.15.69:0/40682747 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79a0109a50 0x7f79a010be70 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:55.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.488+0000 7f79a563c640 1 --2- 172.21.15.69:0/40682747 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79a0104d50 0x7f79a0105160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:55.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.488+0000 7f79a563c640 1 --2- 172.21.15.69:0/40682747 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79a0103b10 0x7f79a0104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:55.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.488+0000 7f79a563c640 1 -- 172.21.15.69:0/40682747 >> 172.21.15.69:0/40682747 conn(0x7f79a00fed40 msgr2=0x7f79a01011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:55.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.488+0000 7f79a563c640 1 -- 172.21.15.69:0/40682747 shutdown_connections 2024-04-25T11:23:55.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.488+0000 7f79a563c640 1 -- 172.21.15.69:0/40682747 wait complete. 2024-04-25T11:23:55.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.489+0000 7f79a563c640 1 Processor -- start 2024-04-25T11:23:55.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.489+0000 7f79a563c640 1 -- start start 2024-04-25T11:23:55.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.489+0000 7f79a563c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79a0103b10 0x7f79a019a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:55.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.489+0000 7f79a563c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79a0104d50 0x7f79a019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:55.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.489+0000 7f79a563c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79a0109a50 0x7f79a01a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:55.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.489+0000 7f79a563c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f79a010f520 con 0x7f79a0109a50 2024-04-25T11:23:55.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.489+0000 7f79a563c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f79a010f3a0 con 0x7f79a0104d50 2024-04-25T11:23:55.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.489+0000 7f79a563c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f79a010f6a0 con 0x7f79a0103b10 2024-04-25T11:23:55.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.489+0000 7f799f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79a0109a50 0x7f79a01a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:55.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.489+0000 7f799f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79a0109a50 0x7f79a01a15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58378/0 (socket says 172.21.15.69:58378) 2024-04-25T11:23:55.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.489+0000 7f799f7fe640 1 -- 172.21.15.69:0/3384948588 learned_addr learned my addr 172.21.15.69:0/3384948588 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:55.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.489+0000 7f7997fff640 1 --2- 172.21.15.69:0/3384948588 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79a0104d50 0x7f79a019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:55.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.490+0000 7f799f7fe640 1 -- 172.21.15.69:0/3384948588 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79a0103b10 msgr2=0x7f79a019a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:55.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.490+0000 7f799effd640 1 --2- 172.21.15.69:0/3384948588 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79a0103b10 0x7f79a019a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:55.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.490+0000 7f799f7fe640 1 --2- 172.21.15.69:0/3384948588 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79a0103b10 0x7f79a019a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:55.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.490+0000 7f799f7fe640 1 -- 172.21.15.69:0/3384948588 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79a0104d50 msgr2=0x7f79a019a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:55.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.490+0000 7f799f7fe640 1 --2- 172.21.15.69:0/3384948588 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79a0104d50 0x7f79a019a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:55.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.490+0000 7f799f7fe640 1 -- 172.21.15.69:0/3384948588 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a01a1b90 con 0x7f79a0109a50 2024-04-25T11:23:55.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.490+0000 7f7997fff640 1 --2- 172.21.15.69:0/3384948588 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79a0104d50 0x7f79a019a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:23:55.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.490+0000 7f799effd640 1 --2- 172.21.15.69:0/3384948588 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79a0103b10 0x7f79a019a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:23:55.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.490+0000 7f799f7fe640 1 --2- 172.21.15.69:0/3384948588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79a0109a50 0x7f79a01a15d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7998031040 tx=0x7f7998003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:55.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.490+0000 7f799cff9640 1 -- 172.21.15.69:0/3384948588 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7998004040 con 0x7f79a0109a50 2024-04-25T11:23:55.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.490+0000 7f799cff9640 1 -- 172.21.15.69:0/3384948588 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7998004670 con 0x7f79a0109a50 2024-04-25T11:23:55.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.490+0000 7f79a563c640 1 -- 172.21.15.69:0/3384948588 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79a01a1e20 con 0x7f79a0109a50 2024-04-25T11:23:55.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.490+0000 7f799cff9640 1 -- 172.21.15.69:0/3384948588 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7998030d50 con 0x7f79a0109a50 2024-04-25T11:23:55.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.490+0000 7f79a563c640 1 -- 172.21.15.69:0/3384948588 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79a01a2280 con 0x7f79a0109a50 2024-04-25T11:23:55.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.491+0000 7f79a563c640 1 -- 172.21.15.69:0/3384948588 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79a006a190 con 0x7f79a0109a50 2024-04-25T11:23:55.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.492+0000 7f799cff9640 1 -- 172.21.15.69:0/3384948588 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f79980305c0 con 0x7f79a0109a50 2024-04-25T11:23:55.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.493+0000 7f799cff9640 1 --2- 172.21.15.69:0/3384948588 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f798c0afaf0 0x7f798c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:55.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.493+0000 7f799effd640 1 --2- 172.21.15.69:0/3384948588 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f798c0afaf0 0x7f798c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:55.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.493+0000 7f799cff9640 1 -- 172.21.15.69:0/3384948588 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7f7998100610 con 0x7f79a0109a50 2024-04-25T11:23:55.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.494+0000 7f799effd640 1 --2- 172.21.15.69:0/3384948588 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f798c0afaf0 0x7f798c0b2000 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f79a0104710 tx=0x7f7990005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:55.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.498+0000 7f799cff9640 1 -- 172.21.15.69:0/3384948588 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f79980c8fb0 con 0x7f79a0109a50 2024-04-25T11:23:55.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.676+0000 7f79a563c640 1 -- 172.21.15.69:0/3384948588 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f79a00637a0 con 0x7f79a0109a50 2024-04-25T11:23:55.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.678+0000 7f799cff9640 1 -- 172.21.15.69:0/3384948588 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+24130 (secure 0 0 0) 0x7f79980c88b0 con 0x7f79a0109a50 2024-04-25T11:23:55.672 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:23:55.673 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":80,"fsid":"f7ed73dc-02f4-11ef-bc93-c7b262605968","created":"2024-04-25T11:15:37.965281+0000","modified":"2024-04-25T11:23:47.662955+0000","last_up_change":"2024-04-25T11:22:50.691869+0000","last_in_change":"2024-04-25T11:22:30.271959+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:18:57.781263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:23:38.685110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:23:39.943803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T11:23:43.650521+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6a0d179-4998-4754-bf82-2ea3dac54e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6803","nonce":3756025235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6805","nonce":3756025235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6809","nonce":3756025235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6807","nonce":3756025235}]},"public_addr":"172.21.15.69:6803/3756025235","cluster_addr":"172.21.15.69:6805/3756025235","heartbeat_back_addr":"172.21.15.69:6809/3756025235","heartbeat_front_addr":"172.21.15.69:6807/3756025235","state":["exists","up"]},{"osd":1,"uuid":"c0c66930-4233-493f-b2bb-99c473994b59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6801","nonce":3166305213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6803","nonce":3166305213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6807","nonce":3166305213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6805","nonce":3166305213}]},"public_addr":"172.21.15.146:6801/3166305213","cluster_addr":"172.21.15.146:6803/3166305213","heartbeat_back_addr":"172.21.15.146:6807/3166305213","heartbeat_front_addr":"172.21.15.146:6805/3166305213","state":["exists","up"]},{"osd":2,"uuid":"d4ef735b-9786-4cb9-83ef-7c3f0691a729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6801","nonce":1474319175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6803","nonce":1474319175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6807","nonce":1474319175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6805","nonce":1474319175}]},"public_addr":"172.21.15.152:6801/1474319175","cluster_addr":"172.21.15.152:6803/1474319175","heartbeat_back_addr":"172.21.15.152:6807/1474319175","heartbeat_front_addr":"172.21.15.152:6805/1474319175","state":["exists","up"]},{"osd":3,"uuid":"0ab85fb8-5bbb-4ffc-bddf-cae7644a64ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6811","nonce":1809071296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6813","nonce":1809071296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6817","nonce":1809071296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6815","nonce":1809071296}]},"public_addr":"172.21.15.69:6811/1809071296","cluster_addr":"172.21.15.69:6813/1809071296","heartbeat_back_addr":"172.21.15.69:6817/1809071296","heartbeat_front_addr":"172.21.15.69:6815/1809071296","state":["exists","up"]},{"osd":4,"uuid":"273e0592-bbbc-4188-8eeb-46a3879cf632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6809","nonce":1898520972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6811","nonce":1898520972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6815","nonce":1898520972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6813","nonce":1898520972}]},"public_addr":"172.21.15.146:6809/1898520972","cluster_addr":"172.21.15.146:6811/1898520972","heartbeat_back_addr":"172.21.15.146:6815/1898520972","heartbeat_front_addr":"172.21.15.146:6813/1898520972","state":["exists","up"]},{"osd":5,"uuid":"bbeae945-9fa2-48f9-a1d6-8bf18dcaef19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6809","nonce":1996555108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6811","nonce":1996555108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6815","nonce":1996555108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6813","nonce":1996555108}]},"public_addr":"172.21.15.152:6809/1996555108","cluster_addr":"172.21.15.152:6811/1996555108","heartbeat_back_addr":"172.21.15.152:6815/1996555108","heartbeat_front_addr":"172.21.15.152:6813/1996555108","state":["exists","up"]},{"osd":6,"uuid":"dbb61253-30bc-418e-bd75-118c80bbf931","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6819","nonce":491215762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6821","nonce":491215762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6825","nonce":491215762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6823","nonce":491215762}]},"public_addr":"172.21.15.69:6819/491215762","cluster_addr":"172.21.15.69:6821/491215762","heartbeat_back_addr":"172.21.15.69:6825/491215762","heartbeat_front_addr":"172.21.15.69:6823/491215762","state":["exists","up"]},{"osd":7,"uuid":"75f23854-e283-4e3e-94cf-7b4079d83876","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6817","nonce":2330294690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6819","nonce":2330294690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6823","nonce":2330294690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6821","nonce":2330294690}]},"public_addr":"172.21.15.146:6817/2330294690","cluster_addr":"172.21.15.146:6819/2330294690","heartbeat_back_addr":"172.21.15.146:6823/2330294690","heartbeat_front_addr":"172.21.15.146:6821/2330294690","state":["exists","up"]},{"osd":8,"uuid":"1aa1060c-0c14-4abd-be56-5ebfeb78759c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6817","nonce":950669381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6819","nonce":950669381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6823","nonce":950669381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6821","nonce":950669381}]},"public_addr":"172.21.15.152:6817/950669381","cluster_addr":"172.21.15.152:6819/950669381","heartbeat_back_addr":"172.21.15.152:6823/950669381","heartbeat_front_addr":"172.21.15.152:6821/950669381","state":["exists","up"]},{"osd":9,"uuid":"3f4bf800-4090-4138-9bb3-d001f13f450c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6827","nonce":2414186758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6829","nonce":2414186758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6833","nonce":2414186758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6831","nonce":2414186758}]},"public_addr":"172.21.15.69:6827/2414186758","cluster_addr":"172.21.15.69:6829/2414186758","heartbeat_back_addr":"172.21.15.69:6833/2414186758","heartbeat_front_addr":"172.21.15.69:6831/2414186758","state":["exists","up"]},{"osd":10,"uuid":"19e967e5-51ce-4506-80c9-0a95441baf99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6825","nonce":707256400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6827","nonce":707256400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6831","nonce":707256400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6829","nonce":707256400}]},"public_addr":"172.21.15.146:6825/707256400","cluster_addr":"172.21.15.146:6827/707256400","heartbeat_back_addr":"172.21.15.146:6831/707256400","heartbeat_front_addr":"172.21.15.146:6829/707256400","state":["exists","up"]},{"osd":11,"uuid":"7788c2de-6930-4808-be0f-f1fd1ae0cb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6825","nonce":2934336833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6827","nonce":2934336833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6831","nonce":2934336833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6829","nonce":2934336833}]},"public_addr":"172.21.15.152:6825/2934336833","cluster_addr":"172.21.15.152:6827/2934336833","heartbeat_back_addr":"172.21.15.152:6831/2934336833","heartbeat_front_addr":"172.21.15.152:6829/2934336833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:07.497448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:31.227494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:55.027978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:20.713437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:46.214704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:20:11.199983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:20:37.177070+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:02.951240+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:28.963680+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:56.095807+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:22:22.221744+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:22:48.929334+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/1372727546":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6801/1097124842":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6800/1508198215":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3103465990":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3540132960":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3899912190":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6800/1097124842":"2024-04-26T11:16:31.688159+0000","172.21.15.69:0/1199907202":"2024-04-26T11:16:04.860792+0000","172.21.15.69:6801/1508198215":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/2879102529":"2024-04-26T11:16:31.688159+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:23:55.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.680+0000 7f79a563c640 1 -- 172.21.15.69:0/3384948588 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f798c0afaf0 msgr2=0x7f798c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:55.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.680+0000 7f79a563c640 1 --2- 172.21.15.69:0/3384948588 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f798c0afaf0 0x7f798c0b2000 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f79a0104710 tx=0x7f7990005db0 comp rx=0 tx=0).stop 2024-04-25T11:23:55.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.680+0000 7f79a563c640 1 -- 172.21.15.69:0/3384948588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79a0109a50 msgr2=0x7f79a01a15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:55.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.680+0000 7f79a563c640 1 --2- 172.21.15.69:0/3384948588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79a0109a50 0x7f79a01a15d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7998031040 tx=0x7f7998003ba0 comp rx=0 tx=0).stop 2024-04-25T11:23:55.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.680+0000 7f79a563c640 1 -- 172.21.15.69:0/3384948588 shutdown_connections 2024-04-25T11:23:55.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.680+0000 7f79a563c640 1 --2- 172.21.15.69:0/3384948588 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f798c0afaf0 0x7f798c0b2000 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:55.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.680+0000 7f79a563c640 1 --2- 172.21.15.69:0/3384948588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79a0109a50 0x7f79a01a15d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:55.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.680+0000 7f79a563c640 1 --2- 172.21.15.69:0/3384948588 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f79a0104d50 0x7f79a019a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:55.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.680+0000 7f79a563c640 1 --2- 172.21.15.69:0/3384948588 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79a0103b10 0x7f79a019a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:55.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.680+0000 7f79a563c640 1 -- 172.21.15.69:0/3384948588 >> 172.21.15.69:0/3384948588 conn(0x7f79a00fed40 msgr2=0x7f79a0105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:55.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.680+0000 7f79a563c640 1 -- 172.21.15.69:0/3384948588 shutdown_connections 2024-04-25T11:23:55.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.680+0000 7f79a563c640 1 -- 172.21.15.69:0/3384948588 wait complete. 2024-04-25T11:23:55.686 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:18:57.781263+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:23:38.685110+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:23:39.943803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T11:23:43.650521+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T11:23:55.686 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:23:55.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.802+0000 7f0153a34640 1 -- 172.21.15.69:0/659768740 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f014c103af0 msgr2=0x7f014c1047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:55.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.802+0000 7f0153a34640 1 --2- 172.21.15.69:0/659768740 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f014c103af0 0x7f014c1047f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0148009900 tx=0x7f014802a410 comp rx=0 tx=0).stop 2024-04-25T11:23:55.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.803+0000 7f0153a34640 1 -- 172.21.15.69:0/659768740 shutdown_connections 2024-04-25T11:23:55.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.803+0000 7f0153a34640 1 --2- 172.21.15.69:0/659768740 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f014c109a30 0x7f014c10be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:55.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.803+0000 7f0153a34640 1 --2- 172.21.15.69:0/659768740 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f014c104d30 0x7f014c105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:55.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.803+0000 7f0153a34640 1 --2- 172.21.15.69:0/659768740 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f014c103af0 0x7f014c1047f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:55.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.803+0000 7f0153a34640 1 -- 172.21.15.69:0/659768740 >> 172.21.15.69:0/659768740 conn(0x7f014c0fece0 msgr2=0x7f014c101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:55.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.803+0000 7f0153a34640 1 -- 172.21.15.69:0/659768740 shutdown_connections 2024-04-25T11:23:55.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.803+0000 7f0153a34640 1 -- 172.21.15.69:0/659768740 wait complete. 2024-04-25T11:23:55.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.803+0000 7f0153a34640 1 Processor -- start 2024-04-25T11:23:55.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.803+0000 7f0153a34640 1 -- start start 2024-04-25T11:23:55.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.803+0000 7f0153a34640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f014c103af0 0x7f014c19a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:55.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.803+0000 7f0153a34640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f014c104d30 0x7f014c19a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:55.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.803+0000 7f0153a34640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f014c109a30 0x7f014c1a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:55.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.803+0000 7f0153a34640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f014c10f4a0 con 0x7f014c103af0 2024-04-25T11:23:55.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.803+0000 7f0153a34640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f014c10f320 con 0x7f014c104d30 2024-04-25T11:23:55.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.803+0000 7f0153a34640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f014c10f620 con 0x7f014c109a30 2024-04-25T11:23:55.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.804+0000 7f01517d0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f014c103af0 0x7f014c19a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:55.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.804+0000 7f01517d0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f014c103af0 0x7f014c19a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58394/0 (socket says 172.21.15.69:58394) 2024-04-25T11:23:55.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.804+0000 7f01517d0640 1 -- 172.21.15.69:0/1682417289 learned_addr learned my addr 172.21.15.69:0/1682417289 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:55.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.804+0000 7f0150fcf640 1 --2- 172.21.15.69:0/1682417289 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f014c104d30 0x7f014c19a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:55.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.804+0000 7f0151fd1640 1 --2- 172.21.15.69:0/1682417289 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f014c109a30 0x7f014c1a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:55.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.804+0000 7f01517d0640 1 -- 172.21.15.69:0/1682417289 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f014c109a30 msgr2=0x7f014c1a1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:55.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.804+0000 7f01517d0640 1 --2- 172.21.15.69:0/1682417289 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f014c109a30 0x7f014c1a1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:55.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.804+0000 7f01517d0640 1 -- 172.21.15.69:0/1682417289 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f014c104d30 msgr2=0x7f014c19a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:55.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.804+0000 7f01517d0640 1 --2- 172.21.15.69:0/1682417289 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f014c104d30 0x7f014c19a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:55.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.804+0000 7f01517d0640 1 -- 172.21.15.69:0/1682417289 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f014c1a1ca0 con 0x7f014c103af0 2024-04-25T11:23:55.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.804+0000 7f01517d0640 1 --2- 172.21.15.69:0/1682417289 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f014c103af0 0x7f014c19a160 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0148005760 tx=0x7f0148004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:55.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.804+0000 7f01427fc640 1 -- 172.21.15.69:0/1682417289 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014803f030 con 0x7f014c103af0 2024-04-25T11:23:55.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.804+0000 7f0153a34640 1 -- 172.21.15.69:0/1682417289 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f014c1a1f30 con 0x7f014c103af0 2024-04-25T11:23:55.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.804+0000 7f01427fc640 1 -- 172.21.15.69:0/1682417289 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0148003a80 con 0x7f014c103af0 2024-04-25T11:23:55.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.804+0000 7f01427fc640 1 -- 172.21.15.69:0/1682417289 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0148035070 con 0x7f014c103af0 2024-04-25T11:23:55.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.804+0000 7f0153a34640 1 -- 172.21.15.69:0/1682417289 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f014c1a2330 con 0x7f014c103af0 2024-04-25T11:23:55.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.805+0000 7f0153a34640 1 -- 172.21.15.69:0/1682417289 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f014c108840 con 0x7f014c103af0 2024-04-25T11:23:55.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.807+0000 7f01427fc640 1 -- 172.21.15.69:0/1682417289 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f0148042030 con 0x7f014c103af0 2024-04-25T11:23:55.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.807+0000 7f01427fc640 1 --2- 172.21.15.69:0/1682417289 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f01380afaf0 0x7f01380b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:55.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.807+0000 7f0150fcf640 1 --2- 172.21.15.69:0/1682417289 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f01380afaf0 0x7f01380b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:55.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.807+0000 7f01427fc640 1 -- 172.21.15.69:0/1682417289 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7f01480ff960 con 0x7f014c103af0 2024-04-25T11:23:55.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.808+0000 7f0150fcf640 1 --2- 172.21.15.69:0/1682417289 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f01380afaf0 0x7f01380b2000 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f014c19b200 tx=0x7f013c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:55.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.811+0000 7f01427fc640 1 -- 172.21.15.69:0/1682417289 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f01480c8300 con 0x7f014c103af0 2024-04-25T11:23:55.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:55 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/1778531150' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T11:23:55.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:55 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/3195797133' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T11:23:55.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:55 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/3384948588' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:23:55.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.991+0000 7f0153a34640 1 -- 172.21.15.69:0/1682417289 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f014c02dce0 con 0x7f014c103af0 2024-04-25T11:23:55.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.991+0000 7f01427fc640 1 -- 172.21.15.69:0/1682417289 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) ==== 93+0+10 (secure 0 0 0) 0x7f01480d0380 con 0x7f014c103af0 2024-04-25T11:23:55.986 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-04-25T11:23:55.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.993+0000 7f0153a34640 1 -- 172.21.15.69:0/1682417289 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f01380afaf0 msgr2=0x7f01380b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:55.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.993+0000 7f0153a34640 1 --2- 172.21.15.69:0/1682417289 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f01380afaf0 0x7f01380b2000 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f014c19b200 tx=0x7f013c00b390 comp rx=0 tx=0).stop 2024-04-25T11:23:55.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.993+0000 7f0153a34640 1 -- 172.21.15.69:0/1682417289 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f014c103af0 msgr2=0x7f014c19a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:55.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.993+0000 7f0153a34640 1 --2- 172.21.15.69:0/1682417289 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f014c103af0 0x7f014c19a160 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0148005760 tx=0x7f0148004270 comp rx=0 tx=0).stop 2024-04-25T11:23:55.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.993+0000 7f0153a34640 1 -- 172.21.15.69:0/1682417289 shutdown_connections 2024-04-25T11:23:55.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.993+0000 7f0153a34640 1 --2- 172.21.15.69:0/1682417289 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f01380afaf0 0x7f01380b2000 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:55.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.993+0000 7f0153a34640 1 --2- 172.21.15.69:0/1682417289 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f014c109a30 0x7f014c1a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:55.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.993+0000 7f0153a34640 1 --2- 172.21.15.69:0/1682417289 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f014c104d30 0x7f014c19a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:55.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.993+0000 7f0153a34640 1 --2- 172.21.15.69:0/1682417289 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f014c103af0 0x7f014c19a160 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:55.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.993+0000 7f0153a34640 1 -- 172.21.15.69:0/1682417289 >> 172.21.15.69:0/1682417289 conn(0x7f014c0fece0 msgr2=0x7f014c105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:55.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.993+0000 7f0153a34640 1 -- 172.21.15.69:0/1682417289 shutdown_connections 2024-04-25T11:23:55.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:55.993+0000 7f0153a34640 1 -- 172.21.15.69:0/1682417289 wait complete. 2024-04-25T11:23:55.998 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:23:56.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:55 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/1778531150' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T11:23:56.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:55 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/3195797133' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T11:23:56.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:55 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/3384948588' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:23:56.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.115+0000 7ffb5da84640 1 -- 172.21.15.69:0/402336148 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb58102a00 msgr2=0x7ffb58103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:56.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.115+0000 7ffb5da84640 1 --2- 172.21.15.69:0/402336148 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb58102a00 0x7ffb58103700 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ffb48009900 tx=0x7ffb4802a410 comp rx=0 tx=0).stop 2024-04-25T11:23:56.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.115+0000 7ffb5da84640 1 -- 172.21.15.69:0/402336148 shutdown_connections 2024-04-25T11:23:56.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.115+0000 7ffb5da84640 1 --2- 172.21.15.69:0/402336148 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb58108940 0x7ffb5810ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:56.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.115+0000 7ffb5da84640 1 --2- 172.21.15.69:0/402336148 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb58103c40 0x7ffb58104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:56.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.115+0000 7ffb5da84640 1 --2- 172.21.15.69:0/402336148 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb58102a00 0x7ffb58103700 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:56.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.115+0000 7ffb5da84640 1 -- 172.21.15.69:0/402336148 >> 172.21.15.69:0/402336148 conn(0x7ffb580fdbf0 msgr2=0x7ffb581000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:56.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.115+0000 7ffb5da84640 1 -- 172.21.15.69:0/402336148 shutdown_connections 2024-04-25T11:23:56.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.115+0000 7ffb5da84640 1 -- 172.21.15.69:0/402336148 wait complete. 2024-04-25T11:23:56.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.116+0000 7ffb5da84640 1 Processor -- start 2024-04-25T11:23:56.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.116+0000 7ffb5da84640 1 -- start start 2024-04-25T11:23:56.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.116+0000 7ffb5da84640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb58102a00 0x7ffb581990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:56.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.116+0000 7ffb5da84640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb58103c40 0x7ffb581995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:56.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.116+0000 7ffb5da84640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb58108940 0x7ffb581a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:56.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.116+0000 7ffb5da84640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ffb5810e3b0 con 0x7ffb58108940 2024-04-25T11:23:56.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.116+0000 7ffb5da84640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ffb5810e230 con 0x7ffb58103c40 2024-04-25T11:23:56.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.116+0000 7ffb5da84640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ffb5810e530 con 0x7ffb58102a00 2024-04-25T11:23:56.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.117+0000 7ffb57fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb58108940 0x7ffb581a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:56.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.117+0000 7ffb57fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb58108940 0x7ffb581a0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58424/0 (socket says 172.21.15.69:58424) 2024-04-25T11:23:56.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.117+0000 7ffb57fff640 1 -- 172.21.15.69:0/129481838 learned_addr learned my addr 172.21.15.69:0/129481838 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:56.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.117+0000 7ffb56ffd640 1 --2- 172.21.15.69:0/129481838 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb58103c40 0x7ffb581995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:56.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.117+0000 7ffb577fe640 1 --2- 172.21.15.69:0/129481838 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb58102a00 0x7ffb581990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:56.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.117+0000 7ffb57fff640 1 -- 172.21.15.69:0/129481838 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb58102a00 msgr2=0x7ffb581990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:56.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.117+0000 7ffb57fff640 1 --2- 172.21.15.69:0/129481838 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb58102a00 0x7ffb581990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:56.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.117+0000 7ffb57fff640 1 -- 172.21.15.69:0/129481838 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb58103c40 msgr2=0x7ffb581995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:56.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.117+0000 7ffb57fff640 1 --2- 172.21.15.69:0/129481838 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb58103c40 0x7ffb581995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:56.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.117+0000 7ffb57fff640 1 -- 172.21.15.69:0/129481838 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb581a0b30 con 0x7ffb58108940 2024-04-25T11:23:56.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.117+0000 7ffb57fff640 1 --2- 172.21.15.69:0/129481838 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb58108940 0x7ffb581a0570 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ffb5000cce0 tx=0x7ffb5000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:56.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.117+0000 7ffb54ff9640 1 -- 172.21.15.69:0/129481838 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5000f450 con 0x7ffb58108940 2024-04-25T11:23:56.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.117+0000 7ffb5da84640 1 -- 172.21.15.69:0/129481838 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb581a0e20 con 0x7ffb58108940 2024-04-25T11:23:56.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.117+0000 7ffb54ff9640 1 -- 172.21.15.69:0/129481838 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffb5000fa80 con 0x7ffb58108940 2024-04-25T11:23:56.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.117+0000 7ffb54ff9640 1 -- 172.21.15.69:0/129481838 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb50018ad0 con 0x7ffb58108940 2024-04-25T11:23:56.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.117+0000 7ffb5da84640 1 -- 172.21.15.69:0/129481838 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb581a1230 con 0x7ffb58108940 2024-04-25T11:23:56.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.118+0000 7ffb5da84640 1 -- 172.21.15.69:0/129481838 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb58107750 con 0x7ffb58108940 2024-04-25T11:23:56.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.120+0000 7ffb54ff9640 1 -- 172.21.15.69:0/129481838 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7ffb5000f5f0 con 0x7ffb58108940 2024-04-25T11:23:56.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.120+0000 7ffb54ff9640 1 --2- 172.21.15.69:0/129481838 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ffb400afaf0 0x7ffb400b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:56.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.120+0000 7ffb577fe640 1 --2- 172.21.15.69:0/129481838 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ffb400afaf0 0x7ffb400b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:56.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.120+0000 7ffb54ff9640 1 -- 172.21.15.69:0/129481838 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7ffb500df660 con 0x7ffb58108940 2024-04-25T11:23:56.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.121+0000 7ffb577fe640 1 --2- 172.21.15.69:0/129481838 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ffb400afaf0 0x7ffb400b2000 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ffb480057c0 tx=0x7ffb48005720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:56.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.124+0000 7ffb54ff9640 1 -- 172.21.15.69:0/129481838 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffb50014030 con 0x7ffb58108940 2024-04-25T11:23:56.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:55 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/1778531150' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T11:23:56.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:55 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/3195797133' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T11:23:56.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:55 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/3384948588' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:23:56.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.299+0000 7ffb5da84640 1 -- 172.21.15.69:0/129481838 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ffb5802dda0 con 0x7ffb58108940 2024-04-25T11:23:56.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.300+0000 7ffb54ff9640 1 -- 172.21.15.69:0/129481838 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) ==== 104+0+11 (secure 0 0 0) 0x7ffb500a8090 con 0x7ffb58108940 2024-04-25T11:23:56.294 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-25T11:23:56.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.301+0000 7ffb5da84640 1 -- 172.21.15.69:0/129481838 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ffb400afaf0 msgr2=0x7ffb400b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:56.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.301+0000 7ffb5da84640 1 --2- 172.21.15.69:0/129481838 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ffb400afaf0 0x7ffb400b2000 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ffb480057c0 tx=0x7ffb48005720 comp rx=0 tx=0).stop 2024-04-25T11:23:56.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.301+0000 7ffb5da84640 1 -- 172.21.15.69:0/129481838 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb58108940 msgr2=0x7ffb581a0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:56.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.301+0000 7ffb5da84640 1 --2- 172.21.15.69:0/129481838 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb58108940 0x7ffb581a0570 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ffb5000cce0 tx=0x7ffb5000a620 comp rx=0 tx=0).stop 2024-04-25T11:23:56.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.302+0000 7ffb5da84640 1 -- 172.21.15.69:0/129481838 shutdown_connections 2024-04-25T11:23:56.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.302+0000 7ffb5da84640 1 --2- 172.21.15.69:0/129481838 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ffb400afaf0 0x7ffb400b2000 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:56.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.302+0000 7ffb5da84640 1 --2- 172.21.15.69:0/129481838 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb58108940 0x7ffb581a0570 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:56.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.302+0000 7ffb5da84640 1 --2- 172.21.15.69:0/129481838 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffb58103c40 0x7ffb581995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:56.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.302+0000 7ffb5da84640 1 --2- 172.21.15.69:0/129481838 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb58102a00 0x7ffb581990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:56.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.302+0000 7ffb5da84640 1 -- 172.21.15.69:0/129481838 >> 172.21.15.69:0/129481838 conn(0x7ffb580fdbf0 msgr2=0x7ffb58104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:56.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.302+0000 7ffb5da84640 1 -- 172.21.15.69:0/129481838 shutdown_connections 2024-04-25T11:23:56.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.302+0000 7ffb5da84640 1 -- 172.21.15.69:0/129481838 wait complete. 2024-04-25T11:23:56.306 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:23:56.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.423+0000 7f90a92f9640 1 -- 172.21.15.69:0/4025734055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90a4101f20 msgr2=0x7f90a4106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:56.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.423+0000 7f90a92f9640 1 --2- 172.21.15.69:0/4025734055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90a4101f20 0x7f90a4106b90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9094009900 tx=0x7f909402a410 comp rx=0 tx=0).stop 2024-04-25T11:23:56.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.423+0000 7f90a92f9640 1 -- 172.21.15.69:0/4025734055 shutdown_connections 2024-04-25T11:23:56.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.423+0000 7f90a92f9640 1 --2- 172.21.15.69:0/4025734055 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90a4109a30 0x7f90a410be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:56.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.423+0000 7f90a92f9640 1 --2- 172.21.15.69:0/4025734055 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90a41070d0 0x7f90a41094f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:56.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.423+0000 7f90a92f9640 1 --2- 172.21.15.69:0/4025734055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90a4101f20 0x7f90a4106b90 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:56.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.423+0000 7f90a92f9640 1 -- 172.21.15.69:0/4025734055 >> 172.21.15.69:0/4025734055 conn(0x7f90a40fd380 msgr2=0x7f90a40ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:56.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.424+0000 7f90a92f9640 1 -- 172.21.15.69:0/4025734055 shutdown_connections 2024-04-25T11:23:56.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.424+0000 7f90a92f9640 1 -- 172.21.15.69:0/4025734055 wait complete. 2024-04-25T11:23:56.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.424+0000 7f90a92f9640 1 Processor -- start 2024-04-25T11:23:56.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.424+0000 7f90a92f9640 1 -- start start 2024-04-25T11:23:56.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.424+0000 7f90a92f9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90a4101f20 0x7f90a419a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:56.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.424+0000 7f90a92f9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90a41070d0 0x7f90a419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:56.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.424+0000 7f90a92f9640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90a4109a30 0x7f90a41a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:56.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.424+0000 7f90a92f9640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f90a410f550 con 0x7f90a41070d0 2024-04-25T11:23:56.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.424+0000 7f90a92f9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f90a410f3d0 con 0x7f90a4101f20 2024-04-25T11:23:56.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.425+0000 7f90a92f9640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f90a410f6d0 con 0x7f90a4109a30 2024-04-25T11:23:56.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.425+0000 7f90a27fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90a41070d0 0x7f90a419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:56.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.425+0000 7f90a27fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90a41070d0 0x7f90a419a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58450/0 (socket says 172.21.15.69:58450) 2024-04-25T11:23:56.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.425+0000 7f90a27fc640 1 -- 172.21.15.69:0/2456161656 learned_addr learned my addr 172.21.15.69:0/2456161656 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:56.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.425+0000 7f90a2ffd640 1 --2- 172.21.15.69:0/2456161656 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90a4101f20 0x7f90a419a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:56.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.425+0000 7f90a37fe640 1 --2- 172.21.15.69:0/2456161656 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90a4109a30 0x7f90a41a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:56.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.425+0000 7f90a27fc640 1 -- 172.21.15.69:0/2456161656 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90a4109a30 msgr2=0x7f90a41a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:56.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.425+0000 7f90a27fc640 1 --2- 172.21.15.69:0/2456161656 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90a4109a30 0x7f90a41a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:56.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.425+0000 7f90a27fc640 1 -- 172.21.15.69:0/2456161656 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90a4101f20 msgr2=0x7f90a419a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:56.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.425+0000 7f90a27fc640 1 --2- 172.21.15.69:0/2456161656 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90a4101f20 0x7f90a419a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:56.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.425+0000 7f90a27fc640 1 -- 172.21.15.69:0/2456161656 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90a41a1d30 con 0x7f90a41070d0 2024-04-25T11:23:56.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.425+0000 7f90a37fe640 1 --2- 172.21.15.69:0/2456161656 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90a4109a30 0x7f90a41a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:23:56.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.425+0000 7f90a27fc640 1 --2- 172.21.15.69:0/2456161656 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90a41070d0 0x7f90a419a730 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f909800d770 tx=0x7f909800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:56.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.425+0000 7f908bfff640 1 -- 172.21.15.69:0/2456161656 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9098017070 con 0x7f90a41070d0 2024-04-25T11:23:56.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.425+0000 7f908bfff640 1 -- 172.21.15.69:0/2456161656 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90980054b0 con 0x7f90a41070d0 2024-04-25T11:23:56.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.425+0000 7f90a92f9640 1 -- 172.21.15.69:0/2456161656 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90a41a1fc0 con 0x7f90a41070d0 2024-04-25T11:23:56.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.425+0000 7f908bfff640 1 -- 172.21.15.69:0/2456161656 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9098004c60 con 0x7f90a41070d0 2024-04-25T11:23:56.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.425+0000 7f90a92f9640 1 -- 172.21.15.69:0/2456161656 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90a41a2460 con 0x7f90a41070d0 2024-04-25T11:23:56.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.426+0000 7f90a92f9640 1 -- 172.21.15.69:0/2456161656 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90a40feea0 con 0x7f90a41070d0 2024-04-25T11:23:56.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.428+0000 7f908bfff640 1 -- 172.21.15.69:0/2456161656 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f9098004e00 con 0x7f90a41070d0 2024-04-25T11:23:56.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.428+0000 7f908bfff640 1 --2- 172.21.15.69:0/2456161656 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f908c0afaf0 0x7f908c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:56.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.428+0000 7f90a2ffd640 1 --2- 172.21.15.69:0/2456161656 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f908c0afaf0 0x7f908c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:56.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.428+0000 7f908bfff640 1 -- 172.21.15.69:0/2456161656 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7f90980dec80 con 0x7f90a41070d0 2024-04-25T11:23:56.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.429+0000 7f90a2ffd640 1 --2- 172.21.15.69:0/2456161656 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f908c0afaf0 0x7f908c0b2000 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9094005ab0 tx=0x7f9094000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:56.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.432+0000 7f908bfff640 1 -- 172.21.15.69:0/2456161656 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f90980a7620 con 0x7f90a41070d0 2024-04-25T11:23:56.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.611+0000 7f90a92f9640 1 -- 172.21.15.69:0/2456161656 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f90a40637a0 con 0x7f90a41070d0 2024-04-25T11:23:56.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.611+0000 7f908bfff640 1 -- 172.21.15.69:0/2456161656 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) ==== 100+0+11 (secure 0 0 0) 0x7f90980a6f20 con 0x7f90a41070d0 2024-04-25T11:23:56.605 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-04-25T11:23:56.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.613+0000 7f90a92f9640 1 -- 172.21.15.69:0/2456161656 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f908c0afaf0 msgr2=0x7f908c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:56.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.613+0000 7f90a92f9640 1 --2- 172.21.15.69:0/2456161656 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f908c0afaf0 0x7f908c0b2000 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9094005ab0 tx=0x7f9094000bc0 comp rx=0 tx=0).stop 2024-04-25T11:23:56.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.613+0000 7f90a92f9640 1 -- 172.21.15.69:0/2456161656 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90a41070d0 msgr2=0x7f90a419a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:56.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.613+0000 7f90a92f9640 1 --2- 172.21.15.69:0/2456161656 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90a41070d0 0x7f90a419a730 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f909800d770 tx=0x7f909800dc30 comp rx=0 tx=0).stop 2024-04-25T11:23:56.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.613+0000 7f90a92f9640 1 -- 172.21.15.69:0/2456161656 shutdown_connections 2024-04-25T11:23:56.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.613+0000 7f90a92f9640 1 --2- 172.21.15.69:0/2456161656 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f908c0afaf0 0x7f908c0b2000 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:56.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.613+0000 7f90a92f9640 1 --2- 172.21.15.69:0/2456161656 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90a4109a30 0x7f90a41a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:56.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.613+0000 7f90a92f9640 1 --2- 172.21.15.69:0/2456161656 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90a41070d0 0x7f90a419a730 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:56.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.613+0000 7f90a92f9640 1 --2- 172.21.15.69:0/2456161656 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90a4101f20 0x7f90a419a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:56.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.613+0000 7f90a92f9640 1 -- 172.21.15.69:0/2456161656 >> 172.21.15.69:0/2456161656 conn(0x7f90a40fd380 msgr2=0x7f90a41054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:56.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.614+0000 7f90a92f9640 1 -- 172.21.15.69:0/2456161656 shutdown_connections 2024-04-25T11:23:56.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.614+0000 7f90a92f9640 1 -- 172.21.15.69:0/2456161656 wait complete. 2024-04-25T11:23:56.618 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T11:23:56.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.737+0000 7fd8ad8ca640 1 -- 172.21.15.69:0/1238960659 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8a8108930 msgr2=0x7fd8a810ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:56.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.737+0000 7fd8ad8ca640 1 --2- 172.21.15.69:0/1238960659 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8a8108930 0x7fd8a810ad50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd898009900 tx=0x7fd89802a410 comp rx=0 tx=0).stop 2024-04-25T11:23:56.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.737+0000 7fd8ad8ca640 1 -- 172.21.15.69:0/1238960659 shutdown_connections 2024-04-25T11:23:56.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.737+0000 7fd8ad8ca640 1 --2- 172.21.15.69:0/1238960659 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8a8108930 0x7fd8a810ad50 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:56.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.737+0000 7fd8ad8ca640 1 --2- 172.21.15.69:0/1238960659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8a8103c30 0x7fd8a8104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:56.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.737+0000 7fd8ad8ca640 1 --2- 172.21.15.69:0/1238960659 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8a81029f0 0x7fd8a81036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:56.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.737+0000 7fd8ad8ca640 1 -- 172.21.15.69:0/1238960659 >> 172.21.15.69:0/1238960659 conn(0x7fd8a80fdbc0 msgr2=0x7fd8a8100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:56.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.737+0000 7fd8ad8ca640 1 -- 172.21.15.69:0/1238960659 shutdown_connections 2024-04-25T11:23:56.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.737+0000 7fd8ad8ca640 1 -- 172.21.15.69:0/1238960659 wait complete. 2024-04-25T11:23:56.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.737+0000 7fd8ad8ca640 1 Processor -- start 2024-04-25T11:23:56.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.738+0000 7fd8ad8ca640 1 -- start start 2024-04-25T11:23:56.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.738+0000 7fd8ad8ca640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8a81029f0 0x7fd8a819a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:56.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.738+0000 7fd8ad8ca640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8a8103c30 0x7fd8a819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:56.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.738+0000 7fd8ad8ca640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8a8108930 0x7fd8a81a1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:56.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.738+0000 7fd8ad8ca640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a810e3d0 con 0x7fd8a81029f0 2024-04-25T11:23:56.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.738+0000 7fd8ad8ca640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a810e250 con 0x7fd8a8108930 2024-04-25T11:23:56.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.738+0000 7fd8ad8ca640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a810e550 con 0x7fd8a8103c30 2024-04-25T11:23:56.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.738+0000 7fd8a6ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8a81029f0 0x7fd8a819a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:56.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.738+0000 7fd8a6ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8a81029f0 0x7fd8a819a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58460/0 (socket says 172.21.15.69:58460) 2024-04-25T11:23:56.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.738+0000 7fd8a6ffd640 1 -- 172.21.15.69:0/3524940741 learned_addr learned my addr 172.21.15.69:0/3524940741 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:56.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.738+0000 7fd8a77fe640 1 --2- 172.21.15.69:0/3524940741 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8a8108930 0x7fd8a81a1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:56.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.738+0000 7fd8a67fc640 1 --2- 172.21.15.69:0/3524940741 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8a8103c30 0x7fd8a819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:56.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.738+0000 7fd8a6ffd640 1 -- 172.21.15.69:0/3524940741 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8a8103c30 msgr2=0x7fd8a819a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:56.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.738+0000 7fd8a6ffd640 1 --2- 172.21.15.69:0/3524940741 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8a8103c30 0x7fd8a819a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:56.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.738+0000 7fd8a6ffd640 1 -- 172.21.15.69:0/3524940741 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8a8108930 msgr2=0x7fd8a81a1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:56.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.738+0000 7fd8a6ffd640 1 --2- 172.21.15.69:0/3524940741 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8a8108930 0x7fd8a81a1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:56.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.738+0000 7fd8a6ffd640 1 -- 172.21.15.69:0/3524940741 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8a81a1c70 con 0x7fd8a81029f0 2024-04-25T11:23:56.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.738+0000 7fd8a6ffd640 1 --2- 172.21.15.69:0/3524940741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8a81029f0 0x7fd8a819a130 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0005d00 tx=0x7fd8a000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:56.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.739+0000 7fd8ac8c8640 1 -- 172.21.15.69:0/3524940741 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0017070 con 0x7fd8a81029f0 2024-04-25T11:23:56.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.739+0000 7fd8ac8c8640 1 -- 172.21.15.69:0/3524940741 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd8a0009c30 con 0x7fd8a81029f0 2024-04-25T11:23:56.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.739+0000 7fd8ad8ca640 1 -- 172.21.15.69:0/3524940741 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8a81a1f60 con 0x7fd8a81029f0 2024-04-25T11:23:56.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.739+0000 7fd8ac8c8640 1 -- 172.21.15.69:0/3524940741 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a000f620 con 0x7fd8a81029f0 2024-04-25T11:23:56.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.739+0000 7fd8ad8ca640 1 -- 172.21.15.69:0/3524940741 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8a81a2370 con 0x7fd8a81029f0 2024-04-25T11:23:56.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.740+0000 7fd8ad8ca640 1 -- 172.21.15.69:0/3524940741 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8a81a25d0 con 0x7fd8a81029f0 2024-04-25T11:23:56.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.742+0000 7fd8ac8c8640 1 -- 172.21.15.69:0/3524940741 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fd8a001a070 con 0x7fd8a81029f0 2024-04-25T11:23:56.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.742+0000 7fd8ac8c8640 1 --2- 172.21.15.69:0/3524940741 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fd8900afaf0 0x7fd8900b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:56.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.743+0000 7fd8a67fc640 1 --2- 172.21.15.69:0/3524940741 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fd8900afaf0 0x7fd8900b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:56.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.743+0000 7fd8ac8c8640 1 -- 172.21.15.69:0/3524940741 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7fd8a00de850 con 0x7fd8a81029f0 2024-04-25T11:23:56.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.743+0000 7fd8a67fc640 1 --2- 172.21.15.69:0/3524940741 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fd8900afaf0 0x7fd8900b2000 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd8a819b1d0 tx=0x7fd89c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:56.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.746+0000 7fd8ac8c8640 1 -- 172.21.15.69:0/3524940741 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd8a0009de0 con 0x7fd8a81029f0 2024-04-25T11:23:56.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.926+0000 7fd8ad8ca640 1 -- 172.21.15.69:0/3524940741 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fd8a81a27a0 con 0x7fd8a81029f0 2024-04-25T11:23:56.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.926+0000 7fd8ac8c8640 1 -- 172.21.15.69:0/3524940741 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v80) ==== 103+0+11 (secure 0 0 0) 0x7fd8a00a71f0 con 0x7fd8a81029f0 2024-04-25T11:23:56.920 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-25T11:23:56.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.928+0000 7fd8ad8ca640 1 -- 172.21.15.69:0/3524940741 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fd8900afaf0 msgr2=0x7fd8900b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:56.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.928+0000 7fd8ad8ca640 1 --2- 172.21.15.69:0/3524940741 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fd8900afaf0 0x7fd8900b2000 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd8a819b1d0 tx=0x7fd89c009500 comp rx=0 tx=0).stop 2024-04-25T11:23:56.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.928+0000 7fd8ad8ca640 1 -- 172.21.15.69:0/3524940741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8a81029f0 msgr2=0x7fd8a819a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:56.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.928+0000 7fd8ad8ca640 1 --2- 172.21.15.69:0/3524940741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8a81029f0 0x7fd8a819a130 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0005d00 tx=0x7fd8a000dde0 comp rx=0 tx=0).stop 2024-04-25T11:23:56.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.928+0000 7fd8ad8ca640 1 -- 172.21.15.69:0/3524940741 shutdown_connections 2024-04-25T11:23:56.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.928+0000 7fd8ad8ca640 1 --2- 172.21.15.69:0/3524940741 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fd8900afaf0 0x7fd8900b2000 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:56.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.928+0000 7fd8ad8ca640 1 --2- 172.21.15.69:0/3524940741 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8a8108930 0x7fd8a81a1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:56.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.928+0000 7fd8ad8ca640 1 --2- 172.21.15.69:0/3524940741 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8a8103c30 0x7fd8a819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:56.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.928+0000 7fd8ad8ca640 1 --2- 172.21.15.69:0/3524940741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8a81029f0 0x7fd8a819a130 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:56.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.928+0000 7fd8ad8ca640 1 -- 172.21.15.69:0/3524940741 >> 172.21.15.69:0/3524940741 conn(0x7fd8a80fdbc0 msgr2=0x7fd8a8074dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:56.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.928+0000 7fd8ad8ca640 1 -- 172.21.15.69:0/3524940741 shutdown_connections 2024-04-25T11:23:56.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:56.928+0000 7fd8ad8ca640 1 -- 172.21.15.69:0/3524940741 wait complete. 2024-04-25T11:23:56.932 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:56 smithi069 ceph-mon[29188]: pgmap v292: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 336 B/s wr, 0 op/s 2024-04-25T11:23:56.932 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:56 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/1682417289' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:23:56.932 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:56 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/129481838' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:23:56.932 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:56 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2456161656' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:23:56.934 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:23:57.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.048+0000 7fe3efa27640 1 -- 172.21.15.69:0/2754373722 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3e8101760 msgr2=0x7fe3e8101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:57.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.048+0000 7fe3efa27640 1 --2- 172.21.15.69:0/2754373722 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3e8101760 0x7fe3e8101bd0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8009900 tx=0x7fe3d802a410 comp rx=0 tx=0).stop 2024-04-25T11:23:57.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.048+0000 7fe3efa27640 1 -- 172.21.15.69:0/2754373722 shutdown_connections 2024-04-25T11:23:57.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.048+0000 7fe3efa27640 1 --2- 172.21.15.69:0/2754373722 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3e8102260 0x7fe3e810bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:57.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.048+0000 7fe3efa27640 1 --2- 172.21.15.69:0/2754373722 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3e8101760 0x7fe3e8101bd0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:57.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.048+0000 7fe3efa27640 1 --2- 172.21.15.69:0/2754373722 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe3e81050b0 0x7fe3e81054c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:57.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.048+0000 7fe3efa27640 1 -- 172.21.15.69:0/2754373722 >> 172.21.15.69:0/2754373722 conn(0x7fe3e80fd380 msgr2=0x7fe3e80ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:57.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.048+0000 7fe3efa27640 1 -- 172.21.15.69:0/2754373722 shutdown_connections 2024-04-25T11:23:57.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.048+0000 7fe3efa27640 1 -- 172.21.15.69:0/2754373722 wait complete. 2024-04-25T11:23:57.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.049+0000 7fe3efa27640 1 Processor -- start 2024-04-25T11:23:57.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.049+0000 7fe3efa27640 1 -- start start 2024-04-25T11:23:57.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.049+0000 7fe3efa27640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3e8101760 0x7fe3e819a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:57.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.049+0000 7fe3efa27640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe3e8102260 0x7fe3e819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:57.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.049+0000 7fe3efa27640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3e81050b0 0x7fe3e81a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:57.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.049+0000 7fe3efa27640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e810f4d0 con 0x7fe3e8101760 2024-04-25T11:23:57.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.049+0000 7fe3efa27640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e810f350 con 0x7fe3e8102260 2024-04-25T11:23:57.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.049+0000 7fe3efa27640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e810f650 con 0x7fe3e81050b0 2024-04-25T11:23:57.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.049+0000 7fe3ed7c3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3e8101760 0x7fe3e819a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:57.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.049+0000 7fe3ed7c3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3e8101760 0x7fe3e819a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58482/0 (socket says 172.21.15.69:58482) 2024-04-25T11:23:57.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.049+0000 7fe3ed7c3640 1 -- 172.21.15.69:0/4168788721 learned_addr learned my addr 172.21.15.69:0/4168788721 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:23:57.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.049+0000 7fe3ecfc2640 1 --2- 172.21.15.69:0/4168788721 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe3e8102260 0x7fe3e819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:57.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.049+0000 7fe3edfc4640 1 --2- 172.21.15.69:0/4168788721 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3e81050b0 0x7fe3e81a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:57.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.049+0000 7fe3ed7c3640 1 -- 172.21.15.69:0/4168788721 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3e81050b0 msgr2=0x7fe3e81a1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:57.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.049+0000 7fe3ed7c3640 1 --2- 172.21.15.69:0/4168788721 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3e81050b0 0x7fe3e81a1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:57.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.049+0000 7fe3ed7c3640 1 -- 172.21.15.69:0/4168788721 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe3e8102260 msgr2=0x7fe3e819a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:57.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.049+0000 7fe3ed7c3640 1 --2- 172.21.15.69:0/4168788721 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe3e8102260 0x7fe3e819a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:57.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.049+0000 7fe3ed7c3640 1 -- 172.21.15.69:0/4168788721 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3e81a1bf0 con 0x7fe3e8101760 2024-04-25T11:23:57.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.049+0000 7fe3ecfc2640 1 --2- 172.21.15.69:0/4168788721 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe3e8102260 0x7fe3e819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:23:57.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.050+0000 7fe3ed7c3640 1 --2- 172.21.15.69:0/4168788721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3e8101760 0x7fe3e819a160 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe3e400d740 tx=0x7fe3e400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:57.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.050+0000 7fe3de7fc640 1 -- 172.21.15.69:0/4168788721 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e4017070 con 0x7fe3e8101760 2024-04-25T11:23:57.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.050+0000 7fe3de7fc640 1 -- 172.21.15.69:0/4168788721 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe3e400de50 con 0x7fe3e8101760 2024-04-25T11:23:57.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.050+0000 7fe3efa27640 1 -- 172.21.15.69:0/4168788721 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3e81a1ee0 con 0x7fe3e8101760 2024-04-25T11:23:57.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.050+0000 7fe3de7fc640 1 -- 172.21.15.69:0/4168788721 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e4003bb0 con 0x7fe3e8101760 2024-04-25T11:23:57.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.050+0000 7fe3efa27640 1 -- 172.21.15.69:0/4168788721 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3e81a2340 con 0x7fe3e8101760 2024-04-25T11:23:57.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.051+0000 7fe3efa27640 1 -- 172.21.15.69:0/4168788721 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3e806a160 con 0x7fe3e8101760 2024-04-25T11:23:57.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.052+0000 7fe3de7fc640 1 -- 172.21.15.69:0/4168788721 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fe3e4003d50 con 0x7fe3e8101760 2024-04-25T11:23:57.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.053+0000 7fe3de7fc640 1 --2- 172.21.15.69:0/4168788721 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fe3d40afaf0 0x7fe3d40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:57.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.053+0000 7fe3de7fc640 1 -- 172.21.15.69:0/4168788721 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7fe3e40dece0 con 0x7fe3e8101760 2024-04-25T11:23:57.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.053+0000 7fe3ecfc2640 1 --2- 172.21.15.69:0/4168788721 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fe3d40afaf0 0x7fe3d40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:57.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.053+0000 7fe3ecfc2640 1 --2- 172.21.15.69:0/4168788721 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fe3d40afaf0 0x7fe3d40b2000 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8006010 tx=0x7fe3d8031420 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:57.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.056+0000 7fe3de7fc640 1 -- 172.21.15.69:0/4168788721 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe3e40a7680 con 0x7fe3e8101760 2024-04-25T11:23:57.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:56 smithi146 ceph-mon[32821]: pgmap v292: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 336 B/s wr, 0 op/s 2024-04-25T11:23:57.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:56 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/1682417289' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:23:57.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:56 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/129481838' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:23:57.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:56 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/2456161656' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:23:57.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:56 smithi152 ceph-mon[32701]: pgmap v292: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 336 B/s wr, 0 op/s 2024-04-25T11:23:57.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:56 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/1682417289' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:23:57.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:56 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/129481838' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:23:57.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:56 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/2456161656' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:23:57.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.246+0000 7fe3efa27640 1 -- 172.21.15.69:0/4168788721 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe3e8108a90 con 0x7fe3e8101760 2024-04-25T11:23:57.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.246+0000 7fe3de7fc640 1 -- 172.21.15.69:0/4168788721 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v80) ==== 81+0+592 (secure 0 0 0) 0x7fe3e8108a90 con 0x7fe3e8101760 2024-04-25T11:23:57.240 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/1372727546 2024-04-26T11:16:31.688159+0000 2024-04-25T11:23:57.241 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6801/1097124842 2024-04-26T11:16:31.688159+0000 2024-04-25T11:23:57.241 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6800/1508198215 2024-04-26T11:16:04.860792+0000 2024-04-25T11:23:57.241 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/3103465990 2024-04-26T11:16:04.860792+0000 2024-04-25T11:23:57.241 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/3540132960 2024-04-26T11:16:04.860792+0000 2024-04-25T11:23:57.241 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/3899912190 2024-04-26T11:16:31.688159+0000 2024-04-25T11:23:57.241 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6800/1097124842 2024-04-26T11:16:31.688159+0000 2024-04-25T11:23:57.241 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/1199907202 2024-04-26T11:16:04.860792+0000 2024-04-25T11:23:57.241 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6801/1508198215 2024-04-26T11:16:04.860792+0000 2024-04-25T11:23:57.241 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/2879102529 2024-04-26T11:16:31.688159+0000 2024-04-25T11:23:57.241 INFO:teuthology.orchestra.run.smithi069.stderr:listed 10 entries 2024-04-25T11:23:57.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.248+0000 7fe3efa27640 1 -- 172.21.15.69:0/4168788721 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fe3d40afaf0 msgr2=0x7fe3d40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:57.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.248+0000 7fe3efa27640 1 --2- 172.21.15.69:0/4168788721 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fe3d40afaf0 0x7fe3d40b2000 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8006010 tx=0x7fe3d8031420 comp rx=0 tx=0).stop 2024-04-25T11:23:57.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.248+0000 7fe3efa27640 1 -- 172.21.15.69:0/4168788721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3e8101760 msgr2=0x7fe3e819a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:57.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.248+0000 7fe3efa27640 1 --2- 172.21.15.69:0/4168788721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3e8101760 0x7fe3e819a160 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe3e400d740 tx=0x7fe3e400dc00 comp rx=0 tx=0).stop 2024-04-25T11:23:57.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.248+0000 7fe3efa27640 1 -- 172.21.15.69:0/4168788721 shutdown_connections 2024-04-25T11:23:57.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.248+0000 7fe3efa27640 1 --2- 172.21.15.69:0/4168788721 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fe3d40afaf0 0x7fe3d40b2000 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:57.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.248+0000 7fe3efa27640 1 --2- 172.21.15.69:0/4168788721 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3e81050b0 0x7fe3e81a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:57.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.248+0000 7fe3efa27640 1 --2- 172.21.15.69:0/4168788721 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe3e8102260 0x7fe3e819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:57.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.248+0000 7fe3efa27640 1 --2- 172.21.15.69:0/4168788721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3e8101760 0x7fe3e819a160 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:57.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.248+0000 7fe3efa27640 1 -- 172.21.15.69:0/4168788721 >> 172.21.15.69:0/4168788721 conn(0x7fe3e80fd380 msgr2=0x7fe3e8105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:57.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.248+0000 7fe3efa27640 1 -- 172.21.15.69:0/4168788721 shutdown_connections 2024-04-25T11:23:57.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:23:57.248+0000 7fe3efa27640 1 -- 172.21.15.69:0/4168788721 wait complete. 2024-04-25T11:23:57.253 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi069.front.sepia.ceph.com... 2024-04-25T11:23:57.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:23:57.254 DEBUG:teuthology.orchestra.run.smithi069:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-25T11:23:57.275 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi069.stderr:ceph-fuse[73034]: fuse finished with error 0 and tester_r 0 2024-04-25T11:23:57.277 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:23:58.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:57 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/3524940741' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T11:23:58.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:57 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/4168788721' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T11:23:58.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:57 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/3524940741' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T11:23:58.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:57 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/4168788721' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T11:23:58.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:57 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/3524940741' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T11:23:58.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:57 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/4168788721' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T11:23:59.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:23:58 smithi146 ceph-mon[32821]: pgmap v293: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 303 B/s wr, 0 op/s 2024-04-25T11:23:59.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:23:58 smithi069 ceph-mon[29188]: pgmap v293: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 303 B/s wr, 0 op/s 2024-04-25T11:23:59.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:23:58 smithi152 ceph-mon[32701]: pgmap v293: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 303 B/s wr, 0 op/s 2024-04-25T11:24:01.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:00 smithi146 ceph-mon[32821]: pgmap v294: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-25T11:24:01.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:00 smithi069 ceph-mon[29188]: pgmap v294: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-25T11:24:01.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:00 smithi152 ceph-mon[32701]: pgmap v294: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-25T11:24:03.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:02 smithi069 ceph-mon[29188]: pgmap v295: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-25T11:24:03.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:02 smithi152 ceph-mon[32701]: pgmap v295: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-25T11:24:03.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:02 smithi146 ceph-mon[32821]: pgmap v295: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-25T11:24:03.380 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi069.front.sepia.ceph.com 2024-04-25T11:24:03.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:24:03.380 DEBUG:teuthology.orchestra.run.smithi069:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-25T11:24:03.498 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:24:03.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.622+0000 7fc92cdce640 1 -- 172.21.15.69:0/3053544122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc928103c10 msgr2=0x7fc928104020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:03.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.622+0000 7fc92cdce640 1 --2- 172.21.15.69:0/3053544122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc928103c10 0x7fc928104020 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc91c009900 tx=0x7fc91c02a410 comp rx=0 tx=0).stop 2024-04-25T11:24:03.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.622+0000 7fc92cdce640 1 -- 172.21.15.69:0/3053544122 shutdown_connections 2024-04-25T11:24:03.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.622+0000 7fc92cdce640 1 --2- 172.21.15.69:0/3053544122 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc928108910 0x7fc92810ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:03.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.622+0000 7fc92cdce640 1 --2- 172.21.15.69:0/3053544122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc928103c10 0x7fc928104020 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:03.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.622+0000 7fc92cdce640 1 --2- 172.21.15.69:0/3053544122 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9281029d0 0x7fc9281036d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:03.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.622+0000 7fc92cdce640 1 -- 172.21.15.69:0/3053544122 >> 172.21.15.69:0/3053544122 conn(0x7fc9280fdba0 msgr2=0x7fc928100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:03.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.623+0000 7fc92cdce640 1 -- 172.21.15.69:0/3053544122 shutdown_connections 2024-04-25T11:24:03.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.623+0000 7fc92cdce640 1 -- 172.21.15.69:0/3053544122 wait complete. 2024-04-25T11:24:03.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.623+0000 7fc92cdce640 1 Processor -- start 2024-04-25T11:24:03.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.623+0000 7fc92cdce640 1 -- start start 2024-04-25T11:24:03.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.624+0000 7fc92cdce640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9281029d0 0x7fc92819a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:03.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.624+0000 7fc92cdce640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc928103c10 0x7fc92819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:03.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.624+0000 7fc92cdce640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc928108910 0x7fc9281a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:03.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.624+0000 7fc92cdce640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc92810e400 con 0x7fc9281029d0 2024-04-25T11:24:03.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.624+0000 7fc92cdce640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc92810e280 con 0x7fc928103c10 2024-04-25T11:24:03.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.624+0000 7fc92cdce640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc92810e580 con 0x7fc928108910 2024-04-25T11:24:03.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.624+0000 7fc92659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9281029d0 0x7fc92819a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:03.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.624+0000 7fc92659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9281029d0 0x7fc92819a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45294/0 (socket says 172.21.15.69:45294) 2024-04-25T11:24:03.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.624+0000 7fc92659c640 1 -- 172.21.15.69:0/135374937 learned_addr learned my addr 172.21.15.69:0/135374937 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:24:03.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.624+0000 7fc925d9b640 1 --2- 172.21.15.69:0/135374937 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc928103c10 0x7fc92819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:03.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.624+0000 7fc92659c640 1 -- 172.21.15.69:0/135374937 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc928108910 msgr2=0x7fc9281a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:03.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.624+0000 7fc92659c640 1 --2- 172.21.15.69:0/135374937 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc928108910 0x7fc9281a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:03.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.624+0000 7fc92659c640 1 -- 172.21.15.69:0/135374937 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc928103c10 msgr2=0x7fc92819a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:03.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.624+0000 7fc92659c640 1 --2- 172.21.15.69:0/135374937 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc928103c10 0x7fc92819a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:03.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.624+0000 7fc92659c640 1 -- 172.21.15.69:0/135374937 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9281a1ce0 con 0x7fc9281029d0 2024-04-25T11:24:03.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.624+0000 7fc925d9b640 1 --2- 172.21.15.69:0/135374937 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc928103c10 0x7fc92819a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:24:03.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.624+0000 7fc92659c640 1 --2- 172.21.15.69:0/135374937 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9281029d0 0x7fc92819a210 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc918006fd0 tx=0x7fc91800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:03.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.624+0000 7fc9177fe640 1 -- 172.21.15.69:0/135374937 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc918011830 con 0x7fc9281029d0 2024-04-25T11:24:03.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.624+0000 7fc9177fe640 1 -- 172.21.15.69:0/135374937 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc918016070 con 0x7fc9281029d0 2024-04-25T11:24:03.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.624+0000 7fc9177fe640 1 -- 172.21.15.69:0/135374937 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc918010d30 con 0x7fc9281029d0 2024-04-25T11:24:03.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.624+0000 7fc92cdce640 1 -- 172.21.15.69:0/135374937 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9281a1fd0 con 0x7fc9281029d0 2024-04-25T11:24:03.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.624+0000 7fc92cdce640 1 -- 172.21.15.69:0/135374937 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9281a2440 con 0x7fc9281029d0 2024-04-25T11:24:03.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.625+0000 7fc92cdce640 1 -- 172.21.15.69:0/135374937 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9281a2700 con 0x7fc9281029d0 2024-04-25T11:24:03.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.628+0000 7fc9177fe640 1 -- 172.21.15.69:0/135374937 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fc9180119d0 con 0x7fc9281029d0 2024-04-25T11:24:03.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.628+0000 7fc9177fe640 1 --2- 172.21.15.69:0/135374937 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc9100afb70 0x7fc9100b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:03.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.628+0000 7fc925d9b640 1 --2- 172.21.15.69:0/135374937 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc9100afb70 0x7fc9100b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:03.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.628+0000 7fc9177fe640 1 -- 172.21.15.69:0/135374937 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7fc9180deb90 con 0x7fc9281029d0 2024-04-25T11:24:03.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.629+0000 7fc925d9b640 1 --2- 172.21.15.69:0/135374937 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc9100afb70 0x7fc9100b2080 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc91c005f70 tx=0x7fc91c005720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:03.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.632+0000 7fc9177fe640 1 -- 172.21.15.69:0/135374937 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc9180a7530 con 0x7fc9281029d0 2024-04-25T11:24:03.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.813+0000 7fc92cdce640 1 -- 172.21.15.69:0/135374937 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc9281a28d0 con 0x7fc9281029d0 2024-04-25T11:24:03.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.815+0000 7fc9177fe640 1 -- 172.21.15.69:0/135374937 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+24130 (secure 0 0 0) 0x7fc9180a6e30 con 0x7fc9281029d0 2024-04-25T11:24:03.809 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:24:03.810 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":80,"fsid":"f7ed73dc-02f4-11ef-bc93-c7b262605968","created":"2024-04-25T11:15:37.965281+0000","modified":"2024-04-25T11:23:47.662955+0000","last_up_change":"2024-04-25T11:22:50.691869+0000","last_in_change":"2024-04-25T11:22:30.271959+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:18:57.781263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:23:38.685110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:23:39.943803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T11:23:43.650521+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6a0d179-4998-4754-bf82-2ea3dac54e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6803","nonce":3756025235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6805","nonce":3756025235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6809","nonce":3756025235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6807","nonce":3756025235}]},"public_addr":"172.21.15.69:6803/3756025235","cluster_addr":"172.21.15.69:6805/3756025235","heartbeat_back_addr":"172.21.15.69:6809/3756025235","heartbeat_front_addr":"172.21.15.69:6807/3756025235","state":["exists","up"]},{"osd":1,"uuid":"c0c66930-4233-493f-b2bb-99c473994b59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6801","nonce":3166305213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6803","nonce":3166305213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6807","nonce":3166305213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6805","nonce":3166305213}]},"public_addr":"172.21.15.146:6801/3166305213","cluster_addr":"172.21.15.146:6803/3166305213","heartbeat_back_addr":"172.21.15.146:6807/3166305213","heartbeat_front_addr":"172.21.15.146:6805/3166305213","state":["exists","up"]},{"osd":2,"uuid":"d4ef735b-9786-4cb9-83ef-7c3f0691a729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6801","nonce":1474319175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6803","nonce":1474319175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6807","nonce":1474319175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6805","nonce":1474319175}]},"public_addr":"172.21.15.152:6801/1474319175","cluster_addr":"172.21.15.152:6803/1474319175","heartbeat_back_addr":"172.21.15.152:6807/1474319175","heartbeat_front_addr":"172.21.15.152:6805/1474319175","state":["exists","up"]},{"osd":3,"uuid":"0ab85fb8-5bbb-4ffc-bddf-cae7644a64ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6811","nonce":1809071296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6813","nonce":1809071296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6817","nonce":1809071296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6815","nonce":1809071296}]},"public_addr":"172.21.15.69:6811/1809071296","cluster_addr":"172.21.15.69:6813/1809071296","heartbeat_back_addr":"172.21.15.69:6817/1809071296","heartbeat_front_addr":"172.21.15.69:6815/1809071296","state":["exists","up"]},{"osd":4,"uuid":"273e0592-bbbc-4188-8eeb-46a3879cf632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6809","nonce":1898520972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6811","nonce":1898520972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6815","nonce":1898520972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6813","nonce":1898520972}]},"public_addr":"172.21.15.146:6809/1898520972","cluster_addr":"172.21.15.146:6811/1898520972","heartbeat_back_addr":"172.21.15.146:6815/1898520972","heartbeat_front_addr":"172.21.15.146:6813/1898520972","state":["exists","up"]},{"osd":5,"uuid":"bbeae945-9fa2-48f9-a1d6-8bf18dcaef19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6809","nonce":1996555108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6811","nonce":1996555108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6815","nonce":1996555108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6813","nonce":1996555108}]},"public_addr":"172.21.15.152:6809/1996555108","cluster_addr":"172.21.15.152:6811/1996555108","heartbeat_back_addr":"172.21.15.152:6815/1996555108","heartbeat_front_addr":"172.21.15.152:6813/1996555108","state":["exists","up"]},{"osd":6,"uuid":"dbb61253-30bc-418e-bd75-118c80bbf931","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6819","nonce":491215762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6821","nonce":491215762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6825","nonce":491215762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6823","nonce":491215762}]},"public_addr":"172.21.15.69:6819/491215762","cluster_addr":"172.21.15.69:6821/491215762","heartbeat_back_addr":"172.21.15.69:6825/491215762","heartbeat_front_addr":"172.21.15.69:6823/491215762","state":["exists","up"]},{"osd":7,"uuid":"75f23854-e283-4e3e-94cf-7b4079d83876","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6817","nonce":2330294690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6819","nonce":2330294690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6823","nonce":2330294690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6821","nonce":2330294690}]},"public_addr":"172.21.15.146:6817/2330294690","cluster_addr":"172.21.15.146:6819/2330294690","heartbeat_back_addr":"172.21.15.146:6823/2330294690","heartbeat_front_addr":"172.21.15.146:6821/2330294690","state":["exists","up"]},{"osd":8,"uuid":"1aa1060c-0c14-4abd-be56-5ebfeb78759c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6817","nonce":950669381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6819","nonce":950669381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6823","nonce":950669381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6821","nonce":950669381}]},"public_addr":"172.21.15.152:6817/950669381","cluster_addr":"172.21.15.152:6819/950669381","heartbeat_back_addr":"172.21.15.152:6823/950669381","heartbeat_front_addr":"172.21.15.152:6821/950669381","state":["exists","up"]},{"osd":9,"uuid":"3f4bf800-4090-4138-9bb3-d001f13f450c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6827","nonce":2414186758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6829","nonce":2414186758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6833","nonce":2414186758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6831","nonce":2414186758}]},"public_addr":"172.21.15.69:6827/2414186758","cluster_addr":"172.21.15.69:6829/2414186758","heartbeat_back_addr":"172.21.15.69:6833/2414186758","heartbeat_front_addr":"172.21.15.69:6831/2414186758","state":["exists","up"]},{"osd":10,"uuid":"19e967e5-51ce-4506-80c9-0a95441baf99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6825","nonce":707256400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6827","nonce":707256400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6831","nonce":707256400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6829","nonce":707256400}]},"public_addr":"172.21.15.146:6825/707256400","cluster_addr":"172.21.15.146:6827/707256400","heartbeat_back_addr":"172.21.15.146:6831/707256400","heartbeat_front_addr":"172.21.15.146:6829/707256400","state":["exists","up"]},{"osd":11,"uuid":"7788c2de-6930-4808-be0f-f1fd1ae0cb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6825","nonce":2934336833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6827","nonce":2934336833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6831","nonce":2934336833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6829","nonce":2934336833}]},"public_addr":"172.21.15.152:6825/2934336833","cluster_addr":"172.21.15.152:6827/2934336833","heartbeat_back_addr":"172.21.15.152:6831/2934336833","heartbeat_front_addr":"172.21.15.152:6829/2934336833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:07.497448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:31.227494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:55.027978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:20.713437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:46.214704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:20:11.199983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:20:37.177070+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:02.951240+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:28.963680+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:56.095807+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:22:22.221744+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:22:48.929334+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/1372727546":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6801/1097124842":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6800/1508198215":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3103465990":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3540132960":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3899912190":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6800/1097124842":"2024-04-26T11:16:31.688159+0000","172.21.15.69:0/1199907202":"2024-04-26T11:16:04.860792+0000","172.21.15.69:6801/1508198215":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/2879102529":"2024-04-26T11:16:31.688159+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:24:03.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.817+0000 7fc92cdce640 1 -- 172.21.15.69:0/135374937 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc9100afb70 msgr2=0x7fc9100b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:03.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.817+0000 7fc92cdce640 1 --2- 172.21.15.69:0/135374937 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc9100afb70 0x7fc9100b2080 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc91c005f70 tx=0x7fc91c005720 comp rx=0 tx=0).stop 2024-04-25T11:24:03.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.817+0000 7fc92cdce640 1 -- 172.21.15.69:0/135374937 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9281029d0 msgr2=0x7fc92819a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:03.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.817+0000 7fc92cdce640 1 --2- 172.21.15.69:0/135374937 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9281029d0 0x7fc92819a210 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc918006fd0 tx=0x7fc91800b8e0 comp rx=0 tx=0).stop 2024-04-25T11:24:03.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.817+0000 7fc92cdce640 1 -- 172.21.15.69:0/135374937 shutdown_connections 2024-04-25T11:24:03.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.817+0000 7fc92cdce640 1 --2- 172.21.15.69:0/135374937 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc9100afb70 0x7fc9100b2080 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:03.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.817+0000 7fc92cdce640 1 --2- 172.21.15.69:0/135374937 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc928108910 0x7fc9281a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:03.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.817+0000 7fc92cdce640 1 --2- 172.21.15.69:0/135374937 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc928103c10 0x7fc92819a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:03.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.817+0000 7fc92cdce640 1 --2- 172.21.15.69:0/135374937 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9281029d0 0x7fc92819a210 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:03.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.817+0000 7fc92cdce640 1 -- 172.21.15.69:0/135374937 >> 172.21.15.69:0/135374937 conn(0x7fc9280fdba0 msgr2=0x7fc928109e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:03.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.817+0000 7fc92cdce640 1 -- 172.21.15.69:0/135374937 shutdown_connections 2024-04-25T11:24:03.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.818+0000 7fc92cdce640 1 -- 172.21.15.69:0/135374937 wait complete. 2024-04-25T11:24:03.823 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:24:03.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.938+0000 7f35946d1640 1 -- 172.21.15.69:0/2314628558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f358c109a40 msgr2=0x7f358c10be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:03.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.938+0000 7f35946d1640 1 --2- 172.21.15.69:0/2314628558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f358c109a40 0x7f358c10be60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3584009ca0 tx=0x7f358402a8c0 comp rx=0 tx=0).stop 2024-04-25T11:24:03.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.939+0000 7f35946d1640 1 -- 172.21.15.69:0/2314628558 shutdown_connections 2024-04-25T11:24:03.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.939+0000 7f35946d1640 1 --2- 172.21.15.69:0/2314628558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f358c109a40 0x7f358c10be60 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:03.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.939+0000 7f35946d1640 1 --2- 172.21.15.69:0/2314628558 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f358c104d40 0x7f358c105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:03.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.939+0000 7f35946d1640 1 --2- 172.21.15.69:0/2314628558 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f358c103b00 0x7f358c104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:03.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.939+0000 7f35946d1640 1 -- 172.21.15.69:0/2314628558 >> 172.21.15.69:0/2314628558 conn(0x7f358c0fed10 msgr2=0x7f358c1011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:03.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.939+0000 7f35946d1640 1 -- 172.21.15.69:0/2314628558 shutdown_connections 2024-04-25T11:24:03.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.939+0000 7f35946d1640 1 -- 172.21.15.69:0/2314628558 wait complete. 2024-04-25T11:24:03.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.939+0000 7f35946d1640 1 Processor -- start 2024-04-25T11:24:03.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.939+0000 7f35946d1640 1 -- start start 2024-04-25T11:24:03.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.939+0000 7f35946d1640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f358c103b00 0x7f358c19a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:03.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.939+0000 7f35946d1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f358c104d40 0x7f358c19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:03.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.939+0000 7f35946d1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f358c109a40 0x7f358c1a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:03.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.939+0000 7f35946d1640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f358c10f510 con 0x7f358c104d40 2024-04-25T11:24:03.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.939+0000 7f35946d1640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f358c10f390 con 0x7f358c109a40 2024-04-25T11:24:03.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.939+0000 7f35946d1640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f358c10f690 con 0x7f358c103b00 2024-04-25T11:24:03.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.940+0000 7f3591c6c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f358c104d40 0x7f358c19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:03.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.940+0000 7f3591c6c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f358c104d40 0x7f358c19a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45324/0 (socket says 172.21.15.69:45324) 2024-04-25T11:24:03.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.940+0000 7f3591c6c640 1 -- 172.21.15.69:0/1708931461 learned_addr learned my addr 172.21.15.69:0/1708931461 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:24:03.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.940+0000 7f359246d640 1 --2- 172.21.15.69:0/1708931461 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f358c103b00 0x7f358c19a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:03.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.940+0000 7f3592c6e640 1 --2- 172.21.15.69:0/1708931461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f358c109a40 0x7f358c1a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:03.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.940+0000 7f3591c6c640 1 -- 172.21.15.69:0/1708931461 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f358c103b00 msgr2=0x7f358c19a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:03.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.940+0000 7f3591c6c640 1 --2- 172.21.15.69:0/1708931461 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f358c103b00 0x7f358c19a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:03.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.940+0000 7f3591c6c640 1 -- 172.21.15.69:0/1708931461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f358c109a40 msgr2=0x7f358c1a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:03.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.940+0000 7f3591c6c640 1 --2- 172.21.15.69:0/1708931461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f358c109a40 0x7f358c1a15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:03.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.940+0000 7f3591c6c640 1 -- 172.21.15.69:0/1708931461 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f358c1a1c20 con 0x7f358c104d40 2024-04-25T11:24:03.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.940+0000 7f3591c6c640 1 --2- 172.21.15.69:0/1708931461 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f358c104d40 0x7f358c19a620 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f357c00d950 tx=0x7f357c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:03.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.940+0000 7f35837fe640 1 -- 172.21.15.69:0/1708931461 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357c017070 con 0x7f358c104d40 2024-04-25T11:24:03.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.940+0000 7f35837fe640 1 -- 172.21.15.69:0/1708931461 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f357c009c30 con 0x7f358c104d40 2024-04-25T11:24:03.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.940+0000 7f35946d1640 1 -- 172.21.15.69:0/1708931461 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f358c1a1f10 con 0x7f358c104d40 2024-04-25T11:24:03.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.940+0000 7f35837fe640 1 -- 172.21.15.69:0/1708931461 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357c00f620 con 0x7f358c104d40 2024-04-25T11:24:03.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.940+0000 7f35946d1640 1 -- 172.21.15.69:0/1708931461 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f358c1a23d0 con 0x7f358c104d40 2024-04-25T11:24:03.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.941+0000 7f35946d1640 1 -- 172.21.15.69:0/1708931461 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f358c108850 con 0x7f358c104d40 2024-04-25T11:24:03.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.943+0000 7f35837fe640 1 -- 172.21.15.69:0/1708931461 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f357c010460 con 0x7f358c104d40 2024-04-25T11:24:03.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.943+0000 7f35837fe640 1 --2- 172.21.15.69:0/1708931461 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f35780afb70 0x7f35780b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:03.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.944+0000 7f359246d640 1 --2- 172.21.15.69:0/1708931461 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f35780afb70 0x7f35780b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:03.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.944+0000 7f35837fe640 1 -- 172.21.15.69:0/1708931461 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7f357c0de990 con 0x7f358c104d40 2024-04-25T11:24:03.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.944+0000 7f359246d640 1 --2- 172.21.15.69:0/1708931461 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f35780afb70 0x7f35780b2080 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f358c104700 tx=0x7f358800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:03.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:03.948+0000 7f35837fe640 1 -- 172.21.15.69:0/1708931461 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f357c009de0 con 0x7f358c104d40 2024-04-25T11:24:04.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.127+0000 7f35946d1640 1 -- 172.21.15.69:0/1708931461 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f358c0637d0 con 0x7f358c104d40 2024-04-25T11:24:04.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.128+0000 7f35837fe640 1 -- 172.21.15.69:0/1708931461 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+24130 (secure 0 0 0) 0x7f357c0a6ce0 con 0x7f358c104d40 2024-04-25T11:24:04.122 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:24:04.122 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":80,"fsid":"f7ed73dc-02f4-11ef-bc93-c7b262605968","created":"2024-04-25T11:15:37.965281+0000","modified":"2024-04-25T11:23:47.662955+0000","last_up_change":"2024-04-25T11:22:50.691869+0000","last_in_change":"2024-04-25T11:22:30.271959+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:18:57.781263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:23:38.685110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:23:39.943803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T11:23:43.650521+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6a0d179-4998-4754-bf82-2ea3dac54e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6803","nonce":3756025235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6805","nonce":3756025235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6809","nonce":3756025235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6807","nonce":3756025235}]},"public_addr":"172.21.15.69:6803/3756025235","cluster_addr":"172.21.15.69:6805/3756025235","heartbeat_back_addr":"172.21.15.69:6809/3756025235","heartbeat_front_addr":"172.21.15.69:6807/3756025235","state":["exists","up"]},{"osd":1,"uuid":"c0c66930-4233-493f-b2bb-99c473994b59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6801","nonce":3166305213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6803","nonce":3166305213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6807","nonce":3166305213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6805","nonce":3166305213}]},"public_addr":"172.21.15.146:6801/3166305213","cluster_addr":"172.21.15.146:6803/3166305213","heartbeat_back_addr":"172.21.15.146:6807/3166305213","heartbeat_front_addr":"172.21.15.146:6805/3166305213","state":["exists","up"]},{"osd":2,"uuid":"d4ef735b-9786-4cb9-83ef-7c3f0691a729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6801","nonce":1474319175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6803","nonce":1474319175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6807","nonce":1474319175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6805","nonce":1474319175}]},"public_addr":"172.21.15.152:6801/1474319175","cluster_addr":"172.21.15.152:6803/1474319175","heartbeat_back_addr":"172.21.15.152:6807/1474319175","heartbeat_front_addr":"172.21.15.152:6805/1474319175","state":["exists","up"]},{"osd":3,"uuid":"0ab85fb8-5bbb-4ffc-bddf-cae7644a64ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6811","nonce":1809071296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6813","nonce":1809071296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6817","nonce":1809071296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6815","nonce":1809071296}]},"public_addr":"172.21.15.69:6811/1809071296","cluster_addr":"172.21.15.69:6813/1809071296","heartbeat_back_addr":"172.21.15.69:6817/1809071296","heartbeat_front_addr":"172.21.15.69:6815/1809071296","state":["exists","up"]},{"osd":4,"uuid":"273e0592-bbbc-4188-8eeb-46a3879cf632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6809","nonce":1898520972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6811","nonce":1898520972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6815","nonce":1898520972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6813","nonce":1898520972}]},"public_addr":"172.21.15.146:6809/1898520972","cluster_addr":"172.21.15.146:6811/1898520972","heartbeat_back_addr":"172.21.15.146:6815/1898520972","heartbeat_front_addr":"172.21.15.146:6813/1898520972","state":["exists","up"]},{"osd":5,"uuid":"bbeae945-9fa2-48f9-a1d6-8bf18dcaef19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6809","nonce":1996555108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6811","nonce":1996555108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6815","nonce":1996555108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6813","nonce":1996555108}]},"public_addr":"172.21.15.152:6809/1996555108","cluster_addr":"172.21.15.152:6811/1996555108","heartbeat_back_addr":"172.21.15.152:6815/1996555108","heartbeat_front_addr":"172.21.15.152:6813/1996555108","state":["exists","up"]},{"osd":6,"uuid":"dbb61253-30bc-418e-bd75-118c80bbf931","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6819","nonce":491215762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6821","nonce":491215762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6825","nonce":491215762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6823","nonce":491215762}]},"public_addr":"172.21.15.69:6819/491215762","cluster_addr":"172.21.15.69:6821/491215762","heartbeat_back_addr":"172.21.15.69:6825/491215762","heartbeat_front_addr":"172.21.15.69:6823/491215762","state":["exists","up"]},{"osd":7,"uuid":"75f23854-e283-4e3e-94cf-7b4079d83876","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6817","nonce":2330294690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6819","nonce":2330294690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6823","nonce":2330294690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6821","nonce":2330294690}]},"public_addr":"172.21.15.146:6817/2330294690","cluster_addr":"172.21.15.146:6819/2330294690","heartbeat_back_addr":"172.21.15.146:6823/2330294690","heartbeat_front_addr":"172.21.15.146:6821/2330294690","state":["exists","up"]},{"osd":8,"uuid":"1aa1060c-0c14-4abd-be56-5ebfeb78759c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6817","nonce":950669381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6819","nonce":950669381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6823","nonce":950669381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6821","nonce":950669381}]},"public_addr":"172.21.15.152:6817/950669381","cluster_addr":"172.21.15.152:6819/950669381","heartbeat_back_addr":"172.21.15.152:6823/950669381","heartbeat_front_addr":"172.21.15.152:6821/950669381","state":["exists","up"]},{"osd":9,"uuid":"3f4bf800-4090-4138-9bb3-d001f13f450c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6827","nonce":2414186758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6829","nonce":2414186758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6833","nonce":2414186758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6831","nonce":2414186758}]},"public_addr":"172.21.15.69:6827/2414186758","cluster_addr":"172.21.15.69:6829/2414186758","heartbeat_back_addr":"172.21.15.69:6833/2414186758","heartbeat_front_addr":"172.21.15.69:6831/2414186758","state":["exists","up"]},{"osd":10,"uuid":"19e967e5-51ce-4506-80c9-0a95441baf99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6825","nonce":707256400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6827","nonce":707256400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6831","nonce":707256400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6829","nonce":707256400}]},"public_addr":"172.21.15.146:6825/707256400","cluster_addr":"172.21.15.146:6827/707256400","heartbeat_back_addr":"172.21.15.146:6831/707256400","heartbeat_front_addr":"172.21.15.146:6829/707256400","state":["exists","up"]},{"osd":11,"uuid":"7788c2de-6930-4808-be0f-f1fd1ae0cb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6825","nonce":2934336833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6827","nonce":2934336833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6831","nonce":2934336833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6829","nonce":2934336833}]},"public_addr":"172.21.15.152:6825/2934336833","cluster_addr":"172.21.15.152:6827/2934336833","heartbeat_back_addr":"172.21.15.152:6831/2934336833","heartbeat_front_addr":"172.21.15.152:6829/2934336833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:07.497448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:31.227494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:55.027978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:20.713437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:46.214704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:20:11.199983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:20:37.177070+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:02.951240+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:28.963680+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:56.095807+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:22:22.221744+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:22:48.929334+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/1372727546":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6801/1097124842":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6800/1508198215":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3103465990":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3540132960":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3899912190":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6800/1097124842":"2024-04-26T11:16:31.688159+0000","172.21.15.69:0/1199907202":"2024-04-26T11:16:04.860792+0000","172.21.15.69:6801/1508198215":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/2879102529":"2024-04-26T11:16:31.688159+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:24:04.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.130+0000 7f35946d1640 1 -- 172.21.15.69:0/1708931461 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f35780afb70 msgr2=0x7f35780b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:04.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.130+0000 7f35946d1640 1 --2- 172.21.15.69:0/1708931461 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f35780afb70 0x7f35780b2080 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f358c104700 tx=0x7f358800a2d0 comp rx=0 tx=0).stop 2024-04-25T11:24:04.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.130+0000 7f35946d1640 1 -- 172.21.15.69:0/1708931461 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f358c104d40 msgr2=0x7f358c19a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:04.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.130+0000 7f35946d1640 1 --2- 172.21.15.69:0/1708931461 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f358c104d40 0x7f358c19a620 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f357c00d950 tx=0x7f357c00de10 comp rx=0 tx=0).stop 2024-04-25T11:24:04.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.130+0000 7f35946d1640 1 -- 172.21.15.69:0/1708931461 shutdown_connections 2024-04-25T11:24:04.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.130+0000 7f35946d1640 1 --2- 172.21.15.69:0/1708931461 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f35780afb70 0x7f35780b2080 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:04.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.130+0000 7f35946d1640 1 --2- 172.21.15.69:0/1708931461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f358c109a40 0x7f358c1a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:04.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.130+0000 7f35946d1640 1 --2- 172.21.15.69:0/1708931461 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f358c104d40 0x7f358c19a620 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:04.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.130+0000 7f35946d1640 1 --2- 172.21.15.69:0/1708931461 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f358c103b00 0x7f358c19a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:04.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.130+0000 7f35946d1640 1 -- 172.21.15.69:0/1708931461 >> 172.21.15.69:0/1708931461 conn(0x7f358c0fed10 msgr2=0x7f358c105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:04.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.130+0000 7f35946d1640 1 -- 172.21.15.69:0/1708931461 shutdown_connections 2024-04-25T11:24:04.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.131+0000 7f35946d1640 1 -- 172.21.15.69:0/1708931461 wait complete. 2024-04-25T11:24:04.135 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:03 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/135374937' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:24:04.136 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T11:24:04.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:03 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/135374937' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:24:04.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.252+0000 7f305d172640 1 -- 172.21.15.69:0/4237087457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3058102a00 msgr2=0x7f3058103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:04.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.252+0000 7f305d172640 1 --2- 172.21.15.69:0/4237087457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3058102a00 0x7f3058103700 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3048009900 tx=0x7f304802a410 comp rx=0 tx=0).stop 2024-04-25T11:24:04.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.252+0000 7f305d172640 1 -- 172.21.15.69:0/4237087457 shutdown_connections 2024-04-25T11:24:04.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.252+0000 7f305d172640 1 --2- 172.21.15.69:0/4237087457 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3058108940 0x7f305810ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:04.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.252+0000 7f305d172640 1 --2- 172.21.15.69:0/4237087457 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3058103c40 0x7f3058104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:04.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.252+0000 7f305d172640 1 --2- 172.21.15.69:0/4237087457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3058102a00 0x7f3058103700 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:04.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.252+0000 7f305d172640 1 -- 172.21.15.69:0/4237087457 >> 172.21.15.69:0/4237087457 conn(0x7f30580fdbf0 msgr2=0x7f30581000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:04.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.252+0000 7f305d172640 1 -- 172.21.15.69:0/4237087457 shutdown_connections 2024-04-25T11:24:04.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.252+0000 7f305d172640 1 -- 172.21.15.69:0/4237087457 wait complete. 2024-04-25T11:24:04.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.253+0000 7f305d172640 1 Processor -- start 2024-04-25T11:24:04.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.253+0000 7f305d172640 1 -- start start 2024-04-25T11:24:04.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.253+0000 7f305d172640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3058102a00 0x7f305819a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:04.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.253+0000 7f305d172640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3058103c40 0x7f305819a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:04.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.253+0000 7f305d172640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3058108940 0x7f30581a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:04.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.253+0000 7f305d172640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f305810e460 con 0x7f3058108940 2024-04-25T11:24:04.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.253+0000 7f305d172640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f305810e2e0 con 0x7f3058102a00 2024-04-25T11:24:04.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.253+0000 7f305d172640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f305810e5e0 con 0x7f3058103c40 2024-04-25T11:24:04.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.253+0000 7f305759e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3058108940 0x7f30581a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:04.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.253+0000 7f305759e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3058108940 0x7f30581a16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45342/0 (socket says 172.21.15.69:45342) 2024-04-25T11:24:04.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.253+0000 7f305759e640 1 -- 172.21.15.69:0/1162522311 learned_addr learned my addr 172.21.15.69:0/1162522311 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:24:04.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.254+0000 7f3056d9d640 1 --2- 172.21.15.69:0/1162522311 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3058102a00 0x7f305819a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:04.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.254+0000 7f304ffff640 1 --2- 172.21.15.69:0/1162522311 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3058103c40 0x7f305819a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:04.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.254+0000 7f305759e640 1 -- 172.21.15.69:0/1162522311 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3058103c40 msgr2=0x7f305819a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:04.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.254+0000 7f305759e640 1 --2- 172.21.15.69:0/1162522311 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3058103c40 0x7f305819a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:04.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.254+0000 7f305759e640 1 -- 172.21.15.69:0/1162522311 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3058102a00 msgr2=0x7f305819a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:04.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.254+0000 7f305759e640 1 --2- 172.21.15.69:0/1162522311 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3058102a00 0x7f305819a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:04.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.254+0000 7f305759e640 1 -- 172.21.15.69:0/1162522311 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30581a1d20 con 0x7f3058108940 2024-04-25T11:24:04.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.254+0000 7f305759e640 1 --2- 172.21.15.69:0/1162522311 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3058108940 0x7f30581a16b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f305000b970 tx=0x7f305000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:04.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.254+0000 7f3054d99640 1 -- 172.21.15.69:0/1162522311 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3050019070 con 0x7f3058108940 2024-04-25T11:24:04.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.254+0000 7f305d172640 1 -- 172.21.15.69:0/1162522311 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30581a1fb0 con 0x7f3058108940 2024-04-25T11:24:04.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.254+0000 7f3054d99640 1 -- 172.21.15.69:0/1162522311 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3050014440 con 0x7f3058108940 2024-04-25T11:24:04.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.254+0000 7f305d172640 1 -- 172.21.15.69:0/1162522311 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30581a24f0 con 0x7f3058108940 2024-04-25T11:24:04.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.254+0000 7f3054d99640 1 -- 172.21.15.69:0/1162522311 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305001d5b0 con 0x7f3058108940 2024-04-25T11:24:04.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.255+0000 7f305d172640 1 -- 172.21.15.69:0/1162522311 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30581a2780 con 0x7f3058108940 2024-04-25T11:24:04.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.256+0000 7f3054d99640 1 -- 172.21.15.69:0/1162522311 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f305001c070 con 0x7f3058108940 2024-04-25T11:24:04.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.257+0000 7f3054d99640 1 --2- 172.21.15.69:0/1162522311 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f30440afaf0 0x7f30440b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:04.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.257+0000 7f3056d9d640 1 --2- 172.21.15.69:0/1162522311 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f30440afaf0 0x7f30440b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:04.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.257+0000 7f3054d99640 1 -- 172.21.15.69:0/1162522311 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7f30500de820 con 0x7f3058108940 2024-04-25T11:24:04.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.258+0000 7f3056d9d640 1 --2- 172.21.15.69:0/1162522311 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f30440afaf0 0x7f30440b2000 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f304802a930 tx=0x7f3048031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:04.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.261+0000 7f3054d99640 1 -- 172.21.15.69:0/1162522311 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f30500a6b70 con 0x7f3058108940 2024-04-25T11:24:04.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:03 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/135374937' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:24:04.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.477+0000 7f305d172640 1 -- 172.21.15.69:0/1162522311 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f30581a2950 con 0x7f3058108940 2024-04-25T11:24:04.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.869+0000 7f3054d99640 1 -- 172.21.15.69:0/1162522311 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f30500af240 con 0x7f3058108940 2024-04-25T11:24:04.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.872+0000 7f305d172640 1 -- 172.21.15.69:0/1162522311 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f30440afaf0 msgr2=0x7f30440b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:04.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.872+0000 7f305d172640 1 --2- 172.21.15.69:0/1162522311 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f30440afaf0 0x7f30440b2000 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f304802a930 tx=0x7f3048031400 comp rx=0 tx=0).stop 2024-04-25T11:24:04.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.872+0000 7f305d172640 1 -- 172.21.15.69:0/1162522311 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3058108940 msgr2=0x7f30581a16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:04.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.872+0000 7f305d172640 1 --2- 172.21.15.69:0/1162522311 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3058108940 0x7f30581a16b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f305000b970 tx=0x7f305000be30 comp rx=0 tx=0).stop 2024-04-25T11:24:04.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.872+0000 7f305d172640 1 -- 172.21.15.69:0/1162522311 shutdown_connections 2024-04-25T11:24:04.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.872+0000 7f305d172640 1 --2- 172.21.15.69:0/1162522311 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f30440afaf0 0x7f30440b2000 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:04.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.872+0000 7f305d172640 1 --2- 172.21.15.69:0/1162522311 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3058108940 0x7f30581a16b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:04.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.872+0000 7f305d172640 1 --2- 172.21.15.69:0/1162522311 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3058103c40 0x7f305819a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:04.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.872+0000 7f305d172640 1 --2- 172.21.15.69:0/1162522311 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3058102a00 0x7f305819a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:04.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.872+0000 7f305d172640 1 -- 172.21.15.69:0/1162522311 >> 172.21.15.69:0/1162522311 conn(0x7f30580fdbf0 msgr2=0x7f3058074dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:04.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.873+0000 7f305d172640 1 -- 172.21.15.69:0/1162522311 shutdown_connections 2024-04-25T11:24:04.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.873+0000 7f305d172640 1 -- 172.21.15.69:0/1162522311 wait complete. 2024-04-25T11:24:04.878 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-04-25T11:24:04.878 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-04-25T11:24:04.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.996+0000 7f2969f7d640 1 -- 172.21.15.69:0/1836626842 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29641050b0 msgr2=0x7f29641054c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:04.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.996+0000 7f2969f7d640 1 --2- 172.21.15.69:0/1836626842 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29641050b0 0x7f29641054c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2954009900 tx=0x7f295402a410 comp rx=0 tx=0).stop 2024-04-25T11:24:04.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.996+0000 7f2969f7d640 1 -- 172.21.15.69:0/1836626842 shutdown_connections 2024-04-25T11:24:04.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.996+0000 7f2969f7d640 1 --2- 172.21.15.69:0/1836626842 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2964102260 0x7f296410bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:04.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.996+0000 7f2969f7d640 1 --2- 172.21.15.69:0/1836626842 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2964101760 0x7f2964101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:04.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.996+0000 7f2969f7d640 1 --2- 172.21.15.69:0/1836626842 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29641050b0 0x7f29641054c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:04.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.996+0000 7f2969f7d640 1 -- 172.21.15.69:0/1836626842 >> 172.21.15.69:0/1836626842 conn(0x7f29640fd380 msgr2=0x7f29640ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:04.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.996+0000 7f2969f7d640 1 -- 172.21.15.69:0/1836626842 shutdown_connections 2024-04-25T11:24:04.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.996+0000 7f2969f7d640 1 -- 172.21.15.69:0/1836626842 wait complete. 2024-04-25T11:24:04.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.997+0000 7f2969f7d640 1 Processor -- start 2024-04-25T11:24:04.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.997+0000 7f2969f7d640 1 -- start start 2024-04-25T11:24:04.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.997+0000 7f2969f7d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2964101760 0x7f296419a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:04.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.997+0000 7f2969f7d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2964102260 0x7f296419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:04.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.997+0000 7f2969f7d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29641050b0 0x7f29641a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:04.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.997+0000 7f2969f7d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f296410f520 con 0x7f2964102260 2024-04-25T11:24:04.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.997+0000 7f2969f7d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f296410f3a0 con 0x7f29641050b0 2024-04-25T11:24:04.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.997+0000 7f2969f7d640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f296410f6a0 con 0x7f2964101760 2024-04-25T11:24:04.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.997+0000 7f2962ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2964102260 0x7f296419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:04.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.997+0000 7f2962ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2964102260 0x7f296419a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45354/0 (socket says 172.21.15.69:45354) 2024-04-25T11:24:04.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.997+0000 7f2962ffd640 1 -- 172.21.15.69:0/3377504706 learned_addr learned my addr 172.21.15.69:0/3377504706 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:24:04.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.998+0000 7f29637fe640 1 --2- 172.21.15.69:0/3377504706 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2964101760 0x7f296419a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:04.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.998+0000 7f2963fff640 1 --2- 172.21.15.69:0/3377504706 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29641050b0 0x7f29641a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:04.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.998+0000 7f2962ffd640 1 -- 172.21.15.69:0/3377504706 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2964101760 msgr2=0x7f296419a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:04.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.998+0000 7f2962ffd640 1 --2- 172.21.15.69:0/3377504706 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2964101760 0x7f296419a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:04.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.998+0000 7f2962ffd640 1 -- 172.21.15.69:0/3377504706 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29641050b0 msgr2=0x7f29641a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:04.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.998+0000 7f2962ffd640 1 --2- 172.21.15.69:0/3377504706 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29641050b0 0x7f29641a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:04.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.998+0000 7f2962ffd640 1 -- 172.21.15.69:0/3377504706 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29641a1cc0 con 0x7f2964102260 2024-04-25T11:24:04.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.998+0000 7f2962ffd640 1 --2- 172.21.15.69:0/3377504706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2964102260 0x7f296419a6c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f295800d950 tx=0x7f295800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:04.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.998+0000 7f2960ff9640 1 -- 172.21.15.69:0/3377504706 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2958017070 con 0x7f2964102260 2024-04-25T11:24:04.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.998+0000 7f2969f7d640 1 -- 172.21.15.69:0/3377504706 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29641a1fb0 con 0x7f2964102260 2024-04-25T11:24:04.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.998+0000 7f2960ff9640 1 -- 172.21.15.69:0/3377504706 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2958009c30 con 0x7f2964102260 2024-04-25T11:24:04.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.998+0000 7f2960ff9640 1 -- 172.21.15.69:0/3377504706 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f295800f620 con 0x7f2964102260 2024-04-25T11:24:04.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.998+0000 7f2969f7d640 1 -- 172.21.15.69:0/3377504706 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29641a2410 con 0x7f2964102260 2024-04-25T11:24:04.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:04.999+0000 7f2969f7d640 1 -- 172.21.15.69:0/3377504706 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2964108a40 con 0x7f2964102260 2024-04-25T11:24:04.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:05.000+0000 7f2960ff9640 1 -- 172.21.15.69:0/3377504706 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f2958005030 con 0x7f2964102260 2024-04-25T11:24:04.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:05.001+0000 7f2960ff9640 1 --2- 172.21.15.69:0/3377504706 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f294c0afaf0 0x7f294c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:04.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:05.001+0000 7f29637fe640 1 --2- 172.21.15.69:0/3377504706 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f294c0afaf0 0x7f294c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:04.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:05.001+0000 7f2960ff9640 1 -- 172.21.15.69:0/3377504706 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7f29580de9a0 con 0x7f2964102260 2024-04-25T11:24:04.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:05.001+0000 7f29637fe640 1 --2- 172.21.15.69:0/3377504706 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f294c0afaf0 0x7f294c0b2000 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f295402a930 tx=0x7f2954031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:04.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:05.005+0000 7f2960ff9640 1 -- 172.21.15.69:0/3377504706 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2958009de0 con 0x7f2964102260 2024-04-25T11:24:05.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:04 smithi069 ceph-mon[29188]: pgmap v296: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-25T11:24:05.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:04 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/1708931461' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:24:05.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:04 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/1162522311' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T11:24:05.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:05.224+0000 7f2969f7d640 1 -- 172.21.15.69:0/3377504706 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f29640637d0 con 0x7f2964102260 2024-04-25T11:24:05.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:04 smithi152 ceph-mon[32701]: pgmap v296: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-25T11:24:05.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:04 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/1708931461' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:24:05.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:04 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/1162522311' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T11:24:05.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:04 smithi146 ceph-mon[32821]: pgmap v296: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-25T11:24:05.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:04 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/1708931461' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:24:05.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:04 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/1162522311' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T11:24:05.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:05.876+0000 7f2960ff9640 1 -- 172.21.15.69:0/3377504706 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f29580a6cf0 con 0x7f2964102260 2024-04-25T11:24:05.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:05.879+0000 7f2969f7d640 1 -- 172.21.15.69:0/3377504706 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f294c0afaf0 msgr2=0x7f294c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:05.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:05.879+0000 7f2969f7d640 1 --2- 172.21.15.69:0/3377504706 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f294c0afaf0 0x7f294c0b2000 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f295402a930 tx=0x7f2954031400 comp rx=0 tx=0).stop 2024-04-25T11:24:05.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:05.879+0000 7f2969f7d640 1 -- 172.21.15.69:0/3377504706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2964102260 msgr2=0x7f296419a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:05.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:05.879+0000 7f2969f7d640 1 --2- 172.21.15.69:0/3377504706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2964102260 0x7f296419a6c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f295800d950 tx=0x7f295800de10 comp rx=0 tx=0).stop 2024-04-25T11:24:05.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:05.879+0000 7f2969f7d640 1 -- 172.21.15.69:0/3377504706 shutdown_connections 2024-04-25T11:24:05.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:05.879+0000 7f2969f7d640 1 --2- 172.21.15.69:0/3377504706 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f294c0afaf0 0x7f294c0b2000 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:05.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:05.879+0000 7f2969f7d640 1 --2- 172.21.15.69:0/3377504706 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f29641050b0 0x7f29641a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:05.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:05.879+0000 7f2969f7d640 1 --2- 172.21.15.69:0/3377504706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2964102260 0x7f296419a6c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:05.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:05.879+0000 7f2969f7d640 1 --2- 172.21.15.69:0/3377504706 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2964101760 0x7f296419a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:05.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:05.879+0000 7f2969f7d640 1 -- 172.21.15.69:0/3377504706 >> 172.21.15.69:0/3377504706 conn(0x7f29640fd380 msgr2=0x7f2964105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:05.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:05.880+0000 7f2969f7d640 1 -- 172.21.15.69:0/3377504706 shutdown_connections 2024-04-25T11:24:05.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:05.880+0000 7f2969f7d640 1 -- 172.21.15.69:0/3377504706 wait complete. 2024-04-25T11:24:05.888 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-25T11:24:06.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.008+0000 7ff4a3a34640 1 -- 172.21.15.69:0/3509826506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff49c103b10 msgr2=0x7ff49c103fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:06.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.008+0000 7ff4a3a34640 1 --2- 172.21.15.69:0/3509826506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff49c103b10 0x7ff49c103fe0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff498009900 tx=0x7ff49802a410 comp rx=0 tx=0).stop 2024-04-25T11:24:06.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.008+0000 7ff4a3a34640 1 -- 172.21.15.69:0/3509826506 shutdown_connections 2024-04-25T11:24:06.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.008+0000 7ff4a3a34640 1 --2- 172.21.15.69:0/3509826506 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff49c104670 0x7ff49c10adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:06.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.008+0000 7ff4a3a34640 1 --2- 172.21.15.69:0/3509826506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff49c103b10 0x7ff49c103fe0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:06.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.008+0000 7ff4a3a34640 1 --2- 172.21.15.69:0/3509826506 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff49c102fb0 0x7ff49c1033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:06.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.008+0000 7ff4a3a34640 1 -- 172.21.15.69:0/3509826506 >> 172.21.15.69:0/3509826506 conn(0x7ff49c0fd350 msgr2=0x7ff49c0ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:06.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.008+0000 7ff4a3a34640 1 -- 172.21.15.69:0/3509826506 shutdown_connections 2024-04-25T11:24:06.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.008+0000 7ff4a3a34640 1 -- 172.21.15.69:0/3509826506 wait complete. 2024-04-25T11:24:06.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.008+0000 7ff4a3a34640 1 Processor -- start 2024-04-25T11:24:06.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.008+0000 7ff4a3a34640 1 -- start start 2024-04-25T11:24:06.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.009+0000 7ff4a3a34640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff49c102fb0 0x7ff49c198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:06.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.009+0000 7ff4a3a34640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff49c103b10 0x7ff49c199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:06.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.009+0000 7ff4a3a34640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff49c19e310 0x7ff49c1a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:06.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.009+0000 7ff4a3a34640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff49c10e5f0 con 0x7ff49c103b10 2024-04-25T11:24:06.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.009+0000 7ff4a3a34640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff49c10e470 con 0x7ff49c102fb0 2024-04-25T11:24:06.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.009+0000 7ff4a3a34640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff49c10e770 con 0x7ff49c19e310 2024-04-25T11:24:06.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.009+0000 7ff4a0fcf640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff49c103b10 0x7ff49c199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:06.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.009+0000 7ff4a0fcf640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff49c103b10 0x7ff49c199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45374/0 (socket says 172.21.15.69:45374) 2024-04-25T11:24:06.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.009+0000 7ff4a0fcf640 1 -- 172.21.15.69:0/1771915752 learned_addr learned my addr 172.21.15.69:0/1771915752 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:24:06.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.009+0000 7ff4a17d0640 1 --2- 172.21.15.69:0/1771915752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff49c102fb0 0x7ff49c198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:06.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.009+0000 7ff4a1fd1640 1 --2- 172.21.15.69:0/1771915752 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff49c19e310 0x7ff49c1a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:06.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.009+0000 7ff4a0fcf640 1 -- 172.21.15.69:0/1771915752 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff49c19e310 msgr2=0x7ff49c1a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:06.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.009+0000 7ff4a0fcf640 1 --2- 172.21.15.69:0/1771915752 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff49c19e310 0x7ff49c1a0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:06.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.009+0000 7ff4a0fcf640 1 -- 172.21.15.69:0/1771915752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff49c102fb0 msgr2=0x7ff49c198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:06.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.009+0000 7ff4a0fcf640 1 --2- 172.21.15.69:0/1771915752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff49c102fb0 0x7ff49c198e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:06.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.009+0000 7ff4a0fcf640 1 -- 172.21.15.69:0/1771915752 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff49c1a0da0 con 0x7ff49c103b10 2024-04-25T11:24:06.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.009+0000 7ff4a1fd1640 1 --2- 172.21.15.69:0/1771915752 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff49c19e310 0x7ff49c1a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:24:06.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.009+0000 7ff4a0fcf640 1 --2- 172.21.15.69:0/1771915752 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff49c103b10 0x7ff49c199390 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff498009230 tx=0x7ff4980040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:06.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.009+0000 7ff4927fc640 1 -- 172.21.15.69:0/1771915752 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff49803f030 con 0x7ff49c103b10 2024-04-25T11:24:06.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.009+0000 7ff4927fc640 1 -- 172.21.15.69:0/1771915752 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4980044c0 con 0x7ff49c103b10 2024-04-25T11:24:06.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.010+0000 7ff4a3a34640 1 -- 172.21.15.69:0/1771915752 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff49c1a1030 con 0x7ff49c103b10 2024-04-25T11:24:06.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.010+0000 7ff4927fc640 1 -- 172.21.15.69:0/1771915752 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff498035070 con 0x7ff49c103b10 2024-04-25T11:24:06.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.010+0000 7ff4a3a34640 1 -- 172.21.15.69:0/1771915752 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff49c1a1520 con 0x7ff49c103b10 2024-04-25T11:24:06.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.011+0000 7ff4a3a34640 1 -- 172.21.15.69:0/1771915752 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff49c06b2d0 con 0x7ff49c103b10 2024-04-25T11:24:06.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.012+0000 7ff4927fc640 1 -- 172.21.15.69:0/1771915752 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7ff498042030 con 0x7ff49c103b10 2024-04-25T11:24:06.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.013+0000 7ff4927fc640 1 --2- 172.21.15.69:0/1771915752 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ff4880afaf0 0x7ff4880b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:06.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.013+0000 7ff4a17d0640 1 --2- 172.21.15.69:0/1771915752 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ff4880afaf0 0x7ff4880b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:06.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.013+0000 7ff4927fc640 1 -- 172.21.15.69:0/1771915752 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7ff4980ff8a0 con 0x7ff49c103b10 2024-04-25T11:24:06.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.014+0000 7ff4a17d0640 1 --2- 172.21.15.69:0/1771915752 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ff4880afaf0 0x7ff4880b2000 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff48c005fd0 tx=0x7ff48c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:06.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.017+0000 7ff4927fc640 1 -- 172.21.15.69:0/1771915752 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff4980c8240 con 0x7ff49c103b10 2024-04-25T11:24:06.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:05 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/1162522311' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T11:24:06.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:05 smithi069 ceph-mon[29188]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T11:24:06.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:05 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/3377504706' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-25T11:24:06.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:05 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/1162522311' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T11:24:06.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:05 smithi152 ceph-mon[32701]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T11:24:06.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:05 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/3377504706' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-25T11:24:06.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.241+0000 7ff4a3a34640 1 -- 172.21.15.69:0/1771915752 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7ff49c1073c0 con 0x7ff49c103b10 2024-04-25T11:24:06.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:05 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/1162522311' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T11:24:06.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:05 smithi146 ceph-mon[32821]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T11:24:06.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:05 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/3377504706' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-25T11:24:06.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.883+0000 7ff4927fc640 1 -- 172.21.15.69:0/1771915752 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7ff4980c7b40 con 0x7ff49c103b10 2024-04-25T11:24:06.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.885+0000 7ff4a3a34640 1 -- 172.21.15.69:0/1771915752 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ff4880afaf0 msgr2=0x7ff4880b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:06.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.885+0000 7ff4a3a34640 1 --2- 172.21.15.69:0/1771915752 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ff4880afaf0 0x7ff4880b2000 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff48c005fd0 tx=0x7ff48c005850 comp rx=0 tx=0).stop 2024-04-25T11:24:06.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.886+0000 7ff4a3a34640 1 -- 172.21.15.69:0/1771915752 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff49c103b10 msgr2=0x7ff49c199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:06.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.886+0000 7ff4a3a34640 1 --2- 172.21.15.69:0/1771915752 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff49c103b10 0x7ff49c199390 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff498009230 tx=0x7ff4980040c0 comp rx=0 tx=0).stop 2024-04-25T11:24:06.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.886+0000 7ff4a3a34640 1 -- 172.21.15.69:0/1771915752 shutdown_connections 2024-04-25T11:24:06.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.886+0000 7ff4a3a34640 1 --2- 172.21.15.69:0/1771915752 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ff4880afaf0 0x7ff4880b2000 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:06.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.886+0000 7ff4a3a34640 1 --2- 172.21.15.69:0/1771915752 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff49c19e310 0x7ff49c1a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:06.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.886+0000 7ff4a3a34640 1 --2- 172.21.15.69:0/1771915752 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff49c103b10 0x7ff49c199390 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:06.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.886+0000 7ff4a3a34640 1 --2- 172.21.15.69:0/1771915752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff49c102fb0 0x7ff49c198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:06.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.886+0000 7ff4a3a34640 1 -- 172.21.15.69:0/1771915752 >> 172.21.15.69:0/1771915752 conn(0x7ff49c0fd350 msgr2=0x7ff49c0ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:06.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.886+0000 7ff4a3a34640 1 -- 172.21.15.69:0/1771915752 shutdown_connections 2024-04-25T11:24:06.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:06.886+0000 7ff4a3a34640 1 -- 172.21.15.69:0/1771915752 wait complete. 2024-04-25T11:24:06.897 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-25T11:24:07.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.015+0000 7f6e6ee30640 1 -- 172.21.15.69:0/3770762242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e68109a30 msgr2=0x7f6e6810be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:07.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.015+0000 7f6e6ee30640 1 --2- 172.21.15.69:0/3770762242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e68109a30 0x7f6e6810be50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6e60009ca0 tx=0x7f6e6002a8c0 comp rx=0 tx=0).stop 2024-04-25T11:24:07.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.015+0000 7f6e6ee30640 1 -- 172.21.15.69:0/3770762242 shutdown_connections 2024-04-25T11:24:07.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.015+0000 7f6e6ee30640 1 --2- 172.21.15.69:0/3770762242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e68109a30 0x7f6e6810be50 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:07.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.015+0000 7f6e6ee30640 1 --2- 172.21.15.69:0/3770762242 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e68104d30 0x7f6e68105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:07.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.015+0000 7f6e6ee30640 1 --2- 172.21.15.69:0/3770762242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6e68103af0 0x7f6e681047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:07.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.015+0000 7f6e6ee30640 1 -- 172.21.15.69:0/3770762242 >> 172.21.15.69:0/3770762242 conn(0x7f6e680fece0 msgr2=0x7f6e68101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:07.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.015+0000 7f6e6ee30640 1 -- 172.21.15.69:0/3770762242 shutdown_connections 2024-04-25T11:24:07.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.015+0000 7f6e6ee30640 1 -- 172.21.15.69:0/3770762242 wait complete. 2024-04-25T11:24:07.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.016+0000 7f6e6ee30640 1 Processor -- start 2024-04-25T11:24:07.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.016+0000 7f6e6ee30640 1 -- start start 2024-04-25T11:24:07.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.016+0000 7f6e6ee30640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e68103af0 0x7f6e6819a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:07.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.016+0000 7f6e6ee30640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6e68104d30 0x7f6e6819a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:07.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.016+0000 7f6e6ee30640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e68109a30 0x7f6e681a15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:07.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.016+0000 7f6e6ee30640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6810f520 con 0x7f6e68103af0 2024-04-25T11:24:07.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.016+0000 7f6e6ee30640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6810f3a0 con 0x7f6e68104d30 2024-04-25T11:24:07.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.016+0000 7f6e6ee30640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6810f6a0 con 0x7f6e68109a30 2024-04-25T11:24:07.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.016+0000 7f6e6cbcc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e68103af0 0x7f6e6819a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:07.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.016+0000 7f6e6cbcc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e68103af0 0x7f6e6819a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45386/0 (socket says 172.21.15.69:45386) 2024-04-25T11:24:07.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.016+0000 7f6e6cbcc640 1 -- 172.21.15.69:0/973231051 learned_addr learned my addr 172.21.15.69:0/973231051 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:24:07.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.016+0000 7f6e67fff640 1 --2- 172.21.15.69:0/973231051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6e68104d30 0x7f6e6819a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:07.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.016+0000 7f6e6d3cd640 1 --2- 172.21.15.69:0/973231051 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e68109a30 0x7f6e681a15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:07.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.017+0000 7f6e6cbcc640 1 -- 172.21.15.69:0/973231051 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e68109a30 msgr2=0x7f6e681a15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:07.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.017+0000 7f6e6cbcc640 1 --2- 172.21.15.69:0/973231051 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e68109a30 0x7f6e681a15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:07.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.017+0000 7f6e6cbcc640 1 -- 172.21.15.69:0/973231051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6e68104d30 msgr2=0x7f6e6819a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:07.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.017+0000 7f6e6cbcc640 1 --2- 172.21.15.69:0/973231051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6e68104d30 0x7f6e6819a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:07.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.017+0000 7f6e6cbcc640 1 -- 172.21.15.69:0/973231051 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e681a1c60 con 0x7f6e68103af0 2024-04-25T11:24:07.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.017+0000 7f6e6cbcc640 1 --2- 172.21.15.69:0/973231051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e68103af0 0x7f6e6819a120 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c00e9f0 tx=0x7f6e5c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:07.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.017+0000 7f6e65ffb640 1 -- 172.21.15.69:0/973231051 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c00ccc0 con 0x7f6e68103af0 2024-04-25T11:24:07.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.017+0000 7f6e6ee30640 1 -- 172.21.15.69:0/973231051 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e681a1f50 con 0x7f6e68103af0 2024-04-25T11:24:07.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.017+0000 7f6e65ffb640 1 -- 172.21.15.69:0/973231051 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6e5c0044b0 con 0x7f6e68103af0 2024-04-25T11:24:07.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.017+0000 7f6e65ffb640 1 -- 172.21.15.69:0/973231051 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c010430 con 0x7f6e68103af0 2024-04-25T11:24:07.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.017+0000 7f6e6ee30640 1 -- 172.21.15.69:0/973231051 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e681a23b0 con 0x7f6e68103af0 2024-04-25T11:24:07.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.018+0000 7f6e6ee30640 1 -- 172.21.15.69:0/973231051 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e6806a160 con 0x7f6e68103af0 2024-04-25T11:24:07.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.020+0000 7f6e65ffb640 1 -- 172.21.15.69:0/973231051 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f6e5c003f90 con 0x7f6e68103af0 2024-04-25T11:24:07.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.020+0000 7f6e65ffb640 1 --2- 172.21.15.69:0/973231051 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f6e580afaf0 0x7f6e580b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:07.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.020+0000 7f6e67fff640 1 --2- 172.21.15.69:0/973231051 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f6e580afaf0 0x7f6e580b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:07.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.020+0000 7f6e65ffb640 1 -- 172.21.15.69:0/973231051 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7f6e5c0df260 con 0x7f6e68103af0 2024-04-25T11:24:07.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.021+0000 7f6e67fff640 1 --2- 172.21.15.69:0/973231051 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f6e580afaf0 0x7f6e580b2000 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6e6819b1c0 tx=0x7f6e54005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:07.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.024+0000 7f6e65ffb640 1 -- 172.21.15.69:0/973231051 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6e5c0a7c90 con 0x7f6e68103af0 2024-04-25T11:24:07.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:06 smithi069 ceph-mon[29188]: pgmap v297: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-25T11:24:07.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:06 smithi069 ceph-mon[29188]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T11:24:07.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:06 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/3377504706' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-25T11:24:07.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:06 smithi069 ceph-mon[29188]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T11:24:07.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:06 smithi069 ceph-mon[29188]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T11:24:07.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:06 smithi069 ceph-mon[29188]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-25T11:24:07.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:06 smithi069 ceph-mon[29188]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-25T11:24:07.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:06 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/1771915752' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T11:24:07.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:06 smithi152 ceph-mon[32701]: pgmap v297: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-25T11:24:07.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:06 smithi152 ceph-mon[32701]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T11:24:07.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:06 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/3377504706' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-25T11:24:07.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:06 smithi152 ceph-mon[32701]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T11:24:07.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:06 smithi152 ceph-mon[32701]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T11:24:07.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:06 smithi152 ceph-mon[32701]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-25T11:24:07.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:06 smithi152 ceph-mon[32701]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-25T11:24:07.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:06 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/1771915752' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T11:24:07.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.240+0000 7f6e6ee30640 1 -- 172.21.15.69:0/973231051 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f6e68108890 con 0x7f6e68103af0 2024-04-25T11:24:07.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:06 smithi146 ceph-mon[32821]: pgmap v297: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-25T11:24:07.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:06 smithi146 ceph-mon[32821]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T11:24:07.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:06 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/3377504706' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-25T11:24:07.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:06 smithi146 ceph-mon[32821]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T11:24:07.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:06 smithi146 ceph-mon[32821]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T11:24:07.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:06 smithi146 ceph-mon[32821]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-25T11:24:07.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:06 smithi146 ceph-mon[32821]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-25T11:24:07.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:06 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/1771915752' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T11:24:07.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.898+0000 7f6e65ffb640 1 -- 172.21.15.69:0/973231051 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) ==== 115+0+0 (secure 0 0 0) 0x7f6e68108890 con 0x7f6e68103af0 2024-04-25T11:24:07.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.901+0000 7f6e6ee30640 1 -- 172.21.15.69:0/973231051 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f6e580afaf0 msgr2=0x7f6e580b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:07.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.901+0000 7f6e6ee30640 1 --2- 172.21.15.69:0/973231051 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f6e580afaf0 0x7f6e580b2000 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6e6819b1c0 tx=0x7f6e54005f90 comp rx=0 tx=0).stop 2024-04-25T11:24:07.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.901+0000 7f6e6ee30640 1 -- 172.21.15.69:0/973231051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e68103af0 msgr2=0x7f6e6819a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:07.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.901+0000 7f6e6ee30640 1 --2- 172.21.15.69:0/973231051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e68103af0 0x7f6e6819a120 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c00e9f0 tx=0x7f6e5c00eeb0 comp rx=0 tx=0).stop 2024-04-25T11:24:07.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.901+0000 7f6e6ee30640 1 -- 172.21.15.69:0/973231051 shutdown_connections 2024-04-25T11:24:07.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.901+0000 7f6e6ee30640 1 --2- 172.21.15.69:0/973231051 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f6e580afaf0 0x7f6e580b2000 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:07.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.901+0000 7f6e6ee30640 1 --2- 172.21.15.69:0/973231051 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e68109a30 0x7f6e681a15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:07.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.901+0000 7f6e6ee30640 1 --2- 172.21.15.69:0/973231051 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6e68104d30 0x7f6e6819a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:07.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.901+0000 7f6e6ee30640 1 --2- 172.21.15.69:0/973231051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e68103af0 0x7f6e6819a120 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:07.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.901+0000 7f6e6ee30640 1 -- 172.21.15.69:0/973231051 >> 172.21.15.69:0/973231051 conn(0x7f6e680fece0 msgr2=0x7f6e68105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:07.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.901+0000 7f6e6ee30640 1 -- 172.21.15.69:0/973231051 shutdown_connections 2024-04-25T11:24:07.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:07.901+0000 7f6e6ee30640 1 -- 172.21.15.69:0/973231051 wait complete. 2024-04-25T11:24:07.910 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-04-25T11:24:07.911 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-25T11:24:08.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.029+0000 7f9ca5538640 1 -- 172.21.15.69:0/504276725 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ca0103af0 msgr2=0x7f9ca01047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:08.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.029+0000 7f9ca5538640 1 --2- 172.21.15.69:0/504276725 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ca0103af0 0x7f9ca01047f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9c90009900 tx=0x7f9c9002a410 comp rx=0 tx=0).stop 2024-04-25T11:24:08.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.030+0000 7f9ca5538640 1 -- 172.21.15.69:0/504276725 shutdown_connections 2024-04-25T11:24:08.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.030+0000 7f9ca5538640 1 --2- 172.21.15.69:0/504276725 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9ca0109a30 0x7f9ca010be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:08.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.030+0000 7f9ca5538640 1 --2- 172.21.15.69:0/504276725 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9ca0104d30 0x7f9ca0105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:08.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.030+0000 7f9ca5538640 1 --2- 172.21.15.69:0/504276725 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ca0103af0 0x7f9ca01047f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:08.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.030+0000 7f9ca5538640 1 -- 172.21.15.69:0/504276725 >> 172.21.15.69:0/504276725 conn(0x7f9ca00fece0 msgr2=0x7f9ca0101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:08.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.030+0000 7f9ca5538640 1 -- 172.21.15.69:0/504276725 shutdown_connections 2024-04-25T11:24:08.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.030+0000 7f9ca5538640 1 -- 172.21.15.69:0/504276725 wait complete. 2024-04-25T11:24:08.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.030+0000 7f9ca5538640 1 Processor -- start 2024-04-25T11:24:08.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.030+0000 7f9ca5538640 1 -- start start 2024-04-25T11:24:08.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.031+0000 7f9ca5538640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ca0103af0 0x7f9ca019a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:08.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.031+0000 7f9ca5538640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9ca0104d30 0x7f9ca019a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:08.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.031+0000 7f9ca5538640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9ca0109a30 0x7f9ca01a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:08.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.031+0000 7f9ca5538640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca010f480 con 0x7f9ca0103af0 2024-04-25T11:24:08.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.031+0000 7f9ca5538640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca010f300 con 0x7f9ca0104d30 2024-04-25T11:24:08.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.031+0000 7f9ca5538640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca010f600 con 0x7f9ca0109a30 2024-04-25T11:24:08.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.031+0000 7f9c9effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ca0103af0 0x7f9ca019a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:08.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.031+0000 7f9c9effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ca0103af0 0x7f9ca019a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45406/0 (socket says 172.21.15.69:45406) 2024-04-25T11:24:08.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.031+0000 7f9c9effd640 1 -- 172.21.15.69:0/1579365210 learned_addr learned my addr 172.21.15.69:0/1579365210 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:24:08.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.031+0000 7f9c9e7fc640 1 --2- 172.21.15.69:0/1579365210 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9ca0104d30 0x7f9ca019a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:08.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.031+0000 7f9c9f7fe640 1 --2- 172.21.15.69:0/1579365210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9ca0109a30 0x7f9ca01a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:08.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.031+0000 7f9c9effd640 1 -- 172.21.15.69:0/1579365210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9ca0109a30 msgr2=0x7f9ca01a1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:08.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.031+0000 7f9c9effd640 1 --2- 172.21.15.69:0/1579365210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9ca0109a30 0x7f9ca01a1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:08.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.031+0000 7f9c9effd640 1 -- 172.21.15.69:0/1579365210 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9ca0104d30 msgr2=0x7f9ca019a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:08.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.031+0000 7f9c9effd640 1 --2- 172.21.15.69:0/1579365210 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9ca0104d30 0x7f9ca019a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:08.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.031+0000 7f9c9effd640 1 -- 172.21.15.69:0/1579365210 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca01a1c00 con 0x7f9ca0103af0 2024-04-25T11:24:08.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.031+0000 7f9c9effd640 1 --2- 172.21.15.69:0/1579365210 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ca0103af0 0x7f9ca019a1e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9c90005760 tx=0x7f9c90004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:08.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.031+0000 7f9c87fff640 1 -- 172.21.15.69:0/1579365210 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9003f030 con 0x7f9ca0103af0 2024-04-25T11:24:08.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.031+0000 7f9c87fff640 1 -- 172.21.15.69:0/1579365210 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c90004440 con 0x7f9ca0103af0 2024-04-25T11:24:08.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.031+0000 7f9ca5538640 1 -- 172.21.15.69:0/1579365210 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ca01a1e90 con 0x7f9ca0103af0 2024-04-25T11:24:08.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.031+0000 7f9c87fff640 1 -- 172.21.15.69:0/1579365210 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c90035070 con 0x7f9ca0103af0 2024-04-25T11:24:08.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.031+0000 7f9ca5538640 1 -- 172.21.15.69:0/1579365210 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ca01a2290 con 0x7f9ca0103af0 2024-04-25T11:24:08.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.032+0000 7f9ca5538640 1 -- 172.21.15.69:0/1579365210 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ca0108840 con 0x7f9ca0103af0 2024-04-25T11:24:08.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.034+0000 7f9c87fff640 1 -- 172.21.15.69:0/1579365210 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f9c90042030 con 0x7f9ca0103af0 2024-04-25T11:24:08.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.034+0000 7f9c87fff640 1 --2- 172.21.15.69:0/1579365210 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f9c880afaf0 0x7f9c880b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:08.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.034+0000 7f9c9e7fc640 1 --2- 172.21.15.69:0/1579365210 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f9c880afaf0 0x7f9c880b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:08.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.035+0000 7f9c87fff640 1 -- 172.21.15.69:0/1579365210 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7f9c900ff620 con 0x7f9ca0103af0 2024-04-25T11:24:08.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.035+0000 7f9c9e7fc640 1 --2- 172.21.15.69:0/1579365210 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f9c880afaf0 0x7f9c880b2000 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9ca019b280 tx=0x7f9c9400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:08.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.038+0000 7f9c87fff640 1 -- 172.21.15.69:0/1579365210 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9c900c7fc0 con 0x7f9ca0103af0 2024-04-25T11:24:08.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:07 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/1771915752' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T11:24:08.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:07 smithi069 ceph-mon[29188]: mds.? [v2:172.21.15.146:6836/2730317134,v1:172.21.15.146:6837/2730317134] up:active 2024-04-25T11:24:08.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:07 smithi069 ceph-mon[29188]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T11:24:08.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:07 smithi069 ceph-mon[29188]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-25T11:24:08.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:07 smithi069 ceph-mon[29188]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-25T11:24:08.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:07 smithi069 ceph-mon[29188]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-25T11:24:08.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:07 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/973231051' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T11:24:08.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:07 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/1771915752' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T11:24:08.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:07 smithi152 ceph-mon[32701]: mds.? [v2:172.21.15.146:6836/2730317134,v1:172.21.15.146:6837/2730317134] up:active 2024-04-25T11:24:08.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:07 smithi152 ceph-mon[32701]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T11:24:08.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:07 smithi152 ceph-mon[32701]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-25T11:24:08.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:07 smithi152 ceph-mon[32701]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-25T11:24:08.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:07 smithi152 ceph-mon[32701]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-25T11:24:08.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:07 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/973231051' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T11:24:08.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.251+0000 7f9ca5538640 1 -- 172.21.15.69:0/1579365210 --> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f9ca0062fe0 con 0x7f9c880afaf0 2024-04-25T11:24:08.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.280+0000 7f9c87fff640 1 -- 172.21.15.69:0/1579365210 <== mgr.14152 v2:172.21.15.69:6800/4220065409 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9ca0062fe0 con 0x7f9c880afaf0 2024-04-25T11:24:08.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.282+0000 7f9ca5538640 1 -- 172.21.15.69:0/1579365210 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f9c880afaf0 msgr2=0x7f9c880b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:08.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.282+0000 7f9ca5538640 1 --2- 172.21.15.69:0/1579365210 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f9c880afaf0 0x7f9c880b2000 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9ca019b280 tx=0x7f9c9400a2d0 comp rx=0 tx=0).stop 2024-04-25T11:24:08.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.282+0000 7f9ca5538640 1 -- 172.21.15.69:0/1579365210 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ca0103af0 msgr2=0x7f9ca019a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:08.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.282+0000 7f9ca5538640 1 --2- 172.21.15.69:0/1579365210 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ca0103af0 0x7f9ca019a1e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9c90005760 tx=0x7f9c90004040 comp rx=0 tx=0).stop 2024-04-25T11:24:08.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.282+0000 7f9ca5538640 1 -- 172.21.15.69:0/1579365210 shutdown_connections 2024-04-25T11:24:08.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.282+0000 7f9ca5538640 1 --2- 172.21.15.69:0/1579365210 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f9c880afaf0 0x7f9c880b2000 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:08.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.282+0000 7f9ca5538640 1 --2- 172.21.15.69:0/1579365210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9ca0109a30 0x7f9ca01a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:08.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.282+0000 7f9ca5538640 1 --2- 172.21.15.69:0/1579365210 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9ca0104d30 0x7f9ca019a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:08.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.282+0000 7f9ca5538640 1 --2- 172.21.15.69:0/1579365210 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ca0103af0 0x7f9ca019a1e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:08.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.282+0000 7f9ca5538640 1 -- 172.21.15.69:0/1579365210 >> 172.21.15.69:0/1579365210 conn(0x7f9ca00fece0 msgr2=0x7f9ca0105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:08.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.283+0000 7f9ca5538640 1 -- 172.21.15.69:0/1579365210 shutdown_connections 2024-04-25T11:24:08.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.283+0000 7f9ca5538640 1 -- 172.21.15.69:0/1579365210 wait complete. 2024-04-25T11:24:08.288 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-25T11:24:08.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:07 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/1771915752' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T11:24:08.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:07 smithi146 ceph-mon[32821]: mds.? [v2:172.21.15.146:6836/2730317134,v1:172.21.15.146:6837/2730317134] up:active 2024-04-25T11:24:08.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:07 smithi146 ceph-mon[32821]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T11:24:08.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:07 smithi146 ceph-mon[32821]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-25T11:24:08.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:07 smithi146 ceph-mon[32821]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-25T11:24:08.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:07 smithi146 ceph-mon[32821]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-25T11:24:08.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:07 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/973231051' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T11:24:08.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.406+0000 7f494a492640 1 -- 172.21.15.69:0/3227572163 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4944102190 msgr2=0x7f4944102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:08.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.406+0000 7f494a492640 1 --2- 172.21.15.69:0/3227572163 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4944102190 0x7f4944102e90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f493400aa10 tx=0x7f493402b720 comp rx=0 tx=0).stop 2024-04-25T11:24:08.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.406+0000 7f494a492640 1 -- 172.21.15.69:0/3227572163 shutdown_connections 2024-04-25T11:24:08.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.406+0000 7f494a492640 1 --2- 172.21.15.69:0/3227572163 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49441080d0 0x7f494410a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:08.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.406+0000 7f494a492640 1 --2- 172.21.15.69:0/3227572163 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49441033d0 0x7f49441037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:08.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.406+0000 7f494a492640 1 --2- 172.21.15.69:0/3227572163 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4944102190 0x7f4944102e90 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:08.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.406+0000 7f494a492640 1 -- 172.21.15.69:0/3227572163 >> 172.21.15.69:0/3227572163 conn(0x7f49440fd380 msgr2=0x7f49440ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:08.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.406+0000 7f494a492640 1 -- 172.21.15.69:0/3227572163 shutdown_connections 2024-04-25T11:24:08.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.406+0000 7f494a492640 1 -- 172.21.15.69:0/3227572163 wait complete. 2024-04-25T11:24:08.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.407+0000 7f494a492640 1 Processor -- start 2024-04-25T11:24:08.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.407+0000 7f494a492640 1 -- start start 2024-04-25T11:24:08.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.407+0000 7f494a492640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4944102190 0x7f494419a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:08.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.407+0000 7f494a492640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f49441033d0 0x7f494419a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:08.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.407+0000 7f494a492640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49441080d0 0x7f49441a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:08.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.407+0000 7f494a492640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f494410d2d0 con 0x7f49441033d0 2024-04-25T11:24:08.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.407+0000 7f494a492640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f494410d150 con 0x7f49441080d0 2024-04-25T11:24:08.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.407+0000 7f494a492640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f494410d450 con 0x7f4944102190 2024-04-25T11:24:08.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.408+0000 7f49437fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f49441033d0 0x7f494419a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:08.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.408+0000 7f49437fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f49441033d0 0x7f494419a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45428/0 (socket says 172.21.15.69:45428) 2024-04-25T11:24:08.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.408+0000 7f49437fe640 1 -- 172.21.15.69:0/2381374713 learned_addr learned my addr 172.21.15.69:0/2381374713 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:24:08.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.408+0000 7f4948a2f640 1 --2- 172.21.15.69:0/2381374713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49441080d0 0x7f49441a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:08.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.408+0000 7f4943fff640 1 --2- 172.21.15.69:0/2381374713 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4944102190 0x7f494419a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:08.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.408+0000 7f49437fe640 1 -- 172.21.15.69:0/2381374713 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4944102190 msgr2=0x7f494419a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:08.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.408+0000 7f49437fe640 1 --2- 172.21.15.69:0/2381374713 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4944102190 0x7f494419a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:08.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.408+0000 7f49437fe640 1 -- 172.21.15.69:0/2381374713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49441080d0 msgr2=0x7f49441a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:08.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.408+0000 7f49437fe640 1 --2- 172.21.15.69:0/2381374713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49441080d0 0x7f49441a1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:08.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.408+0000 7f49437fe640 1 -- 172.21.15.69:0/2381374713 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49441a1ce0 con 0x7f49441033d0 2024-04-25T11:24:08.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.408+0000 7f4943fff640 1 --2- 172.21.15.69:0/2381374713 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4944102190 0x7f494419a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:24:08.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.408+0000 7f49437fe640 1 --2- 172.21.15.69:0/2381374713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f49441033d0 0x7f494419a6e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f493800d950 tx=0x7f493800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:08.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.408+0000 7f49417fa640 1 -- 172.21.15.69:0/2381374713 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4938017070 con 0x7f49441033d0 2024-04-25T11:24:08.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.408+0000 7f49417fa640 1 -- 172.21.15.69:0/2381374713 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4938009c30 con 0x7f49441033d0 2024-04-25T11:24:08.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.408+0000 7f494a492640 1 -- 172.21.15.69:0/2381374713 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49441a1fd0 con 0x7f49441033d0 2024-04-25T11:24:08.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.408+0000 7f49417fa640 1 -- 172.21.15.69:0/2381374713 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493800f620 con 0x7f49441033d0 2024-04-25T11:24:08.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.408+0000 7f494a492640 1 -- 172.21.15.69:0/2381374713 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49441a2430 con 0x7f49441033d0 2024-04-25T11:24:08.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.409+0000 7f494a492640 1 -- 172.21.15.69:0/2381374713 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49441a2690 con 0x7f49441033d0 2024-04-25T11:24:08.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.411+0000 7f49417fa640 1 -- 172.21.15.69:0/2381374713 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f4938005030 con 0x7f49441033d0 2024-04-25T11:24:08.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.411+0000 7f49417fa640 1 --2- 172.21.15.69:0/2381374713 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f492c0afaf0 0x7f492c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:08.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.412+0000 7f4943fff640 1 --2- 172.21.15.69:0/2381374713 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f492c0afaf0 0x7f492c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:08.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.412+0000 7f49417fa640 1 -- 172.21.15.69:0/2381374713 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7f49380dea50 con 0x7f49441033d0 2024-04-25T11:24:08.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.412+0000 7f4943fff640 1 --2- 172.21.15.69:0/2381374713 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f492c0afaf0 0x7f492c0b2000 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f49340057c0 tx=0x7f4934000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:08.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.415+0000 7f49417fa640 1 -- 172.21.15.69:0/2381374713 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f49380a73f0 con 0x7f49441033d0 2024-04-25T11:24:08.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.623+0000 7f494a492640 1 -- 172.21.15.69:0/2381374713 --> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f49441077e0 con 0x7f492c0afaf0 2024-04-25T11:24:08.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.652+0000 7f49417fa640 1 -- 172.21.15.69:0/2381374713 <== mgr.14152 v2:172.21.15.69:6800/4220065409 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f49441077e0 con 0x7f492c0afaf0 2024-04-25T11:24:08.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.654+0000 7f494a492640 1 -- 172.21.15.69:0/2381374713 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f492c0afaf0 msgr2=0x7f492c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:08.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.654+0000 7f494a492640 1 --2- 172.21.15.69:0/2381374713 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f492c0afaf0 0x7f492c0b2000 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f49340057c0 tx=0x7f4934000bc0 comp rx=0 tx=0).stop 2024-04-25T11:24:08.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.654+0000 7f494a492640 1 -- 172.21.15.69:0/2381374713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f49441033d0 msgr2=0x7f494419a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:08.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.654+0000 7f494a492640 1 --2- 172.21.15.69:0/2381374713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f49441033d0 0x7f494419a6e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f493800d950 tx=0x7f493800de10 comp rx=0 tx=0).stop 2024-04-25T11:24:08.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.654+0000 7f494a492640 1 -- 172.21.15.69:0/2381374713 shutdown_connections 2024-04-25T11:24:08.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.654+0000 7f494a492640 1 --2- 172.21.15.69:0/2381374713 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f492c0afaf0 0x7f492c0b2000 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:08.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.654+0000 7f494a492640 1 --2- 172.21.15.69:0/2381374713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49441080d0 0x7f49441a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:08.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.654+0000 7f494a492640 1 --2- 172.21.15.69:0/2381374713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f49441033d0 0x7f494419a6e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:08.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.654+0000 7f494a492640 1 --2- 172.21.15.69:0/2381374713 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4944102190 0x7f494419a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:08.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.654+0000 7f494a492640 1 -- 172.21.15.69:0/2381374713 >> 172.21.15.69:0/2381374713 conn(0x7f49440fd380 msgr2=0x7f49441060c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:08.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.654+0000 7f494a492640 1 -- 172.21.15.69:0/2381374713 shutdown_connections 2024-04-25T11:24:08.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.654+0000 7f494a492640 1 -- 172.21.15.69:0/2381374713 wait complete. 2024-04-25T11:24:08.658 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:24:08 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:24:08.256+0000 7f2672441700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T11:24:08.658 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:24:08 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:24:08.256+0000 7f2672441700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T11:24:08.658 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:24:08 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:24:08.256+0000 7f2672441700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T11:24:08.659 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:24:08 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:24:08.256+0000 7f2672441700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T11:24:08.659 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:24:08 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:24:08.256+0000 7f2672441700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T11:24:08.660 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-25T11:24:08.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.779+0000 7f2299420640 1 -- 172.21.15.69:0/4069371296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22941029e0 msgr2=0x7f22941036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:08.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.779+0000 7f2299420640 1 --2- 172.21.15.69:0/4069371296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22941029e0 0x7f22941036e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2284009ca0 tx=0x7f228402a8c0 comp rx=0 tx=0).stop 2024-04-25T11:24:08.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.779+0000 7f2299420640 1 -- 172.21.15.69:0/4069371296 shutdown_connections 2024-04-25T11:24:08.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.779+0000 7f2299420640 1 --2- 172.21.15.69:0/4069371296 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2294108920 0x7f229410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:08.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.779+0000 7f2299420640 1 --2- 172.21.15.69:0/4069371296 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2294103c20 0x7f2294104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:08.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.779+0000 7f2299420640 1 --2- 172.21.15.69:0/4069371296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22941029e0 0x7f22941036e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:08.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.779+0000 7f2299420640 1 -- 172.21.15.69:0/4069371296 >> 172.21.15.69:0/4069371296 conn(0x7f22940fdbd0 msgr2=0x7f2294100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:08.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.779+0000 7f2299420640 1 -- 172.21.15.69:0/4069371296 shutdown_connections 2024-04-25T11:24:08.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.779+0000 7f2299420640 1 -- 172.21.15.69:0/4069371296 wait complete. 2024-04-25T11:24:08.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.780+0000 7f2299420640 1 Processor -- start 2024-04-25T11:24:08.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.780+0000 7f2299420640 1 -- start start 2024-04-25T11:24:08.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.780+0000 7f2299420640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22941029e0 0x7f2294199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:08.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.780+0000 7f2299420640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2294103c20 0x7f2294199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:08.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.780+0000 7f2299420640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2294108920 0x7f22941a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:08.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.780+0000 7f2299420640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f229410e3e0 con 0x7f22941029e0 2024-04-25T11:24:08.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.780+0000 7f2299420640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f229410e260 con 0x7f2294103c20 2024-04-25T11:24:08.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.780+0000 7f2299420640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f229410e560 con 0x7f2294108920 2024-04-25T11:24:08.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.781+0000 7f2292ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22941029e0 0x7f2294199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:08.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.781+0000 7f2292ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22941029e0 0x7f2294199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45446/0 (socket says 172.21.15.69:45446) 2024-04-25T11:24:08.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.781+0000 7f2292ffd640 1 -- 172.21.15.69:0/3813924597 learned_addr learned my addr 172.21.15.69:0/3813924597 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:24:08.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.781+0000 7f22927fc640 1 --2- 172.21.15.69:0/3813924597 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2294103c20 0x7f2294199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:08.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.781+0000 7f22937fe640 1 --2- 172.21.15.69:0/3813924597 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2294108920 0x7f22941a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:08.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.781+0000 7f2292ffd640 1 -- 172.21.15.69:0/3813924597 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2294108920 msgr2=0x7f22941a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:08.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.781+0000 7f2292ffd640 1 --2- 172.21.15.69:0/3813924597 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2294108920 0x7f22941a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:08.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.781+0000 7f2292ffd640 1 -- 172.21.15.69:0/3813924597 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2294103c20 msgr2=0x7f2294199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:08.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.781+0000 7f2292ffd640 1 --2- 172.21.15.69:0/3813924597 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2294103c20 0x7f2294199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:08.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.781+0000 7f2292ffd640 1 -- 172.21.15.69:0/3813924597 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22941a0c40 con 0x7f22941029e0 2024-04-25T11:24:08.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.781+0000 7f22927fc640 1 --2- 172.21.15.69:0/3813924597 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2294103c20 0x7f2294199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:24:08.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.781+0000 7f22937fe640 1 --2- 172.21.15.69:0/3813924597 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2294108920 0x7f22941a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:24:08.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.781+0000 7f2292ffd640 1 --2- 172.21.15.69:0/3813924597 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22941029e0 0x7f2294199100 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2284000f80 tx=0x7f2284009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:08.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.781+0000 7f228bfff640 1 -- 172.21.15.69:0/3813924597 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2284003c70 con 0x7f22941029e0 2024-04-25T11:24:08.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.781+0000 7f228bfff640 1 -- 172.21.15.69:0/3813924597 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2284003e10 con 0x7f22941029e0 2024-04-25T11:24:08.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.781+0000 7f2299420640 1 -- 172.21.15.69:0/3813924597 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22941a0ed0 con 0x7f22941029e0 2024-04-25T11:24:08.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.781+0000 7f228bfff640 1 -- 172.21.15.69:0/3813924597 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2284030ba0 con 0x7f22941029e0 2024-04-25T11:24:08.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.781+0000 7f2299420640 1 -- 172.21.15.69:0/3813924597 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22941a1330 con 0x7f22941029e0 2024-04-25T11:24:08.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.782+0000 7f2299420640 1 -- 172.21.15.69:0/3813924597 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2294107730 con 0x7f22941029e0 2024-04-25T11:24:08.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.784+0000 7f228bfff640 1 -- 172.21.15.69:0/3813924597 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f2284004b40 con 0x7f22941029e0 2024-04-25T11:24:08.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.784+0000 7f228bfff640 1 --2- 172.21.15.69:0/3813924597 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f22800afaf0 0x7f22800b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:08.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.784+0000 7f22927fc640 1 --2- 172.21.15.69:0/3813924597 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f22800afaf0 0x7f22800b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:08.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.784+0000 7f228bfff640 1 -- 172.21.15.69:0/3813924597 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7f2284100320 con 0x7f22941029e0 2024-04-25T11:24:08.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.785+0000 7f22927fc640 1 --2- 172.21.15.69:0/3813924597 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f22800afaf0 0x7f22800b2000 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f229419a1a0 tx=0x7f227c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:08.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:08.788+0000 7f228bfff640 1 -- 172.21.15.69:0/3813924597 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f22840c8d50 con 0x7f22941029e0 2024-04-25T11:24:08.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:08 smithi069 ceph-mon[29188]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-04-25T11:24:08.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:08 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/973231051' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T11:24:08.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:08 smithi069 ceph-mon[29188]: mds.? [v2:172.21.15.152:6838/3008615657,v1:172.21.15.152:6839/3008615657] up:active 2024-04-25T11:24:08.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:08 smithi069 ceph-mon[29188]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-25T11:24:08.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:08 smithi069 ceph-mon[29188]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-04-25T11:24:08.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:08 smithi069 ceph-mon[29188]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:creating} 8 up:standby 2024-04-25T11:24:08.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:08 smithi069 ceph-mon[29188]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-04-25T11:24:08.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:08 smithi069 ceph-mon[29188]: mds.? [v2:172.21.15.146:6834/1876533699,v1:172.21.15.146:6835/1876533699] up:active 2024-04-25T11:24:08.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:08 smithi069 ceph-mon[29188]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-25T11:24:08.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:08 smithi069 ceph-mon[29188]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T11:24:08.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:08 smithi069 ceph-mon[29188]: Cluster is now healthy 2024-04-25T11:24:08.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:08 smithi069 ceph-mon[29188]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:active} 4 up:standby-replay 4 up:standby 2024-04-25T11:24:08.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:08 smithi069 ceph-mon[29188]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-04-25T11:24:08.944 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:08 smithi069 ceph-mon[29188]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-04-25T11:24:08.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.001+0000 7f2299420640 1 -- 172.21.15.69:0/3813924597 --> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f2294062fe0 con 0x7f22800afaf0 2024-04-25T11:24:08.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.006+0000 7f228bfff640 1 -- 172.21.15.69:0/3813924597 <== mgr.14152 v2:172.21.15.69:6800/4220065409 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f2294062fe0 con 0x7f22800afaf0 2024-04-25T11:24:09.000 INFO:teuthology.orchestra.run.smithi069.stdout:/volumes/qa/sv_0/71b2a7dc-2e75-4036-a954-acca3df9cf14 2024-04-25T11:24:09.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.008+0000 7f2299420640 1 -- 172.21.15.69:0/3813924597 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f22800afaf0 msgr2=0x7f22800b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:09.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.008+0000 7f2299420640 1 --2- 172.21.15.69:0/3813924597 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f22800afaf0 0x7f22800b2000 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f229419a1a0 tx=0x7f227c005f90 comp rx=0 tx=0).stop 2024-04-25T11:24:09.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.008+0000 7f2299420640 1 -- 172.21.15.69:0/3813924597 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22941029e0 msgr2=0x7f2294199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:09.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.008+0000 7f2299420640 1 --2- 172.21.15.69:0/3813924597 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22941029e0 0x7f2294199100 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2284000f80 tx=0x7f2284009360 comp rx=0 tx=0).stop 2024-04-25T11:24:09.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.008+0000 7f2299420640 1 -- 172.21.15.69:0/3813924597 shutdown_connections 2024-04-25T11:24:09.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.008+0000 7f2299420640 1 --2- 172.21.15.69:0/3813924597 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f22800afaf0 0x7f22800b2000 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:09.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.008+0000 7f2299420640 1 --2- 172.21.15.69:0/3813924597 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2294108920 0x7f22941a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:09.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.008+0000 7f2299420640 1 --2- 172.21.15.69:0/3813924597 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2294103c20 0x7f2294199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:09.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.008+0000 7f2299420640 1 --2- 172.21.15.69:0/3813924597 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22941029e0 0x7f2294199100 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:09.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.008+0000 7f2299420640 1 -- 172.21.15.69:0/3813924597 >> 172.21.15.69:0/3813924597 conn(0x7f22940fdbd0 msgr2=0x7f2294104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:09.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.008+0000 7f2299420640 1 -- 172.21.15.69:0/3813924597 shutdown_connections 2024-04-25T11:24:09.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.008+0000 7f2299420640 1 -- 172.21.15.69:0/3813924597 wait complete. 2024-04-25T11:24:09.013 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-25T11:24:09.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.138+0000 7f9da1da8640 1 -- 172.21.15.69:0/832955243 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d9c1098a0 msgr2=0x7f9d9c10bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:09.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.138+0000 7f9da1da8640 1 --2- 172.21.15.69:0/832955243 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d9c1098a0 0x7f9d9c10bcc0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9d94009ca0 tx=0x7f9d9402a8c0 comp rx=0 tx=0).stop 2024-04-25T11:24:09.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.138+0000 7f9da1da8640 1 -- 172.21.15.69:0/832955243 shutdown_connections 2024-04-25T11:24:09.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.138+0000 7f9da1da8640 1 --2- 172.21.15.69:0/832955243 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d9c1098a0 0x7f9d9c10bcc0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:09.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.138+0000 7f9da1da8640 1 --2- 172.21.15.69:0/832955243 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d9c106f40 0x7f9d9c109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:09.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.138+0000 7f9da1da8640 1 --2- 172.21.15.69:0/832955243 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d9c1045e0 0x7f9d9c106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:09.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.138+0000 7f9da1da8640 1 -- 172.21.15.69:0/832955243 >> 172.21.15.69:0/832955243 conn(0x7f9d9c0fdbc0 msgr2=0x7f9d9c100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:09.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.138+0000 7f9da1da8640 1 -- 172.21.15.69:0/832955243 shutdown_connections 2024-04-25T11:24:09.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.138+0000 7f9da1da8640 1 -- 172.21.15.69:0/832955243 wait complete. 2024-04-25T11:24:09.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.139+0000 7f9da1da8640 1 Processor -- start 2024-04-25T11:24:09.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.139+0000 7f9da1da8640 1 -- start start 2024-04-25T11:24:09.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.139+0000 7f9da1da8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d9c1045e0 0x7f9d9c19a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:09.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.139+0000 7f9da1da8640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d9c106f40 0x7f9d9c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:09.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.139+0000 7f9da1da8640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d9c1098a0 0x7f9d9c1a1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:09.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.139+0000 7f9da1da8640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c10f520 con 0x7f9d9c1098a0 2024-04-25T11:24:09.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.139+0000 7f9da1da8640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c10f3a0 con 0x7f9d9c1045e0 2024-04-25T11:24:09.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.139+0000 7f9da1da8640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c10f6a0 con 0x7f9d9c106f40 2024-04-25T11:24:09.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.140+0000 7f9d9bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d9c1098a0 0x7f9d9c1a1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:09.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.140+0000 7f9d9b7fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d9c1045e0 0x7f9d9c19a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:09.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.140+0000 7f9d9bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d9c1098a0 0x7f9d9c1a1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45464/0 (socket says 172.21.15.69:45464) 2024-04-25T11:24:09.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.140+0000 7f9d9bfff640 1 -- 172.21.15.69:0/488392033 learned_addr learned my addr 172.21.15.69:0/488392033 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:24:09.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.140+0000 7f9d9b7fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d9c1045e0 0x7f9d9c19a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.69:45662/0 (socket says 172.21.15.69:45662) 2024-04-25T11:24:09.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.140+0000 7f9d9affd640 1 --2- 172.21.15.69:0/488392033 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d9c106f40 0x7f9d9c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:09.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.140+0000 7f9d9bfff640 1 -- 172.21.15.69:0/488392033 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d9c106f40 msgr2=0x7f9d9c19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:09.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.140+0000 7f9d9bfff640 1 --2- 172.21.15.69:0/488392033 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d9c106f40 0x7f9d9c19a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:09.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.140+0000 7f9d9bfff640 1 -- 172.21.15.69:0/488392033 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d9c1045e0 msgr2=0x7f9d9c19a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:09.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.140+0000 7f9d9bfff640 1 --2- 172.21.15.69:0/488392033 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d9c1045e0 0x7f9d9c19a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:09.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.140+0000 7f9d9bfff640 1 -- 172.21.15.69:0/488392033 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d9c1a1c70 con 0x7f9d9c1098a0 2024-04-25T11:24:09.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.140+0000 7f9d9affd640 1 --2- 172.21.15.69:0/488392033 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d9c106f40 0x7f9d9c19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:24:09.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.140+0000 7f9d9b7fe640 1 --2- 172.21.15.69:0/488392033 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d9c1045e0 0x7f9d9c19a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:24:09.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.140+0000 7f9d9bfff640 1 --2- 172.21.15.69:0/488392033 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d9c1098a0 0x7f9d9c1a1600 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9d9402aeb0 tx=0x7f9d940093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:09.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.140+0000 7f9d98ff9640 1 -- 172.21.15.69:0/488392033 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d94003db0 con 0x7f9d9c1098a0 2024-04-25T11:24:09.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.140+0000 7f9d98ff9640 1 -- 172.21.15.69:0/488392033 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d940044d0 con 0x7f9d9c1098a0 2024-04-25T11:24:09.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.140+0000 7f9da1da8640 1 -- 172.21.15.69:0/488392033 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d9c1a1f00 con 0x7f9d9c1098a0 2024-04-25T11:24:09.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.140+0000 7f9d98ff9640 1 -- 172.21.15.69:0/488392033 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d94030ca0 con 0x7f9d9c1098a0 2024-04-25T11:24:09.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.140+0000 7f9da1da8640 1 -- 172.21.15.69:0/488392033 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d9c1a2360 con 0x7f9d9c1098a0 2024-04-25T11:24:09.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.141+0000 7f9da1da8640 1 -- 172.21.15.69:0/488392033 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d9c06a160 con 0x7f9d9c1098a0 2024-04-25T11:24:09.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.142+0000 7f9d98ff9640 1 -- 172.21.15.69:0/488392033 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7f9d94004040 con 0x7f9d9c1098a0 2024-04-25T11:24:09.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.143+0000 7f9d98ff9640 1 --2- 172.21.15.69:0/488392033 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f9d840afaf0 0x7f9d840b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:09.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.143+0000 7f9d9b7fe640 1 --2- 172.21.15.69:0/488392033 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f9d840afaf0 0x7f9d840b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:09.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.143+0000 7f9d98ff9640 1 -- 172.21.15.69:0/488392033 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7f9d941004e0 con 0x7f9d9c1098a0 2024-04-25T11:24:09.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.144+0000 7f9d9b7fe640 1 --2- 172.21.15.69:0/488392033 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f9d840afaf0 0x7f9d840b2000 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c0757f0 tx=0x7f9d8c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:09.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.147+0000 7f9d98ff9640 1 -- 172.21.15.69:0/488392033 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9d940c8e80 con 0x7f9d9c1098a0 2024-04-25T11:24:09.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:08 smithi152 ceph-mon[32701]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-04-25T11:24:09.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:08 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/973231051' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T11:24:09.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:08 smithi152 ceph-mon[32701]: mds.? [v2:172.21.15.152:6838/3008615657,v1:172.21.15.152:6839/3008615657] up:active 2024-04-25T11:24:09.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:08 smithi152 ceph-mon[32701]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-25T11:24:09.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:08 smithi152 ceph-mon[32701]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-04-25T11:24:09.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:08 smithi152 ceph-mon[32701]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:creating} 8 up:standby 2024-04-25T11:24:09.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:08 smithi152 ceph-mon[32701]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-04-25T11:24:09.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:08 smithi152 ceph-mon[32701]: mds.? [v2:172.21.15.146:6834/1876533699,v1:172.21.15.146:6835/1876533699] up:active 2024-04-25T11:24:09.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:08 smithi152 ceph-mon[32701]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-25T11:24:09.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:08 smithi152 ceph-mon[32701]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T11:24:09.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:08 smithi152 ceph-mon[32701]: Cluster is now healthy 2024-04-25T11:24:09.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:08 smithi152 ceph-mon[32701]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:active} 4 up:standby-replay 4 up:standby 2024-04-25T11:24:09.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:08 smithi152 ceph-mon[32701]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-04-25T11:24:09.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:08 smithi152 ceph-mon[32701]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-04-25T11:24:09.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.355+0000 7f9da1da8640 1 -- 172.21.15.69:0/488392033 --> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f9d9c062fe0 con 0x7f9d840afaf0 2024-04-25T11:24:09.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:08 smithi146 ceph-mon[32821]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-04-25T11:24:09.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:08 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/973231051' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T11:24:09.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:08 smithi146 ceph-mon[32821]: mds.? [v2:172.21.15.152:6838/3008615657,v1:172.21.15.152:6839/3008615657] up:active 2024-04-25T11:24:09.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:08 smithi146 ceph-mon[32821]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-25T11:24:09.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:08 smithi146 ceph-mon[32821]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-04-25T11:24:09.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:08 smithi146 ceph-mon[32821]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:creating} 8 up:standby 2024-04-25T11:24:09.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:08 smithi146 ceph-mon[32821]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-04-25T11:24:09.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:08 smithi146 ceph-mon[32821]: mds.? [v2:172.21.15.146:6834/1876533699,v1:172.21.15.146:6835/1876533699] up:active 2024-04-25T11:24:09.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:08 smithi146 ceph-mon[32821]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-25T11:24:09.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:08 smithi146 ceph-mon[32821]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T11:24:09.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:08 smithi146 ceph-mon[32821]: Cluster is now healthy 2024-04-25T11:24:09.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:08 smithi146 ceph-mon[32821]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:active} 4 up:standby-replay 4 up:standby 2024-04-25T11:24:09.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:08 smithi146 ceph-mon[32821]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-04-25T11:24:09.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:08 smithi146 ceph-mon[32821]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-04-25T11:24:09.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.380+0000 7f9d98ff9640 1 -- 172.21.15.69:0/488392033 <== mgr.14152 v2:172.21.15.69:6800/4220065409 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9d9c062fe0 con 0x7f9d840afaf0 2024-04-25T11:24:09.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.382+0000 7f9da1da8640 1 -- 172.21.15.69:0/488392033 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f9d840afaf0 msgr2=0x7f9d840b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:09.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.382+0000 7f9da1da8640 1 --2- 172.21.15.69:0/488392033 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f9d840afaf0 0x7f9d840b2000 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c0757f0 tx=0x7f9d8c005f90 comp rx=0 tx=0).stop 2024-04-25T11:24:09.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.382+0000 7f9da1da8640 1 -- 172.21.15.69:0/488392033 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d9c1098a0 msgr2=0x7f9d9c1a1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:09.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.382+0000 7f9da1da8640 1 --2- 172.21.15.69:0/488392033 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d9c1098a0 0x7f9d9c1a1600 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9d9402aeb0 tx=0x7f9d940093e0 comp rx=0 tx=0).stop 2024-04-25T11:24:09.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.382+0000 7f9da1da8640 1 -- 172.21.15.69:0/488392033 shutdown_connections 2024-04-25T11:24:09.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.382+0000 7f9da1da8640 1 --2- 172.21.15.69:0/488392033 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f9d840afaf0 0x7f9d840b2000 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:09.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.382+0000 7f9da1da8640 1 --2- 172.21.15.69:0/488392033 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d9c1098a0 0x7f9d9c1a1600 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:09.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.382+0000 7f9da1da8640 1 --2- 172.21.15.69:0/488392033 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d9c106f40 0x7f9d9c19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:09.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.382+0000 7f9da1da8640 1 --2- 172.21.15.69:0/488392033 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d9c1045e0 0x7f9d9c19a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:09.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.382+0000 7f9da1da8640 1 -- 172.21.15.69:0/488392033 >> 172.21.15.69:0/488392033 conn(0x7f9d9c0fdbc0 msgr2=0x7f9d9c1051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:09.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.383+0000 7f9da1da8640 1 -- 172.21.15.69:0/488392033 shutdown_connections 2024-04-25T11:24:09.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.383+0000 7f9da1da8640 1 -- 172.21.15.69:0/488392033 wait complete. 2024-04-25T11:24:09.388 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-25T11:24:09.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.506+0000 7ff42689f640 1 -- 172.21.15.69:0/199446635 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff420106c20 msgr2=0x7ff420109100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:09.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.506+0000 7ff42689f640 1 --2- 172.21.15.69:0/199446635 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff420106c20 0x7ff420109100 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff410009900 tx=0x7ff41002a410 comp rx=0 tx=0).stop 2024-04-25T11:24:09.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.506+0000 7ff42689f640 1 -- 172.21.15.69:0/199446635 shutdown_connections 2024-04-25T11:24:09.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.506+0000 7ff42689f640 1 --2- 172.21.15.69:0/199446635 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff420109790 0x7ff42010bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:09.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.506+0000 7ff42689f640 1 --2- 172.21.15.69:0/199446635 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff420106c20 0x7ff420109100 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:09.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.506+0000 7ff42689f640 1 --2- 172.21.15.69:0/199446635 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff420101e40 0x7ff420102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:09.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.506+0000 7ff42689f640 1 -- 172.21.15.69:0/199446635 >> 172.21.15.69:0/199446635 conn(0x7ff4200fd350 msgr2=0x7ff4200ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:09.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.506+0000 7ff42689f640 1 -- 172.21.15.69:0/199446635 shutdown_connections 2024-04-25T11:24:09.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.506+0000 7ff42689f640 1 -- 172.21.15.69:0/199446635 wait complete. 2024-04-25T11:24:09.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.507+0000 7ff42689f640 1 Processor -- start 2024-04-25T11:24:09.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.507+0000 7ff42689f640 1 -- start start 2024-04-25T11:24:09.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.507+0000 7ff42689f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff420101e40 0x7ff42019a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:09.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.507+0000 7ff42689f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff420106c20 0x7ff42019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:09.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.507+0000 7ff42689f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff420109790 0x7ff4201a1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:09.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.507+0000 7ff42689f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff42010f4d0 con 0x7ff420106c20 2024-04-25T11:24:09.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.507+0000 7ff42689f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff42010f350 con 0x7ff420101e40 2024-04-25T11:24:09.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.507+0000 7ff42689f640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff42010f650 con 0x7ff420109790 2024-04-25T11:24:09.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.507+0000 7ff41f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff420106c20 0x7ff42019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:09.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.507+0000 7ff41f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff420106c20 0x7ff42019a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45494/0 (socket says 172.21.15.69:45494) 2024-04-25T11:24:09.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.507+0000 7ff41f7fe640 1 -- 172.21.15.69:0/2457333053 learned_addr learned my addr 172.21.15.69:0/2457333053 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:24:09.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.507+0000 7ff41ffff640 1 --2- 172.21.15.69:0/2457333053 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff420101e40 0x7ff42019a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:09.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.507+0000 7ff41f7fe640 1 -- 172.21.15.69:0/2457333053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff420109790 msgr2=0x7ff4201a1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:09.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.507+0000 7ff41f7fe640 1 --2- 172.21.15.69:0/2457333053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff420109790 0x7ff4201a1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:09.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.507+0000 7ff41f7fe640 1 -- 172.21.15.69:0/2457333053 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff420101e40 msgr2=0x7ff42019a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:09.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.507+0000 7ff41f7fe640 1 --2- 172.21.15.69:0/2457333053 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff420101e40 0x7ff42019a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:09.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.507+0000 7ff41f7fe640 1 -- 172.21.15.69:0/2457333053 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4201a1c70 con 0x7ff420106c20 2024-04-25T11:24:09.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.508+0000 7ff41f7fe640 1 --2- 172.21.15.69:0/2457333053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff420106c20 0x7ff42019a670 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff410009230 tx=0x7ff4100040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:09.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.508+0000 7ff41d7fa640 1 -- 172.21.15.69:0/2457333053 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41003f030 con 0x7ff420106c20 2024-04-25T11:24:09.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.508+0000 7ff41d7fa640 1 -- 172.21.15.69:0/2457333053 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff4100044c0 con 0x7ff420106c20 2024-04-25T11:24:09.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.508+0000 7ff42689f640 1 -- 172.21.15.69:0/2457333053 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4201a1f00 con 0x7ff420106c20 2024-04-25T11:24:09.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.508+0000 7ff41d7fa640 1 -- 172.21.15.69:0/2457333053 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff410035070 con 0x7ff420106c20 2024-04-25T11:24:09.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.508+0000 7ff42689f640 1 -- 172.21.15.69:0/2457333053 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4201a2370 con 0x7ff420106c20 2024-04-25T11:24:09.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.509+0000 7ff42689f640 1 -- 172.21.15.69:0/2457333053 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4200fee70 con 0x7ff420106c20 2024-04-25T11:24:09.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.510+0000 7ff41d7fa640 1 -- 172.21.15.69:0/2457333053 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7ff410042030 con 0x7ff420106c20 2024-04-25T11:24:09.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.511+0000 7ff41d7fa640 1 --2- 172.21.15.69:0/2457333053 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ff4080afb70 0x7ff4080b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:09.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.511+0000 7ff41ffff640 1 --2- 172.21.15.69:0/2457333053 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ff4080afb70 0x7ff4080b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:09.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.511+0000 7ff41d7fa640 1 -- 172.21.15.69:0/2457333053 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7ff4100ff320 con 0x7ff420106c20 2024-04-25T11:24:09.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.511+0000 7ff41ffff640 1 --2- 172.21.15.69:0/2457333053 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ff4080afb70 0x7ff4080b2080 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff414005fd0 tx=0x7ff41400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:09.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.515+0000 7ff41d7fa640 1 -- 172.21.15.69:0/2457333053 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff4100c7cc0 con 0x7ff420106c20 2024-04-25T11:24:09.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.727+0000 7ff42689f640 1 -- 172.21.15.69:0/2457333053 --> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7ff420063070 con 0x7ff4080afb70 2024-04-25T11:24:09.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.731+0000 7ff41d7fa640 1 -- 172.21.15.69:0/2457333053 <== mgr.14152 v2:172.21.15.69:6800/4220065409 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7ff420063070 con 0x7ff4080afb70 2024-04-25T11:24:09.725 INFO:teuthology.orchestra.run.smithi069.stdout:/volumes/qa/sv_1/c99b04d9-8ad7-4ad0-a757-e72035b81fcc 2024-04-25T11:24:09.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.733+0000 7ff42689f640 1 -- 172.21.15.69:0/2457333053 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ff4080afb70 msgr2=0x7ff4080b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:09.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.733+0000 7ff42689f640 1 --2- 172.21.15.69:0/2457333053 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ff4080afb70 0x7ff4080b2080 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff414005fd0 tx=0x7ff41400b390 comp rx=0 tx=0).stop 2024-04-25T11:24:09.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.733+0000 7ff42689f640 1 -- 172.21.15.69:0/2457333053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff420106c20 msgr2=0x7ff42019a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:09.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.733+0000 7ff42689f640 1 --2- 172.21.15.69:0/2457333053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff420106c20 0x7ff42019a670 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff410009230 tx=0x7ff4100040c0 comp rx=0 tx=0).stop 2024-04-25T11:24:09.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.734+0000 7ff42689f640 1 -- 172.21.15.69:0/2457333053 shutdown_connections 2024-04-25T11:24:09.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.734+0000 7ff42689f640 1 --2- 172.21.15.69:0/2457333053 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ff4080afb70 0x7ff4080b2080 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:09.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.734+0000 7ff42689f640 1 --2- 172.21.15.69:0/2457333053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff420109790 0x7ff4201a1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:09.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.734+0000 7ff42689f640 1 --2- 172.21.15.69:0/2457333053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff420106c20 0x7ff42019a670 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:09.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.734+0000 7ff42689f640 1 --2- 172.21.15.69:0/2457333053 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff420101e40 0x7ff42019a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:09.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.734+0000 7ff42689f640 1 -- 172.21.15.69:0/2457333053 >> 172.21.15.69:0/2457333053 conn(0x7ff4200fd350 msgr2=0x7ff420107d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:09.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.734+0000 7ff42689f640 1 -- 172.21.15.69:0/2457333053 shutdown_connections 2024-04-25T11:24:09.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.734+0000 7ff42689f640 1 -- 172.21.15.69:0/2457333053 wait complete. 2024-04-25T11:24:09.739 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:24:09.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.852+0000 7fc6463d5640 1 -- 172.21.15.69:0/510150910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc640102fd0 msgr2=0x7fc6401033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:09.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.852+0000 7fc6463d5640 1 --2- 172.21.15.69:0/510150910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc640102fd0 0x7fc6401033e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc630009900 tx=0x7fc63002a410 comp rx=0 tx=0).stop 2024-04-25T11:24:09.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.852+0000 7fc6463d5640 1 -- 172.21.15.69:0/510150910 shutdown_connections 2024-04-25T11:24:09.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.852+0000 7fc6463d5640 1 --2- 172.21.15.69:0/510150910 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6401044d0 0x7fc64010ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:09.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.852+0000 7fc6463d5640 1 --2- 172.21.15.69:0/510150910 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6401039b0 0x7fc640103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:09.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.852+0000 7fc6463d5640 1 --2- 172.21.15.69:0/510150910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc640102fd0 0x7fc6401033e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:09.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.852+0000 7fc6463d5640 1 -- 172.21.15.69:0/510150910 >> 172.21.15.69:0/510150910 conn(0x7fc6400fd3b0 msgr2=0x7fc6400ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:09.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.852+0000 7fc6463d5640 1 -- 172.21.15.69:0/510150910 shutdown_connections 2024-04-25T11:24:09.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.852+0000 7fc6463d5640 1 -- 172.21.15.69:0/510150910 wait complete. 2024-04-25T11:24:09.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.853+0000 7fc6463d5640 1 Processor -- start 2024-04-25T11:24:09.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.853+0000 7fc6463d5640 1 -- start start 2024-04-25T11:24:09.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.853+0000 7fc6463d5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc640102fd0 0x7fc640102270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:09.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.853+0000 7fc6463d5640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6401039b0 0x7fc6401a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:09.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.853+0000 7fc6463d5640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6401044d0 0x7fc6401a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:09.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.853+0000 7fc6463d5640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc64010e410 con 0x7fc640102fd0 2024-04-25T11:24:09.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.853+0000 7fc6463d5640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc64010e290 con 0x7fc6401044d0 2024-04-25T11:24:09.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.853+0000 7fc6463d5640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc64010e590 con 0x7fc6401039b0 2024-04-25T11:24:09.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.853+0000 7fc63ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc640102fd0 0x7fc640102270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:09.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.854+0000 7fc63ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc640102fd0 0x7fc640102270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45516/0 (socket says 172.21.15.69:45516) 2024-04-25T11:24:09.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.854+0000 7fc63ffff640 1 -- 172.21.15.69:0/2448485462 learned_addr learned my addr 172.21.15.69:0/2448485462 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:24:09.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.854+0000 7fc644972640 1 --2- 172.21.15.69:0/2448485462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6401044d0 0x7fc6401a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:09.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.854+0000 7fc63f7fe640 1 --2- 172.21.15.69:0/2448485462 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6401039b0 0x7fc6401a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:09.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.854+0000 7fc63ffff640 1 -- 172.21.15.69:0/2448485462 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6401039b0 msgr2=0x7fc6401a1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:09.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.854+0000 7fc63ffff640 1 --2- 172.21.15.69:0/2448485462 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6401039b0 0x7fc6401a1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:09.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.854+0000 7fc63ffff640 1 -- 172.21.15.69:0/2448485462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6401044d0 msgr2=0x7fc6401a3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:09.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.854+0000 7fc63ffff640 1 --2- 172.21.15.69:0/2448485462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6401044d0 0x7fc6401a3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:09.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.854+0000 7fc63ffff640 1 -- 172.21.15.69:0/2448485462 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6401a3e90 con 0x7fc640102fd0 2024-04-25T11:24:09.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.854+0000 7fc63ffff640 1 --2- 172.21.15.69:0/2448485462 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc640102fd0 0x7fc640102270 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc630031460 tx=0x7fc630004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:09.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.854+0000 7fc63d7fa640 1 -- 172.21.15.69:0/2448485462 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63003f030 con 0x7fc640102fd0 2024-04-25T11:24:09.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.854+0000 7fc6463d5640 1 -- 172.21.15.69:0/2448485462 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6401a4120 con 0x7fc640102fd0 2024-04-25T11:24:09.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.854+0000 7fc63d7fa640 1 -- 172.21.15.69:0/2448485462 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc630003a80 con 0x7fc640102fd0 2024-04-25T11:24:09.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.854+0000 7fc63d7fa640 1 -- 172.21.15.69:0/2448485462 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc630035070 con 0x7fc640102fd0 2024-04-25T11:24:09.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.854+0000 7fc6463d5640 1 -- 172.21.15.69:0/2448485462 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6401a4660 con 0x7fc640102fd0 2024-04-25T11:24:09.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.855+0000 7fc6463d5640 1 -- 172.21.15.69:0/2448485462 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc640107970 con 0x7fc640102fd0 2024-04-25T11:24:09.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.857+0000 7fc63d7fa640 1 -- 172.21.15.69:0/2448485462 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 147520+0+0 (secure 0 0 0) 0x7fc630042030 con 0x7fc640102fd0 2024-04-25T11:24:09.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.858+0000 7fc63d7fa640 1 --2- 172.21.15.69:0/2448485462 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc6280afaf0 0x7fc6280b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:09.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.858+0000 7fc63d7fa640 1 -- 172.21.15.69:0/2448485462 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7fc6300ff980 con 0x7fc640102fd0 2024-04-25T11:24:09.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.858+0000 7fc63f7fe640 1 --2- 172.21.15.69:0/2448485462 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc6280afaf0 0x7fc6280b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:09.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.858+0000 7fc63f7fe640 1 --2- 172.21.15.69:0/2448485462 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc6280afaf0 0x7fc6280b2000 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc640101c10 tx=0x7fc634009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:09.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:09.861+0000 7fc63d7fa640 1 -- 172.21.15.69:0/2448485462 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc6300c8320 con 0x7fc640102fd0 2024-04-25T11:24:10.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.079+0000 7fc63d7fa640 1 -- 172.21.15.69:0/2448485462 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fc6300c3eb0 con 0x7fc640102fd0 2024-04-25T11:24:10.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.086+0000 7fc6463d5640 1 -- 172.21.15.69:0/2448485462 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc64002db50 con 0x7fc640102fd0 2024-04-25T11:24:10.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.087+0000 7fc63d7fa640 1 -- 172.21.15.69:0/2448485462 <== mon.0 v2:172.21.15.69:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11944 (secure 0 0 0) 0x7fc63004e430 con 0x7fc640102fd0 2024-04-25T11:24:10.081 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:24:10.081 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3399424094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3399424094},{"type":"v1","addr":"172.21.15.69:6835","nonce":3399424094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1463217771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1463217771},{"type":"v1","addr":"172.21.15.69:6837","nonce":1463217771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T11:23:40.984930+0000","modified":"2024-04-25T11:24:09.061188+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24485,"mds_2":24475,"mds_3":24458,"mds_4":14625},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6839/966316001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":966316001},{"type":"v1","addr":"172.21.15.69:6839","nonce":966316001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.69:6841/2060225181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":2060225181},{"type":"v1","addr":"172.21.15.69:6841","nonce":2060225181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6833/4197482473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":4197482473},{"type":"v1","addr":"172.21.15.152:6833","nonce":4197482473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/4267885323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4267885323},{"type":"v1","addr":"172.21.15.146:6833","nonce":4267885323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6835/3188843362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3188843362},{"type":"v1","addr":"172.21.15.152:6835","nonce":3188843362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6837/563274982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":563274982},{"type":"v1","addr":"172.21.15.152:6837","nonce":563274982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.146:6835/1876533699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1876533699},{"type":"v1","addr":"172.21.15.146:6835","nonce":1876533699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.152:6839/3008615657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":3008615657},{"type":"v1","addr":"172.21.15.152:6839","nonce":3008615657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.146:6837/2730317134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2730317134},{"type":"v1","addr":"172.21.15.146:6837","nonce":2730317134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6839/2013283448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":2013283448},{"type":"v1","addr":"172.21.15.146:6839","nonce":2013283448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24485,24515,24475,24458,14625]},"id":1}]} 2024-04-25T11:24:10.081 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 28 2024-04-25T11:24:10.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.089+0000 7fc6463d5640 1 -- 172.21.15.69:0/2448485462 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc6280afaf0 msgr2=0x7fc6280b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:10.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.089+0000 7fc6463d5640 1 --2- 172.21.15.69:0/2448485462 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc6280afaf0 0x7fc6280b2000 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc640101c10 tx=0x7fc634009110 comp rx=0 tx=0).stop 2024-04-25T11:24:10.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.089+0000 7fc6463d5640 1 -- 172.21.15.69:0/2448485462 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc640102fd0 msgr2=0x7fc640102270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:10.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.089+0000 7fc6463d5640 1 --2- 172.21.15.69:0/2448485462 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc640102fd0 0x7fc640102270 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc630031460 tx=0x7fc630004270 comp rx=0 tx=0).stop 2024-04-25T11:24:10.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.089+0000 7fc6463d5640 1 -- 172.21.15.69:0/2448485462 shutdown_connections 2024-04-25T11:24:10.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.089+0000 7fc6463d5640 1 --2- 172.21.15.69:0/2448485462 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc6280afaf0 0x7fc6280b2000 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:10.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.089+0000 7fc6463d5640 1 --2- 172.21.15.69:0/2448485462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6401044d0 0x7fc6401a3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:10.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.089+0000 7fc6463d5640 1 --2- 172.21.15.69:0/2448485462 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6401039b0 0x7fc6401a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:10.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.089+0000 7fc6463d5640 1 --2- 172.21.15.69:0/2448485462 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc640102fd0 0x7fc640102270 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:10.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.089+0000 7fc6463d5640 1 -- 172.21.15.69:0/2448485462 >> 172.21.15.69:0/2448485462 conn(0x7fc6400fd3b0 msgr2=0x7fc640104b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:10.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.089+0000 7fc6463d5640 1 -- 172.21.15.69:0/2448485462 shutdown_connections 2024-04-25T11:24:10.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.089+0000 7fc6463d5640 1 -- 172.21.15.69:0/2448485462 wait complete. 2024-04-25T11:24:10.093 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:09 smithi069 ceph-mon[29188]: from='client.14883 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:24:10.093 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:09 smithi069 ceph-mon[29188]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:24:10.094 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:09 smithi069 ceph-mon[29188]: mds.? [v2:172.21.15.69:6840/2060225181,v1:172.21.15.69:6841/2060225181] up:active 2024-04-25T11:24:10.094 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:09 smithi069 ceph-mon[29188]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-25T11:24:10.094 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:09 smithi069 ceph-mon[29188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T11:24:10.095 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:24:10.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.209+0000 7f898828c640 1 -- 172.21.15.69:0/2501307118 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8980109a40 msgr2=0x7f898010be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:10.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.209+0000 7f898828c640 1 --2- 172.21.15.69:0/2501307118 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8980109a40 0x7f898010be60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8978009ca0 tx=0x7f897802a8c0 comp rx=0 tx=0).stop 2024-04-25T11:24:10.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.209+0000 7f898828c640 1 -- 172.21.15.69:0/2501307118 shutdown_connections 2024-04-25T11:24:10.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.209+0000 7f898828c640 1 --2- 172.21.15.69:0/2501307118 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8980109a40 0x7f898010be60 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:10.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.209+0000 7f898828c640 1 --2- 172.21.15.69:0/2501307118 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8980104d40 0x7f8980105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:10.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.209+0000 7f898828c640 1 --2- 172.21.15.69:0/2501307118 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8980103b00 0x7f8980104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:10.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.209+0000 7f898828c640 1 -- 172.21.15.69:0/2501307118 >> 172.21.15.69:0/2501307118 conn(0x7f89800fed30 msgr2=0x7f89801011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:10.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.209+0000 7f898828c640 1 -- 172.21.15.69:0/2501307118 shutdown_connections 2024-04-25T11:24:10.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.209+0000 7f898828c640 1 -- 172.21.15.69:0/2501307118 wait complete. 2024-04-25T11:24:10.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.209+0000 7f898828c640 1 Processor -- start 2024-04-25T11:24:10.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.209+0000 7f898828c640 1 -- start start 2024-04-25T11:24:10.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.210+0000 7f898828c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8980103b00 0x7f898019a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:10.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.210+0000 7f898828c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8980104d40 0x7f898019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:10.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.210+0000 7f898828c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8980109a40 0x7f89801a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:10.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.210+0000 7f898828c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f898010f530 con 0x7f8980103b00 2024-04-25T11:24:10.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.210+0000 7f898828c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f898010f3b0 con 0x7f8980109a40 2024-04-25T11:24:10.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.210+0000 7f898828c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f898010f6b0 con 0x7f8980104d40 2024-04-25T11:24:10.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.210+0000 7f8986028640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8980103b00 0x7f898019a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:10.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.210+0000 7f8986028640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8980103b00 0x7f898019a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45536/0 (socket says 172.21.15.69:45536) 2024-04-25T11:24:10.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.210+0000 7f8986028640 1 -- 172.21.15.69:0/1155999108 learned_addr learned my addr 172.21.15.69:0/1155999108 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:24:10.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.210+0000 7f8985827640 1 --2- 172.21.15.69:0/1155999108 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8980104d40 0x7f898019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:10.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.210+0000 7f8986028640 1 -- 172.21.15.69:0/1155999108 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8980104d40 msgr2=0x7f898019a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:10.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.210+0000 7f8986829640 1 --2- 172.21.15.69:0/1155999108 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8980109a40 0x7f89801a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:10.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.210+0000 7f8986028640 1 --2- 172.21.15.69:0/1155999108 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8980104d40 0x7f898019a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:10.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.210+0000 7f8986028640 1 -- 172.21.15.69:0/1155999108 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8980109a40 msgr2=0x7f89801a15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:10.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.210+0000 7f8986028640 1 --2- 172.21.15.69:0/1155999108 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8980109a40 0x7f89801a15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:10.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.210+0000 7f8986028640 1 -- 172.21.15.69:0/1155999108 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89801a1c10 con 0x7f8980103b00 2024-04-25T11:24:10.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.210+0000 7f8986028640 1 --2- 172.21.15.69:0/1155999108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8980103b00 0x7f898019a0d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f897c00e9f0 tx=0x7f897c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:10.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.210+0000 7f89737fe640 1 -- 172.21.15.69:0/1155999108 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c00ccc0 con 0x7f8980103b00 2024-04-25T11:24:10.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.210+0000 7f89737fe640 1 -- 172.21.15.69:0/1155999108 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f897c0035f0 con 0x7f8980103b00 2024-04-25T11:24:10.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.210+0000 7f898828c640 1 -- 172.21.15.69:0/1155999108 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89801a1f00 con 0x7f8980103b00 2024-04-25T11:24:10.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.210+0000 7f89737fe640 1 -- 172.21.15.69:0/1155999108 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c00ce60 con 0x7f8980103b00 2024-04-25T11:24:10.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.210+0000 7f898828c640 1 -- 172.21.15.69:0/1155999108 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89801a2360 con 0x7f8980103b00 2024-04-25T11:24:10.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.211+0000 7f898828c640 1 -- 172.21.15.69:0/1155999108 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8980108850 con 0x7f8980103b00 2024-04-25T11:24:10.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.213+0000 7f89737fe640 1 -- 172.21.15.69:0/1155999108 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f897c004000 con 0x7f8980103b00 2024-04-25T11:24:10.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.213+0000 7f89737fe640 1 --2- 172.21.15.69:0/1155999108 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f896c0afb80 0x7f896c0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:10.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.213+0000 7f8985827640 1 --2- 172.21.15.69:0/1155999108 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f896c0afb80 0x7f896c0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:10.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.213+0000 7f89737fe640 1 -- 172.21.15.69:0/1155999108 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7f897c0df4e0 con 0x7f8980103b00 2024-04-25T11:24:10.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.214+0000 7f8985827640 1 --2- 172.21.15.69:0/1155999108 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f896c0afb80 0x7f896c0b2090 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f898019b170 tx=0x7f897400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:10.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.217+0000 7f89737fe640 1 -- 172.21.15.69:0/1155999108 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f897c0a7d20 con 0x7f8980103b00 2024-04-25T11:24:10.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:09 smithi152 ceph-mon[32701]: from='client.14883 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:24:10.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:09 smithi152 ceph-mon[32701]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:24:10.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:09 smithi152 ceph-mon[32701]: mds.? [v2:172.21.15.69:6840/2060225181,v1:172.21.15.69:6841/2060225181] up:active 2024-04-25T11:24:10.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:09 smithi152 ceph-mon[32701]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-25T11:24:10.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:09 smithi152 ceph-mon[32701]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T11:24:10.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:09 smithi146 ceph-mon[32821]: from='client.14883 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:24:10.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:09 smithi146 ceph-mon[32821]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:24:10.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:09 smithi146 ceph-mon[32821]: mds.? [v2:172.21.15.69:6840/2060225181,v1:172.21.15.69:6841/2060225181] up:active 2024-04-25T11:24:10.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:09 smithi146 ceph-mon[32821]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-25T11:24:10.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:09 smithi146 ceph-mon[32821]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T11:24:10.350 INFO:journalctl@ceph.mds.b.smithi146.stdout:Apr 25 11:24:10 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-b[62892]: 2024-04-25T11:24:10.076+0000 7f44c9446700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-25T11:24:10.350 INFO:journalctl@ceph.mds.b.smithi146.stdout:Apr 25 11:24:10 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-b[62892]: 2024-04-25T11:24:10.076+0000 7f44c9446700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-04-25T11:24:10.350 INFO:journalctl@ceph.mds.b.smithi146.stdout:Apr 25 11:24:10 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-b[62892]: 2024-04-25T11:24:10.076+0000 7f44c9446700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-04-25T11:24:10.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.396+0000 7f898828c640 1 -- 172.21.15.69:0/1155999108 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f898002dda0 con 0x7f8980103b00 2024-04-25T11:24:10.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.397+0000 7f89737fe640 1 -- 172.21.15.69:0/1155999108 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+24130 (secure 0 0 0) 0x7f897c0a7620 con 0x7f8980103b00 2024-04-25T11:24:10.391 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:24:10.391 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":80,"fsid":"f7ed73dc-02f4-11ef-bc93-c7b262605968","created":"2024-04-25T11:15:37.965281+0000","modified":"2024-04-25T11:23:47.662955+0000","last_up_change":"2024-04-25T11:22:50.691869+0000","last_in_change":"2024-04-25T11:22:30.271959+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:18:57.781263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:23:38.685110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:23:39.943803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T11:23:43.650521+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6a0d179-4998-4754-bf82-2ea3dac54e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6803","nonce":3756025235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6805","nonce":3756025235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6809","nonce":3756025235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6807","nonce":3756025235}]},"public_addr":"172.21.15.69:6803/3756025235","cluster_addr":"172.21.15.69:6805/3756025235","heartbeat_back_addr":"172.21.15.69:6809/3756025235","heartbeat_front_addr":"172.21.15.69:6807/3756025235","state":["exists","up"]},{"osd":1,"uuid":"c0c66930-4233-493f-b2bb-99c473994b59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6801","nonce":3166305213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6803","nonce":3166305213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6807","nonce":3166305213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6805","nonce":3166305213}]},"public_addr":"172.21.15.146:6801/3166305213","cluster_addr":"172.21.15.146:6803/3166305213","heartbeat_back_addr":"172.21.15.146:6807/3166305213","heartbeat_front_addr":"172.21.15.146:6805/3166305213","state":["exists","up"]},{"osd":2,"uuid":"d4ef735b-9786-4cb9-83ef-7c3f0691a729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6801","nonce":1474319175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6803","nonce":1474319175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6807","nonce":1474319175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6805","nonce":1474319175}]},"public_addr":"172.21.15.152:6801/1474319175","cluster_addr":"172.21.15.152:6803/1474319175","heartbeat_back_addr":"172.21.15.152:6807/1474319175","heartbeat_front_addr":"172.21.15.152:6805/1474319175","state":["exists","up"]},{"osd":3,"uuid":"0ab85fb8-5bbb-4ffc-bddf-cae7644a64ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6811","nonce":1809071296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6813","nonce":1809071296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6817","nonce":1809071296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6815","nonce":1809071296}]},"public_addr":"172.21.15.69:6811/1809071296","cluster_addr":"172.21.15.69:6813/1809071296","heartbeat_back_addr":"172.21.15.69:6817/1809071296","heartbeat_front_addr":"172.21.15.69:6815/1809071296","state":["exists","up"]},{"osd":4,"uuid":"273e0592-bbbc-4188-8eeb-46a3879cf632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6809","nonce":1898520972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6811","nonce":1898520972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6815","nonce":1898520972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6813","nonce":1898520972}]},"public_addr":"172.21.15.146:6809/1898520972","cluster_addr":"172.21.15.146:6811/1898520972","heartbeat_back_addr":"172.21.15.146:6815/1898520972","heartbeat_front_addr":"172.21.15.146:6813/1898520972","state":["exists","up"]},{"osd":5,"uuid":"bbeae945-9fa2-48f9-a1d6-8bf18dcaef19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6809","nonce":1996555108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6811","nonce":1996555108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6815","nonce":1996555108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6813","nonce":1996555108}]},"public_addr":"172.21.15.152:6809/1996555108","cluster_addr":"172.21.15.152:6811/1996555108","heartbeat_back_addr":"172.21.15.152:6815/1996555108","heartbeat_front_addr":"172.21.15.152:6813/1996555108","state":["exists","up"]},{"osd":6,"uuid":"dbb61253-30bc-418e-bd75-118c80bbf931","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6819","nonce":491215762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6821","nonce":491215762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6825","nonce":491215762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6823","nonce":491215762}]},"public_addr":"172.21.15.69:6819/491215762","cluster_addr":"172.21.15.69:6821/491215762","heartbeat_back_addr":"172.21.15.69:6825/491215762","heartbeat_front_addr":"172.21.15.69:6823/491215762","state":["exists","up"]},{"osd":7,"uuid":"75f23854-e283-4e3e-94cf-7b4079d83876","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6817","nonce":2330294690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6819","nonce":2330294690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6823","nonce":2330294690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6821","nonce":2330294690}]},"public_addr":"172.21.15.146:6817/2330294690","cluster_addr":"172.21.15.146:6819/2330294690","heartbeat_back_addr":"172.21.15.146:6823/2330294690","heartbeat_front_addr":"172.21.15.146:6821/2330294690","state":["exists","up"]},{"osd":8,"uuid":"1aa1060c-0c14-4abd-be56-5ebfeb78759c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6817","nonce":950669381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6819","nonce":950669381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6823","nonce":950669381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6821","nonce":950669381}]},"public_addr":"172.21.15.152:6817/950669381","cluster_addr":"172.21.15.152:6819/950669381","heartbeat_back_addr":"172.21.15.152:6823/950669381","heartbeat_front_addr":"172.21.15.152:6821/950669381","state":["exists","up"]},{"osd":9,"uuid":"3f4bf800-4090-4138-9bb3-d001f13f450c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6827","nonce":2414186758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6829","nonce":2414186758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6833","nonce":2414186758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6831","nonce":2414186758}]},"public_addr":"172.21.15.69:6827/2414186758","cluster_addr":"172.21.15.69:6829/2414186758","heartbeat_back_addr":"172.21.15.69:6833/2414186758","heartbeat_front_addr":"172.21.15.69:6831/2414186758","state":["exists","up"]},{"osd":10,"uuid":"19e967e5-51ce-4506-80c9-0a95441baf99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6825","nonce":707256400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6827","nonce":707256400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6831","nonce":707256400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6829","nonce":707256400}]},"public_addr":"172.21.15.146:6825/707256400","cluster_addr":"172.21.15.146:6827/707256400","heartbeat_back_addr":"172.21.15.146:6831/707256400","heartbeat_front_addr":"172.21.15.146:6829/707256400","state":["exists","up"]},{"osd":11,"uuid":"7788c2de-6930-4808-be0f-f1fd1ae0cb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6825","nonce":2934336833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6827","nonce":2934336833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6831","nonce":2934336833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6829","nonce":2934336833}]},"public_addr":"172.21.15.152:6825/2934336833","cluster_addr":"172.21.15.152:6827/2934336833","heartbeat_back_addr":"172.21.15.152:6831/2934336833","heartbeat_front_addr":"172.21.15.152:6829/2934336833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:07.497448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:31.227494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:55.027978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:20.713437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:46.214704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:20:11.199983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:20:37.177070+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:02.951240+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:28.963680+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:56.095807+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:22:22.221744+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:22:48.929334+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/1372727546":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6801/1097124842":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6800/1508198215":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3103465990":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3540132960":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3899912190":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6800/1097124842":"2024-04-26T11:16:31.688159+0000","172.21.15.69:0/1199907202":"2024-04-26T11:16:04.860792+0000","172.21.15.69:6801/1508198215":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/2879102529":"2024-04-26T11:16:31.688159+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:24:10.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.399+0000 7f898828c640 1 -- 172.21.15.69:0/1155999108 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f896c0afb80 msgr2=0x7f896c0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:10.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.399+0000 7f898828c640 1 --2- 172.21.15.69:0/1155999108 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f896c0afb80 0x7f896c0b2090 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f898019b170 tx=0x7f897400b390 comp rx=0 tx=0).stop 2024-04-25T11:24:10.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.399+0000 7f898828c640 1 -- 172.21.15.69:0/1155999108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8980103b00 msgr2=0x7f898019a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:10.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.399+0000 7f898828c640 1 --2- 172.21.15.69:0/1155999108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8980103b00 0x7f898019a0d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f897c00e9f0 tx=0x7f897c00eeb0 comp rx=0 tx=0).stop 2024-04-25T11:24:10.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.399+0000 7f898828c640 1 -- 172.21.15.69:0/1155999108 shutdown_connections 2024-04-25T11:24:10.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.399+0000 7f898828c640 1 --2- 172.21.15.69:0/1155999108 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f896c0afb80 0x7f896c0b2090 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:10.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.399+0000 7f898828c640 1 --2- 172.21.15.69:0/1155999108 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8980109a40 0x7f89801a15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:10.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.399+0000 7f898828c640 1 --2- 172.21.15.69:0/1155999108 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8980104d40 0x7f898019a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:10.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.399+0000 7f898828c640 1 --2- 172.21.15.69:0/1155999108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8980103b00 0x7f898019a0d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:10.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.399+0000 7f898828c640 1 -- 172.21.15.69:0/1155999108 >> 172.21.15.69:0/1155999108 conn(0x7f89800fed30 msgr2=0x7f8980105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:10.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.399+0000 7f898828c640 1 -- 172.21.15.69:0/1155999108 shutdown_connections 2024-04-25T11:24:10.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.399+0000 7f898828c640 1 -- 172.21.15.69:0/1155999108 wait complete. 2024-04-25T11:24:10.405 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T11:24:10.405 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:24:10.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.520+0000 7f3483bb4640 1 -- 172.21.15.69:0/2219304551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f347c103c20 msgr2=0x7f347c104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:10.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.520+0000 7f3483bb4640 1 --2- 172.21.15.69:0/2219304551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f347c103c20 0x7f347c104030 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f346c009900 tx=0x7f346c02a410 comp rx=0 tx=0).stop 2024-04-25T11:24:10.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.520+0000 7f3483bb4640 1 -- 172.21.15.69:0/2219304551 shutdown_connections 2024-04-25T11:24:10.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.520+0000 7f3483bb4640 1 --2- 172.21.15.69:0/2219304551 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f347c108920 0x7f347c10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:10.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.520+0000 7f3483bb4640 1 --2- 172.21.15.69:0/2219304551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f347c103c20 0x7f347c104030 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:10.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.520+0000 7f3483bb4640 1 --2- 172.21.15.69:0/2219304551 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f347c1029e0 0x7f347c1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:10.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.520+0000 7f3483bb4640 1 -- 172.21.15.69:0/2219304551 >> 172.21.15.69:0/2219304551 conn(0x7f347c0fdbd0 msgr2=0x7f347c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:10.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.520+0000 7f3483bb4640 1 -- 172.21.15.69:0/2219304551 shutdown_connections 2024-04-25T11:24:10.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.520+0000 7f3483bb4640 1 -- 172.21.15.69:0/2219304551 wait complete. 2024-04-25T11:24:10.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.521+0000 7f3483bb4640 1 Processor -- start 2024-04-25T11:24:10.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.521+0000 7f3483bb4640 1 -- start start 2024-04-25T11:24:10.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.521+0000 7f3483bb4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f347c1029e0 0x7f347c075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:10.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.521+0000 7f3483bb4640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f347c103c20 0x7f347c19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:10.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.521+0000 7f3483bb4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f347c108920 0x7f347c1a1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:10.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.521+0000 7f3483bb4640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f347c10e3c0 con 0x7f347c1029e0 2024-04-25T11:24:10.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.521+0000 7f3483bb4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f347c10e240 con 0x7f347c108920 2024-04-25T11:24:10.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.521+0000 7f3483bb4640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f347c10e540 con 0x7f347c103c20 2024-04-25T11:24:10.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.521+0000 7f3481950640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f347c1029e0 0x7f347c075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:10.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.521+0000 7f3481950640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f347c1029e0 0x7f347c075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45552/0 (socket says 172.21.15.69:45552) 2024-04-25T11:24:10.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.521+0000 7f3481950640 1 -- 172.21.15.69:0/2186184095 learned_addr learned my addr 172.21.15.69:0/2186184095 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:24:10.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.521+0000 7f3482151640 1 --2- 172.21.15.69:0/2186184095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f347c108920 0x7f347c1a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:10.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.521+0000 7f3481950640 1 -- 172.21.15.69:0/2186184095 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f347c103c20 msgr2=0x7f347c19ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:10.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.521+0000 7f3481950640 1 --2- 172.21.15.69:0/2186184095 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f347c103c20 0x7f347c19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:10.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.521+0000 7f3481950640 1 -- 172.21.15.69:0/2186184095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f347c108920 msgr2=0x7f347c1a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:10.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.521+0000 7f3481950640 1 --2- 172.21.15.69:0/2186184095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f347c108920 0x7f347c1a1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:10.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.521+0000 7f3481950640 1 -- 172.21.15.69:0/2186184095 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f347c1a1ca0 con 0x7f347c1029e0 2024-04-25T11:24:10.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.522+0000 7f3481950640 1 --2- 172.21.15.69:0/2186184095 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f347c1029e0 0x7f347c075840 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f347400eb90 tx=0x7f347400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:10.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.522+0000 7f3472ffd640 1 -- 172.21.15.69:0/2186184095 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3474003b70 con 0x7f347c1029e0 2024-04-25T11:24:10.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.522+0000 7f3472ffd640 1 -- 172.21.15.69:0/2186184095 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f347400ee90 con 0x7f347c1029e0 2024-04-25T11:24:10.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.522+0000 7f3483bb4640 1 -- 172.21.15.69:0/2186184095 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f347c1a1f90 con 0x7f347c1029e0 2024-04-25T11:24:10.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.522+0000 7f3472ffd640 1 -- 172.21.15.69:0/2186184095 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34740196c0 con 0x7f347c1029e0 2024-04-25T11:24:10.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.522+0000 7f3483bb4640 1 -- 172.21.15.69:0/2186184095 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f347c1a2450 con 0x7f347c1029e0 2024-04-25T11:24:10.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.523+0000 7f3483bb4640 1 -- 172.21.15.69:0/2186184095 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f347c107730 con 0x7f347c1029e0 2024-04-25T11:24:10.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.524+0000 7f3472ffd640 1 -- 172.21.15.69:0/2186184095 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f3474004b60 con 0x7f347c1029e0 2024-04-25T11:24:10.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.525+0000 7f3472ffd640 1 --2- 172.21.15.69:0/2186184095 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f34680afb80 0x7f34680b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:10.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.525+0000 7f348114f640 1 --2- 172.21.15.69:0/2186184095 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f34680afb80 0x7f34680b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:10.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.525+0000 7f3472ffd640 1 -- 172.21.15.69:0/2186184095 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7f34740dfa80 con 0x7f347c1029e0 2024-04-25T11:24:10.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.525+0000 7f348114f640 1 --2- 172.21.15.69:0/2186184095 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f34680afb80 0x7f34680b2090 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f346c02a930 tx=0x7f346c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:10.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.529+0000 7f3472ffd640 1 -- 172.21.15.69:0/2186184095 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f34740a8350 con 0x7f347c1029e0 2024-04-25T11:24:10.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.703+0000 7f3483bb4640 1 -- 172.21.15.69:0/2186184095 --> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f347c062fe0 con 0x7f34680afb80 2024-04-25T11:24:10.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.713+0000 7f3472ffd640 1 -- 172.21.15.69:0/2186184095 <== mgr.14152 v2:172.21.15.69:6800/4220065409 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340316 (secure 0 0 0) 0x7f347c062fe0 con 0x7f34680afb80 2024-04-25T11:24:10.711 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:24:10.713 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-04-25T11:24:10.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.716+0000 7f3483bb4640 1 -- 172.21.15.69:0/2186184095 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f34680afb80 msgr2=0x7f34680b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:10.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.716+0000 7f3483bb4640 1 --2- 172.21.15.69:0/2186184095 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f34680afb80 0x7f34680b2090 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f346c02a930 tx=0x7f346c000bc0 comp rx=0 tx=0).stop 2024-04-25T11:24:10.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.716+0000 7f3483bb4640 1 -- 172.21.15.69:0/2186184095 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f347c1029e0 msgr2=0x7f347c075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:10.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.716+0000 7f3483bb4640 1 --2- 172.21.15.69:0/2186184095 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f347c1029e0 0x7f347c075840 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f347400eb90 tx=0x7f347400c6a0 comp rx=0 tx=0).stop 2024-04-25T11:24:10.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.716+0000 7f3483bb4640 1 -- 172.21.15.69:0/2186184095 shutdown_connections 2024-04-25T11:24:10.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.716+0000 7f3483bb4640 1 --2- 172.21.15.69:0/2186184095 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f34680afb80 0x7f34680b2090 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:10.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.716+0000 7f3483bb4640 1 --2- 172.21.15.69:0/2186184095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f347c108920 0x7f347c1a1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:10.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.716+0000 7f3483bb4640 1 --2- 172.21.15.69:0/2186184095 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f347c103c20 0x7f347c19ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:10.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.716+0000 7f3483bb4640 1 --2- 172.21.15.69:0/2186184095 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f347c1029e0 0x7f347c075840 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:10.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.716+0000 7f3483bb4640 1 -- 172.21.15.69:0/2186184095 >> 172.21.15.69:0/2186184095 conn(0x7f347c0fdbd0 msgr2=0x7f347c104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:10.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.716+0000 7f3483bb4640 1 -- 172.21.15.69:0/2186184095 shutdown_connections 2024-04-25T11:24:10.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.716+0000 7f3483bb4640 1 -- 172.21.15.69:0/2186184095 wait complete. 2024-04-25T11:24:10.721 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":299,"stamp":"2024-04-25T11:24:09.786537+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":617971,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":101,"num_read_kb":88,"num_write":289,"num_write_kb":2228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":47,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":235,"ondisk_log_size":235,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365720,"kb_used_data":12384,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576104,"statfs":{"total":1151940427776,"available":1151565930496,"internally_reserved":0,"allocated":12681216,"data_stored":6989366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23104,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":25,"num_read_kb":24,"num_write":119,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003462"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.809037+0000","last_change":"2024-04-25T11:23:41.646759+0000","last_active":"2024-04-25T11:23:47.809037+0000","last_peered":"2024-04-25T11:23:47.809037+0000","last_clean":"2024-04-25T11:23:47.809037+0000","last_became_active":"2024-04-25T11:23:41.645882+0000","last_became_peered":"2024-04-25T11:23:41.645882+0000","last_unstale":"2024-04-25T11:23:47.809037+0000","last_undegraded":"2024-04-25T11:23:47.809037+0000","last_fullsized":"2024-04-25T11:23:47.809037+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:05.776414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807560+0000","last_change":"2024-04-25T11:23:41.648561+0000","last_active":"2024-04-25T11:23:47.807560+0000","last_peered":"2024-04-25T11:23:47.807560+0000","last_clean":"2024-04-25T11:23:47.807560+0000","last_became_active":"2024-04-25T11:23:41.647735+0000","last_became_peered":"2024-04-25T11:23:41.647735+0000","last_unstale":"2024-04-25T11:23:47.807560+0000","last_undegraded":"2024-04-25T11:23:47.807560+0000","last_fullsized":"2024-04-25T11:23:47.807560+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:20.303934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807099+0000","last_change":"2024-04-25T11:23:41.646656+0000","last_active":"2024-04-25T11:23:47.807099+0000","last_peered":"2024-04-25T11:23:47.807099+0000","last_clean":"2024-04-25T11:23:47.807099+0000","last_became_active":"2024-04-25T11:23:41.645487+0000","last_became_peered":"2024-04-25T11:23:41.645487+0000","last_unstale":"2024-04-25T11:23:47.807099+0000","last_undegraded":"2024-04-25T11:23:47.807099+0000","last_fullsized":"2024-04-25T11:23:47.807099+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:32.747478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.809763+0000","last_change":"2024-04-25T11:23:41.651994+0000","last_active":"2024-04-25T11:23:47.809763+0000","last_peered":"2024-04-25T11:23:47.809763+0000","last_clean":"2024-04-25T11:23:47.809763+0000","last_became_active":"2024-04-25T11:23:41.651640+0000","last_became_peered":"2024-04-25T11:23:41.651640+0000","last_unstale":"2024-04-25T11:23:47.809763+0000","last_undegraded":"2024-04-25T11:23:47.809763+0000","last_fullsized":"2024-04-25T11:23:47.809763+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:09.376398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808522+0000","last_change":"2024-04-25T11:23:41.650954+0000","last_active":"2024-04-25T11:23:47.808522+0000","last_peered":"2024-04-25T11:23:47.808522+0000","last_clean":"2024-04-25T11:23:47.808522+0000","last_became_active":"2024-04-25T11:23:41.650553+0000","last_became_peered":"2024-04-25T11:23:41.650553+0000","last_unstale":"2024-04-25T11:23:47.808522+0000","last_undegraded":"2024-04-25T11:23:47.808522+0000","last_fullsized":"2024-04-25T11:23:47.808522+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:52:34.348370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808283+0000","last_change":"2024-04-25T11:23:41.650770+0000","last_active":"2024-04-25T11:23:47.808283+0000","last_peered":"2024-04-25T11:23:47.808283+0000","last_clean":"2024-04-25T11:23:47.808283+0000","last_became_active":"2024-04-25T11:23:41.649794+0000","last_became_peered":"2024-04-25T11:23:41.649794+0000","last_unstale":"2024-04-25T11:23:47.808283+0000","last_undegraded":"2024-04-25T11:23:47.808283+0000","last_fullsized":"2024-04-25T11:23:47.808283+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:04.032001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807801+0000","last_change":"2024-04-25T11:23:41.643753+0000","last_active":"2024-04-25T11:23:47.807801+0000","last_peered":"2024-04-25T11:23:47.807801+0000","last_clean":"2024-04-25T11:23:47.807801+0000","last_became_active":"2024-04-25T11:23:41.642854+0000","last_became_peered":"2024-04-25T11:23:41.642854+0000","last_unstale":"2024-04-25T11:23:47.807801+0000","last_undegraded":"2024-04-25T11:23:47.807801+0000","last_fullsized":"2024-04-25T11:23:47.807801+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:40.868312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.668451+0000","last_change":"2024-04-25T11:23:41.646501+0000","last_active":"2024-04-25T11:23:47.668451+0000","last_peered":"2024-04-25T11:23:47.668451+0000","last_clean":"2024-04-25T11:23:47.668451+0000","last_became_active":"2024-04-25T11:23:41.646178+0000","last_became_peered":"2024-04-25T11:23:41.646178+0000","last_unstale":"2024-04-25T11:23:47.668451+0000","last_undegraded":"2024-04-25T11:23:47.668451+0000","last_fullsized":"2024-04-25T11:23:47.668451+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:34.699979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.667783+0000","last_change":"2024-04-25T11:23:41.647025+0000","last_active":"2024-04-25T11:23:47.667783+0000","last_peered":"2024-04-25T11:23:47.667783+0000","last_clean":"2024-04-25T11:23:47.667783+0000","last_became_active":"2024-04-25T11:23:41.646645+0000","last_became_peered":"2024-04-25T11:23:41.646645+0000","last_unstale":"2024-04-25T11:23:47.667783+0000","last_undegraded":"2024-04-25T11:23:47.667783+0000","last_fullsized":"2024-04-25T11:23:47.667783+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:24:57.108535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808683+0000","last_change":"2024-04-25T11:23:41.660957+0000","last_active":"2024-04-25T11:23:47.808683+0000","last_peered":"2024-04-25T11:23:47.808683+0000","last_clean":"2024-04-25T11:23:47.808683+0000","last_became_active":"2024-04-25T11:23:41.660529+0000","last_became_peered":"2024-04-25T11:23:41.660529+0000","last_unstale":"2024-04-25T11:23:47.808683+0000","last_undegraded":"2024-04-25T11:23:47.808683+0000","last_fullsized":"2024-04-25T11:23:47.808683+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:51.113991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.809101+0000","last_change":"2024-04-25T11:23:41.648777+0000","last_active":"2024-04-25T11:23:47.809101+0000","last_peered":"2024-04-25T11:23:47.809101+0000","last_clean":"2024-04-25T11:23:47.809101+0000","last_became_active":"2024-04-25T11:23:41.647282+0000","last_became_peered":"2024-04-25T11:23:41.647282+0000","last_unstale":"2024-04-25T11:23:47.809101+0000","last_undegraded":"2024-04-25T11:23:47.809101+0000","last_fullsized":"2024-04-25T11:23:47.809101+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:14.396228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808728+0000","last_change":"2024-04-25T11:23:41.648855+0000","last_active":"2024-04-25T11:23:47.808728+0000","last_peered":"2024-04-25T11:23:47.808728+0000","last_clean":"2024-04-25T11:23:47.808728+0000","last_became_active":"2024-04-25T11:23:41.648527+0000","last_became_peered":"2024-04-25T11:23:41.648527+0000","last_unstale":"2024-04-25T11:23:47.808728+0000","last_undegraded":"2024-04-25T11:23:47.808728+0000","last_fullsized":"2024-04-25T11:23:47.808728+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:13.569916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807392+0000","last_change":"2024-04-25T11:23:41.646377+0000","last_active":"2024-04-25T11:23:47.807392+0000","last_peered":"2024-04-25T11:23:47.807392+0000","last_clean":"2024-04-25T11:23:47.807392+0000","last_became_active":"2024-04-25T11:23:41.646063+0000","last_became_peered":"2024-04-25T11:23:41.646063+0000","last_unstale":"2024-04-25T11:23:47.807392+0000","last_undegraded":"2024-04-25T11:23:47.807392+0000","last_fullsized":"2024-04-25T11:23:47.807392+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:50.365317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.806904+0000","last_change":"2024-04-25T11:23:41.648147+0000","last_active":"2024-04-25T11:23:47.806904+0000","last_peered":"2024-04-25T11:23:47.806904+0000","last_clean":"2024-04-25T11:23:47.806904+0000","last_became_active":"2024-04-25T11:23:41.647771+0000","last_became_peered":"2024-04-25T11:23:41.647771+0000","last_unstale":"2024-04-25T11:23:47.806904+0000","last_undegraded":"2024-04-25T11:23:47.806904+0000","last_fullsized":"2024-04-25T11:23:47.806904+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:20.157971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807368+0000","last_change":"2024-04-25T11:23:41.652784+0000","last_active":"2024-04-25T11:23:47.807368+0000","last_peered":"2024-04-25T11:23:47.807368+0000","last_clean":"2024-04-25T11:23:47.807368+0000","last_became_active":"2024-04-25T11:23:41.652434+0000","last_became_peered":"2024-04-25T11:23:41.652434+0000","last_unstale":"2024-04-25T11:23:47.807368+0000","last_undegraded":"2024-04-25T11:23:47.807368+0000","last_fullsized":"2024-04-25T11:23:47.807368+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:48.468138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.667177+0000","last_change":"2024-04-25T11:23:41.648218+0000","last_active":"2024-04-25T11:23:47.667177+0000","last_peered":"2024-04-25T11:23:47.667177+0000","last_clean":"2024-04-25T11:23:47.667177+0000","last_became_active":"2024-04-25T11:23:41.647871+0000","last_became_peered":"2024-04-25T11:23:41.647871+0000","last_unstale":"2024-04-25T11:23:47.667177+0000","last_undegraded":"2024-04-25T11:23:47.667177+0000","last_fullsized":"2024-04-25T11:23:47.667177+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:49.919090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808862+0000","last_change":"2024-04-25T11:23:41.647087+0000","last_active":"2024-04-25T11:23:47.808862+0000","last_peered":"2024-04-25T11:23:47.808862+0000","last_clean":"2024-04-25T11:23:47.808862+0000","last_became_active":"2024-04-25T11:23:41.646359+0000","last_became_peered":"2024-04-25T11:23:41.646359+0000","last_unstale":"2024-04-25T11:23:47.808862+0000","last_undegraded":"2024-04-25T11:23:47.808862+0000","last_fullsized":"2024-04-25T11:23:47.808862+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:32.129538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808431+0000","last_change":"2024-04-25T11:23:41.644143+0000","last_active":"2024-04-25T11:23:47.808431+0000","last_peered":"2024-04-25T11:23:47.808431+0000","last_clean":"2024-04-25T11:23:47.808431+0000","last_became_active":"2024-04-25T11:23:41.643738+0000","last_became_peered":"2024-04-25T11:23:41.643738+0000","last_unstale":"2024-04-25T11:23:47.808431+0000","last_undegraded":"2024-04-25T11:23:47.808431+0000","last_fullsized":"2024-04-25T11:23:47.808431+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:19.786933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.809443+0000","last_change":"2024-04-25T11:23:41.636421+0000","last_active":"2024-04-25T11:23:47.809443+0000","last_peered":"2024-04-25T11:23:47.809443+0000","last_clean":"2024-04-25T11:23:47.809443+0000","last_became_active":"2024-04-25T11:23:41.635392+0000","last_became_peered":"2024-04-25T11:23:41.635392+0000","last_unstale":"2024-04-25T11:23:47.809443+0000","last_undegraded":"2024-04-25T11:23:47.809443+0000","last_fullsized":"2024-04-25T11:23:47.809443+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:59.859340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.809291+0000","last_change":"2024-04-25T11:23:45.675875+0000","last_active":"2024-04-25T11:23:47.809291+0000","last_peered":"2024-04-25T11:23:47.809291+0000","last_clean":"2024-04-25T11:23:47.809291+0000","last_became_active":"2024-04-25T11:23:45.675405+0000","last_became_peered":"2024-04-25T11:23:45.675405+0000","last_unstale":"2024-04-25T11:23:47.809291+0000","last_undegraded":"2024-04-25T11:23:47.809291+0000","last_fullsized":"2024-04-25T11:23:47.809291+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:06.732509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":38,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:24:08.066349+0000","last_change":"2024-04-25T11:23:40.630862+0000","last_active":"2024-04-25T11:24:08.066349+0000","last_peered":"2024-04-25T11:24:08.066349+0000","last_clean":"2024-04-25T11:24:08.066349+0000","last_became_active":"2024-04-25T11:23:40.630498+0000","last_became_peered":"2024-04-25T11:23:40.630498+0000","last_unstale":"2024-04-25T11:24:08.066349+0000","last_undegraded":"2024-04-25T11:24:08.066349+0000","last_fullsized":"2024-04-25T11:24:08.066349+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:07:44.028640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.668492+0000","last_change":"2024-04-25T11:23:41.639879+0000","last_active":"2024-04-25T11:23:47.668492+0000","last_peered":"2024-04-25T11:23:47.668492+0000","last_clean":"2024-04-25T11:23:47.668492+0000","last_became_active":"2024-04-25T11:23:41.639359+0000","last_became_peered":"2024-04-25T11:23:41.639359+0000","last_unstale":"2024-04-25T11:23:47.668492+0000","last_undegraded":"2024-04-25T11:23:47.668492+0000","last_fullsized":"2024-04-25T11:23:47.668492+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:47.525089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.809141+0000","last_change":"2024-04-25T11:23:45.672219+0000","last_active":"2024-04-25T11:23:47.809141+0000","last_peered":"2024-04-25T11:23:47.809141+0000","last_clean":"2024-04-25T11:23:47.809141+0000","last_became_active":"2024-04-25T11:23:45.671788+0000","last_became_peered":"2024-04-25T11:23:45.671788+0000","last_unstale":"2024-04-25T11:23:47.809141+0000","last_undegraded":"2024-04-25T11:23:47.809141+0000","last_fullsized":"2024-04-25T11:23:47.809141+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:36.228061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.668792+0000","last_change":"2024-04-25T11:23:40.643435+0000","last_active":"2024-04-25T11:23:47.668792+0000","last_peered":"2024-04-25T11:23:47.668792+0000","last_clean":"2024-04-25T11:23:47.668792+0000","last_became_active":"2024-04-25T11:23:40.642990+0000","last_became_peered":"2024-04-25T11:23:40.642990+0000","last_unstale":"2024-04-25T11:23:47.668792+0000","last_undegraded":"2024-04-25T11:23:47.668792+0000","last_fullsized":"2024-04-25T11:23:47.668792+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:42.723486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807765+0000","last_change":"2024-04-25T11:23:41.645977+0000","last_active":"2024-04-25T11:23:47.807765+0000","last_peered":"2024-04-25T11:23:47.807765+0000","last_clean":"2024-04-25T11:23:47.807765+0000","last_became_active":"2024-04-25T11:23:41.645562+0000","last_became_peered":"2024-04-25T11:23:41.645562+0000","last_unstale":"2024-04-25T11:23:47.807765+0000","last_undegraded":"2024-04-25T11:23:47.807765+0000","last_fullsized":"2024-04-25T11:23:47.807765+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:04.027324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807920+0000","last_change":"2024-04-25T11:23:45.672843+0000","last_active":"2024-04-25T11:23:47.807920+0000","last_peered":"2024-04-25T11:23:47.807920+0000","last_clean":"2024-04-25T11:23:47.807920+0000","last_became_active":"2024-04-25T11:23:45.672429+0000","last_became_peered":"2024-04-25T11:23:45.672429+0000","last_unstale":"2024-04-25T11:23:47.807920+0000","last_undegraded":"2024-04-25T11:23:47.807920+0000","last_fullsized":"2024-04-25T11:23:47.807920+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:48.818624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"74'2","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808083+0000","last_change":"2024-04-25T11:23:40.634393+0000","last_active":"2024-04-25T11:23:47.808083+0000","last_peered":"2024-04-25T11:23:47.808083+0000","last_clean":"2024-04-25T11:23:47.808083+0000","last_became_active":"2024-04-25T11:23:40.633518+0000","last_became_peered":"2024-04-25T11:23:40.633518+0000","last_unstale":"2024-04-25T11:23:47.808083+0000","last_undegraded":"2024-04-25T11:23:47.808083+0000","last_fullsized":"2024-04-25T11:23:47.808083+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:59.623135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.667422+0000","last_change":"2024-04-25T11:23:41.656152+0000","last_active":"2024-04-25T11:23:47.667422+0000","last_peered":"2024-04-25T11:23:47.667422+0000","last_clean":"2024-04-25T11:23:47.667422+0000","last_became_active":"2024-04-25T11:23:41.655344+0000","last_became_peered":"2024-04-25T11:23:41.655344+0000","last_unstale":"2024-04-25T11:23:47.667422+0000","last_undegraded":"2024-04-25T11:23:47.667422+0000","last_fullsized":"2024-04-25T11:23:47.667422+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:07.649534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.809772+0000","last_change":"2024-04-25T11:23:45.668602+0000","last_active":"2024-04-25T11:23:47.809772+0000","last_peered":"2024-04-25T11:23:47.809772+0000","last_clean":"2024-04-25T11:23:47.809772+0000","last_became_active":"2024-04-25T11:23:45.668143+0000","last_became_peered":"2024-04-25T11:23:45.668143+0000","last_unstale":"2024-04-25T11:23:47.809772+0000","last_undegraded":"2024-04-25T11:23:47.809772+0000","last_fullsized":"2024-04-25T11:23:47.809772+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:13.463604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"74'2","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807985+0000","last_change":"2024-04-25T11:23:40.639644+0000","last_active":"2024-04-25T11:23:47.807985+0000","last_peered":"2024-04-25T11:23:47.807985+0000","last_clean":"2024-04-25T11:23:47.807985+0000","last_became_active":"2024-04-25T11:23:40.639295+0000","last_became_peered":"2024-04-25T11:23:40.639295+0000","last_unstale":"2024-04-25T11:23:47.807985+0000","last_undegraded":"2024-04-25T11:23:47.807985+0000","last_fullsized":"2024-04-25T11:23:47.807985+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:43.849692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.667803+0000","last_change":"2024-04-25T11:23:41.657112+0000","last_active":"2024-04-25T11:23:47.667803+0000","last_peered":"2024-04-25T11:23:47.667803+0000","last_clean":"2024-04-25T11:23:47.667803+0000","last_became_active":"2024-04-25T11:23:41.656623+0000","last_became_peered":"2024-04-25T11:23:41.656623+0000","last_unstale":"2024-04-25T11:23:47.667803+0000","last_undegraded":"2024-04-25T11:23:47.667803+0000","last_fullsized":"2024-04-25T11:23:47.667803+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:41.644825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.809246+0000","last_change":"2024-04-25T11:23:45.673373+0000","last_active":"2024-04-25T11:23:47.809246+0000","last_peered":"2024-04-25T11:23:47.809246+0000","last_clean":"2024-04-25T11:23:47.809246+0000","last_became_active":"2024-04-25T11:23:45.672928+0000","last_became_peered":"2024-04-25T11:23:45.672928+0000","last_unstale":"2024-04-25T11:23:47.809246+0000","last_undegraded":"2024-04-25T11:23:47.809246+0000","last_fullsized":"2024-04-25T11:23:47.809246+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:10.553189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"80'5","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:24:08.059490+0000","last_change":"2024-04-25T11:23:40.635821+0000","last_active":"2024-04-25T11:24:08.059490+0000","last_peered":"2024-04-25T11:24:08.059490+0000","last_clean":"2024-04-25T11:24:08.059490+0000","last_became_active":"2024-04-25T11:23:40.635438+0000","last_became_peered":"2024-04-25T11:23:40.635438+0000","last_unstale":"2024-04-25T11:24:08.059490+0000","last_undegraded":"2024-04-25T11:24:08.059490+0000","last_fullsized":"2024-04-25T11:24:08.059490+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:55.750408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808248+0000","last_change":"2024-04-25T11:23:41.648771+0000","last_active":"2024-04-25T11:23:47.808248+0000","last_peered":"2024-04-25T11:23:47.808248+0000","last_clean":"2024-04-25T11:23:47.808248+0000","last_became_active":"2024-04-25T11:23:41.648059+0000","last_became_peered":"2024-04-25T11:23:41.648059+0000","last_unstale":"2024-04-25T11:23:47.808248+0000","last_undegraded":"2024-04-25T11:23:47.808248+0000","last_fullsized":"2024-04-25T11:23:47.808248+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:27.873310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.809609+0000","last_change":"2024-04-25T11:23:45.672676+0000","last_active":"2024-04-25T11:23:47.809609+0000","last_peered":"2024-04-25T11:23:47.809609+0000","last_clean":"2024-04-25T11:23:47.809609+0000","last_became_active":"2024-04-25T11:23:45.672229+0000","last_became_peered":"2024-04-25T11:23:45.672229+0000","last_unstale":"2024-04-25T11:23:47.809609+0000","last_undegraded":"2024-04-25T11:23:47.809609+0000","last_fullsized":"2024-04-25T11:23:47.809609+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:16.648809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:24:07.916201+0000","last_change":"2024-04-25T11:23:40.635988+0000","last_active":"2024-04-25T11:24:07.916201+0000","last_peered":"2024-04-25T11:24:07.916201+0000","last_clean":"2024-04-25T11:24:07.916201+0000","last_became_active":"2024-04-25T11:23:40.635366+0000","last_became_peered":"2024-04-25T11:23:40.635366+0000","last_unstale":"2024-04-25T11:24:07.916201+0000","last_undegraded":"2024-04-25T11:24:07.916201+0000","last_fullsized":"2024-04-25T11:24:07.916201+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:56.336232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.667736+0000","last_change":"2024-04-25T11:23:41.656031+0000","last_active":"2024-04-25T11:23:47.667736+0000","last_peered":"2024-04-25T11:23:47.667736+0000","last_clean":"2024-04-25T11:23:47.667736+0000","last_became_active":"2024-04-25T11:23:41.655656+0000","last_became_peered":"2024-04-25T11:23:41.655656+0000","last_unstale":"2024-04-25T11:23:47.667736+0000","last_undegraded":"2024-04-25T11:23:47.667736+0000","last_fullsized":"2024-04-25T11:23:47.667736+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:37.991435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808673+0000","last_change":"2024-04-25T11:23:45.675380+0000","last_active":"2024-04-25T11:23:47.808673+0000","last_peered":"2024-04-25T11:23:47.808673+0000","last_clean":"2024-04-25T11:23:47.808673+0000","last_became_active":"2024-04-25T11:23:45.674914+0000","last_became_peered":"2024-04-25T11:23:45.674914+0000","last_unstale":"2024-04-25T11:23:47.808673+0000","last_undegraded":"2024-04-25T11:23:47.808673+0000","last_fullsized":"2024-04-25T11:23:47.808673+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:26.231510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:24:06.903481+0000","last_change":"2024-04-25T11:23:40.631581+0000","last_active":"2024-04-25T11:24:06.903481+0000","last_peered":"2024-04-25T11:24:06.903481+0000","last_clean":"2024-04-25T11:24:06.903481+0000","last_became_active":"2024-04-25T11:23:40.631169+0000","last_became_peered":"2024-04-25T11:23:40.631169+0000","last_unstale":"2024-04-25T11:24:06.903481+0000","last_undegraded":"2024-04-25T11:24:06.903481+0000","last_fullsized":"2024-04-25T11:24:06.903481+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:15.737402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.668234+0000","last_change":"2024-04-25T11:23:41.633740+0000","last_active":"2024-04-25T11:23:47.668234+0000","last_peered":"2024-04-25T11:23:47.668234+0000","last_clean":"2024-04-25T11:23:47.668234+0000","last_became_active":"2024-04-25T11:23:41.633445+0000","last_became_peered":"2024-04-25T11:23:41.633445+0000","last_unstale":"2024-04-25T11:23:47.668234+0000","last_undegraded":"2024-04-25T11:23:47.668234+0000","last_fullsized":"2024-04-25T11:23:47.668234+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:46.103646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807455+0000","last_change":"2024-04-25T11:23:45.678899+0000","last_active":"2024-04-25T11:23:47.807455+0000","last_peered":"2024-04-25T11:23:47.807455+0000","last_clean":"2024-04-25T11:23:47.807455+0000","last_became_active":"2024-04-25T11:23:45.675305+0000","last_became_peered":"2024-04-25T11:23:45.675305+0000","last_unstale":"2024-04-25T11:23:47.807455+0000","last_undegraded":"2024-04-25T11:23:47.807455+0000","last_fullsized":"2024-04-25T11:23:47.807455+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:49.456551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"80'2","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:24:05.895935+0000","last_change":"2024-04-25T11:23:40.638090+0000","last_active":"2024-04-25T11:24:05.895935+0000","last_peered":"2024-04-25T11:24:05.895935+0000","last_clean":"2024-04-25T11:24:05.895935+0000","last_became_active":"2024-04-25T11:23:40.637707+0000","last_became_peered":"2024-04-25T11:23:40.637707+0000","last_unstale":"2024-04-25T11:24:05.895935+0000","last_undegraded":"2024-04-25T11:24:05.895935+0000","last_fullsized":"2024-04-25T11:24:05.895935+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:38:43.413273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808127+0000","last_change":"2024-04-25T11:23:41.641580+0000","last_active":"2024-04-25T11:23:47.808127+0000","last_peered":"2024-04-25T11:23:47.808127+0000","last_clean":"2024-04-25T11:23:47.808127+0000","last_became_active":"2024-04-25T11:23:41.641199+0000","last_became_peered":"2024-04-25T11:23:41.641199+0000","last_unstale":"2024-04-25T11:23:47.808127+0000","last_undegraded":"2024-04-25T11:23:47.808127+0000","last_fullsized":"2024-04-25T11:23:47.808127+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:34:10.471476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.668708+0000","last_change":"2024-04-25T11:23:45.675573+0000","last_active":"2024-04-25T11:23:47.668708+0000","last_peered":"2024-04-25T11:23:47.668708+0000","last_clean":"2024-04-25T11:23:47.668708+0000","last_became_active":"2024-04-25T11:23:45.675197+0000","last_became_peered":"2024-04-25T11:23:45.675197+0000","last_unstale":"2024-04-25T11:23:47.668708+0000","last_undegraded":"2024-04-25T11:23:47.668708+0000","last_fullsized":"2024-04-25T11:23:47.668708+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:39.740949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:24:08.058947+0000","last_change":"2024-04-25T11:23:40.635020+0000","last_active":"2024-04-25T11:24:08.058947+0000","last_peered":"2024-04-25T11:24:08.058947+0000","last_clean":"2024-04-25T11:24:08.058947+0000","last_became_active":"2024-04-25T11:23:40.634549+0000","last_became_peered":"2024-04-25T11:23:40.634549+0000","last_unstale":"2024-04-25T11:24:08.058947+0000","last_undegraded":"2024-04-25T11:24:08.058947+0000","last_fullsized":"2024-04-25T11:24:08.058947+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:15:16.289370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.667738+0000","last_change":"2024-04-25T11:23:41.654903+0000","last_active":"2024-04-25T11:23:47.667738+0000","last_peered":"2024-04-25T11:23:47.667738+0000","last_clean":"2024-04-25T11:23:47.667738+0000","last_became_active":"2024-04-25T11:23:41.654564+0000","last_became_peered":"2024-04-25T11:23:41.654564+0000","last_unstale":"2024-04-25T11:23:47.667738+0000","last_undegraded":"2024-04-25T11:23:47.667738+0000","last_fullsized":"2024-04-25T11:23:47.667738+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:29.025240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808068+0000","last_change":"2024-04-25T11:23:45.677683+0000","last_active":"2024-04-25T11:23:47.808068+0000","last_peered":"2024-04-25T11:23:47.808068+0000","last_clean":"2024-04-25T11:23:47.808068+0000","last_became_active":"2024-04-25T11:23:45.677286+0000","last_became_peered":"2024-04-25T11:23:45.677286+0000","last_unstale":"2024-04-25T11:23:47.808068+0000","last_undegraded":"2024-04-25T11:23:47.808068+0000","last_fullsized":"2024-04-25T11:23:47.808068+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:43.236060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808590+0000","last_change":"2024-04-25T11:23:40.640844+0000","last_active":"2024-04-25T11:23:47.808590+0000","last_peered":"2024-04-25T11:23:47.808590+0000","last_clean":"2024-04-25T11:23:47.808590+0000","last_became_active":"2024-04-25T11:23:40.640434+0000","last_became_peered":"2024-04-25T11:23:40.640434+0000","last_unstale":"2024-04-25T11:23:47.808590+0000","last_undegraded":"2024-04-25T11:23:47.808590+0000","last_fullsized":"2024-04-25T11:23:47.808590+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:16.701418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808035+0000","last_change":"2024-04-25T11:23:41.651962+0000","last_active":"2024-04-25T11:23:47.808035+0000","last_peered":"2024-04-25T11:23:47.808035+0000","last_clean":"2024-04-25T11:23:47.808035+0000","last_became_active":"2024-04-25T11:23:41.650870+0000","last_became_peered":"2024-04-25T11:23:41.650870+0000","last_unstale":"2024-04-25T11:23:47.808035+0000","last_undegraded":"2024-04-25T11:23:47.808035+0000","last_fullsized":"2024-04-25T11:23:47.808035+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:45.623722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808282+0000","last_change":"2024-04-25T11:23:45.668287+0000","last_active":"2024-04-25T11:23:47.808282+0000","last_peered":"2024-04-25T11:23:47.808282+0000","last_clean":"2024-04-25T11:23:47.808282+0000","last_became_active":"2024-04-25T11:23:45.667727+0000","last_became_peered":"2024-04-25T11:23:45.667727+0000","last_unstale":"2024-04-25T11:23:47.808282+0000","last_undegraded":"2024-04-25T11:23:47.808282+0000","last_fullsized":"2024-04-25T11:23:47.808282+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:44:07.699179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:24:07.912927+0000","last_change":"2024-04-25T11:23:40.642112+0000","last_active":"2024-04-25T11:24:07.912927+0000","last_peered":"2024-04-25T11:24:07.912927+0000","last_clean":"2024-04-25T11:24:07.912927+0000","last_became_active":"2024-04-25T11:23:40.641745+0000","last_became_peered":"2024-04-25T11:23:40.641745+0000","last_unstale":"2024-04-25T11:24:07.912927+0000","last_undegraded":"2024-04-25T11:24:07.912927+0000","last_fullsized":"2024-04-25T11:24:07.912927+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:14.899432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.667580+0000","last_change":"2024-04-25T11:23:41.656460+0000","last_active":"2024-04-25T11:23:47.667580+0000","last_peered":"2024-04-25T11:23:47.667580+0000","last_clean":"2024-04-25T11:23:47.667580+0000","last_became_active":"2024-04-25T11:23:41.655800+0000","last_became_peered":"2024-04-25T11:23:41.655800+0000","last_unstale":"2024-04-25T11:23:47.667580+0000","last_undegraded":"2024-04-25T11:23:47.667580+0000","last_fullsized":"2024-04-25T11:23:47.667580+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:42.505142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808104+0000","last_change":"2024-04-25T11:23:45.671650+0000","last_active":"2024-04-25T11:23:47.808104+0000","last_peered":"2024-04-25T11:23:47.808104+0000","last_clean":"2024-04-25T11:23:47.808104+0000","last_became_active":"2024-04-25T11:23:45.671254+0000","last_became_peered":"2024-04-25T11:23:45.671254+0000","last_unstale":"2024-04-25T11:23:47.808104+0000","last_undegraded":"2024-04-25T11:23:47.808104+0000","last_fullsized":"2024-04-25T11:23:47.808104+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:29.404898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"80'21","reported_seq":50,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:24:08.642197+0000","last_change":"2024-04-25T11:23:40.633712+0000","last_active":"2024-04-25T11:24:08.642197+0000","last_peered":"2024-04-25T11:24:08.642197+0000","last_clean":"2024-04-25T11:24:08.642197+0000","last_became_active":"2024-04-25T11:23:40.633274+0000","last_became_peered":"2024-04-25T11:23:40.633274+0000","last_unstale":"2024-04-25T11:24:08.642197+0000","last_undegraded":"2024-04-25T11:24:08.642197+0000","last_fullsized":"2024-04-25T11:24:08.642197+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:47.576213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23141,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":6,"num_write":23,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807871+0000","last_change":"2024-04-25T11:23:41.649007+0000","last_active":"2024-04-25T11:23:47.807871+0000","last_peered":"2024-04-25T11:23:47.807871+0000","last_clean":"2024-04-25T11:23:47.807871+0000","last_became_active":"2024-04-25T11:23:41.647778+0000","last_became_peered":"2024-04-25T11:23:41.647778+0000","last_unstale":"2024-04-25T11:23:47.807871+0000","last_undegraded":"2024-04-25T11:23:47.807871+0000","last_fullsized":"2024-04-25T11:23:47.807871+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:33.097482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.668274+0000","last_change":"2024-04-25T11:23:45.671645+0000","last_active":"2024-04-25T11:23:47.668274+0000","last_peered":"2024-04-25T11:23:47.668274+0000","last_clean":"2024-04-25T11:23:47.668274+0000","last_became_active":"2024-04-25T11:23:45.670005+0000","last_became_peered":"2024-04-25T11:23:45.670005+0000","last_unstale":"2024-04-25T11:23:47.668274+0000","last_undegraded":"2024-04-25T11:23:47.668274+0000","last_fullsized":"2024-04-25T11:23:47.668274+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:36.047222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"80'7","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:24:06.902822+0000","last_change":"2024-04-25T11:23:40.639922+0000","last_active":"2024-04-25T11:24:06.902822+0000","last_peered":"2024-04-25T11:24:06.902822+0000","last_clean":"2024-04-25T11:24:06.902822+0000","last_became_active":"2024-04-25T11:23:40.639544+0000","last_became_peered":"2024-04-25T11:23:40.639544+0000","last_unstale":"2024-04-25T11:24:06.902822+0000","last_undegraded":"2024-04-25T11:24:06.902822+0000","last_fullsized":"2024-04-25T11:24:06.902822+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:06.281281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807072+0000","last_change":"2024-04-25T11:23:41.629257+0000","last_active":"2024-04-25T11:23:47.807072+0000","last_peered":"2024-04-25T11:23:47.807072+0000","last_clean":"2024-04-25T11:23:47.807072+0000","last_became_active":"2024-04-25T11:23:41.628933+0000","last_became_peered":"2024-04-25T11:23:41.628933+0000","last_unstale":"2024-04-25T11:23:47.807072+0000","last_undegraded":"2024-04-25T11:23:47.807072+0000","last_fullsized":"2024-04-25T11:23:47.807072+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:04.031468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808813+0000","last_change":"2024-04-25T11:23:45.680460+0000","last_active":"2024-04-25T11:23:47.808813+0000","last_peered":"2024-04-25T11:23:47.808813+0000","last_clean":"2024-04-25T11:23:47.808813+0000","last_became_active":"2024-04-25T11:23:45.679963+0000","last_became_peered":"2024-04-25T11:23:45.679963+0000","last_unstale":"2024-04-25T11:23:47.808813+0000","last_undegraded":"2024-04-25T11:23:47.808813+0000","last_fullsized":"2024-04-25T11:23:47.808813+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:11.199855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"80'4","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:24:05.896179+0000","last_change":"2024-04-25T11:23:40.644026+0000","last_active":"2024-04-25T11:24:05.896179+0000","last_peered":"2024-04-25T11:24:05.896179+0000","last_clean":"2024-04-25T11:24:05.896179+0000","last_became_active":"2024-04-25T11:23:40.643665+0000","last_became_peered":"2024-04-25T11:23:40.643665+0000","last_unstale":"2024-04-25T11:24:05.896179+0000","last_undegraded":"2024-04-25T11:24:05.896179+0000","last_fullsized":"2024-04-25T11:24:05.896179+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:32.950387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808598+0000","last_change":"2024-04-25T11:23:41.642285+0000","last_active":"2024-04-25T11:23:47.808598+0000","last_peered":"2024-04-25T11:23:47.808598+0000","last_clean":"2024-04-25T11:23:47.808598+0000","last_became_active":"2024-04-25T11:23:41.641712+0000","last_became_peered":"2024-04-25T11:23:41.641712+0000","last_unstale":"2024-04-25T11:23:47.808598+0000","last_undegraded":"2024-04-25T11:23:47.808598+0000","last_fullsized":"2024-04-25T11:23:47.808598+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:22.813546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808373+0000","last_change":"2024-04-25T11:23:45.677632+0000","last_active":"2024-04-25T11:23:47.808373+0000","last_peered":"2024-04-25T11:23:47.808373+0000","last_clean":"2024-04-25T11:23:47.808373+0000","last_became_active":"2024-04-25T11:23:45.677244+0000","last_became_peered":"2024-04-25T11:23:45.677244+0000","last_unstale":"2024-04-25T11:23:47.808373+0000","last_undegraded":"2024-04-25T11:23:47.808373+0000","last_fullsized":"2024-04-25T11:23:47.808373+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:03.982435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"80'5","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:24:08.059729+0000","last_change":"2024-04-25T11:23:40.634661+0000","last_active":"2024-04-25T11:24:08.059729+0000","last_peered":"2024-04-25T11:24:08.059729+0000","last_clean":"2024-04-25T11:24:08.059729+0000","last_became_active":"2024-04-25T11:23:40.634211+0000","last_became_peered":"2024-04-25T11:23:40.634211+0000","last_unstale":"2024-04-25T11:24:08.059729+0000","last_undegraded":"2024-04-25T11:24:08.059729+0000","last_fullsized":"2024-04-25T11:24:08.059729+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:54.928339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808206+0000","last_change":"2024-04-25T11:23:41.644147+0000","last_active":"2024-04-25T11:23:47.808206+0000","last_peered":"2024-04-25T11:23:47.808206+0000","last_clean":"2024-04-25T11:23:47.808206+0000","last_became_active":"2024-04-25T11:23:41.643103+0000","last_became_peered":"2024-04-25T11:23:41.643103+0000","last_unstale":"2024-04-25T11:23:47.808206+0000","last_undegraded":"2024-04-25T11:23:47.808206+0000","last_fullsized":"2024-04-25T11:23:47.808206+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:00.956551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807838+0000","last_change":"2024-04-25T11:23:45.671504+0000","last_active":"2024-04-25T11:23:47.807838+0000","last_peered":"2024-04-25T11:23:47.807838+0000","last_clean":"2024-04-25T11:23:47.807838+0000","last_became_active":"2024-04-25T11:23:45.671148+0000","last_became_peered":"2024-04-25T11:23:45.671148+0000","last_unstale":"2024-04-25T11:23:47.807838+0000","last_undegraded":"2024-04-25T11:23:47.807838+0000","last_fullsized":"2024-04-25T11:23:47.807838+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:21.795839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"80'10","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:24:05.892611+0000","last_change":"2024-04-25T11:23:40.642021+0000","last_active":"2024-04-25T11:24:05.892611+0000","last_peered":"2024-04-25T11:24:05.892611+0000","last_clean":"2024-04-25T11:24:05.892611+0000","last_became_active":"2024-04-25T11:23:40.641639+0000","last_became_peered":"2024-04-25T11:23:40.641639+0000","last_unstale":"2024-04-25T11:24:05.892611+0000","last_undegraded":"2024-04-25T11:24:05.892611+0000","last_fullsized":"2024-04-25T11:24:05.892611+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:27.101032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.667187+0000","last_change":"2024-04-25T11:23:41.658283+0000","last_active":"2024-04-25T11:23:47.667187+0000","last_peered":"2024-04-25T11:23:47.667187+0000","last_clean":"2024-04-25T11:23:47.667187+0000","last_became_active":"2024-04-25T11:23:41.657898+0000","last_became_peered":"2024-04-25T11:23:41.657898+0000","last_unstale":"2024-04-25T11:23:47.667187+0000","last_undegraded":"2024-04-25T11:23:47.667187+0000","last_fullsized":"2024-04-25T11:23:47.667187+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:52:55.941232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.667374+0000","last_change":"2024-04-25T11:23:45.677362+0000","last_active":"2024-04-25T11:23:47.667374+0000","last_peered":"2024-04-25T11:23:47.667374+0000","last_clean":"2024-04-25T11:23:47.667374+0000","last_became_active":"2024-04-25T11:23:45.677024+0000","last_became_peered":"2024-04-25T11:23:45.677024+0000","last_unstale":"2024-04-25T11:23:47.667374+0000","last_undegraded":"2024-04-25T11:23:47.667374+0000","last_fullsized":"2024-04-25T11:23:47.667374+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:19.154764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"74'1","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807777+0000","last_change":"2024-04-25T11:23:40.639442+0000","last_active":"2024-04-25T11:23:47.807777+0000","last_peered":"2024-04-25T11:23:47.807777+0000","last_clean":"2024-04-25T11:23:47.807777+0000","last_became_active":"2024-04-25T11:23:40.639068+0000","last_became_peered":"2024-04-25T11:23:40.639068+0000","last_unstale":"2024-04-25T11:23:47.807777+0000","last_undegraded":"2024-04-25T11:23:47.807777+0000","last_fullsized":"2024-04-25T11:23:47.807777+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:21.491233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808780+0000","last_change":"2024-04-25T11:23:41.652309+0000","last_active":"2024-04-25T11:23:47.808780+0000","last_peered":"2024-04-25T11:23:47.808780+0000","last_clean":"2024-04-25T11:23:47.808780+0000","last_became_active":"2024-04-25T11:23:41.651927+0000","last_became_peered":"2024-04-25T11:23:41.651927+0000","last_unstale":"2024-04-25T11:23:47.808780+0000","last_undegraded":"2024-04-25T11:23:47.808780+0000","last_fullsized":"2024-04-25T11:23:47.808780+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:51.002480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808249+0000","last_change":"2024-04-25T11:23:46.384203+0000","last_active":"2024-04-25T11:23:47.808249+0000","last_peered":"2024-04-25T11:23:47.808249+0000","last_clean":"2024-04-25T11:23:47.808249+0000","last_became_active":"2024-04-25T11:23:46.383532+0000","last_became_peered":"2024-04-25T11:23:46.383532+0000","last_unstale":"2024-04-25T11:23:47.808249+0000","last_undegraded":"2024-04-25T11:23:47.808249+0000","last_fullsized":"2024-04-25T11:23:47.808249+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:25.265484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808435+0000","last_change":"2024-04-25T11:23:40.640082+0000","last_active":"2024-04-25T11:23:47.808435+0000","last_peered":"2024-04-25T11:23:47.808435+0000","last_clean":"2024-04-25T11:23:47.808435+0000","last_became_active":"2024-04-25T11:23:40.639725+0000","last_became_peered":"2024-04-25T11:23:40.639725+0000","last_unstale":"2024-04-25T11:23:47.808435+0000","last_undegraded":"2024-04-25T11:23:47.808435+0000","last_fullsized":"2024-04-25T11:23:47.808435+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:01.244116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808522+0000","last_change":"2024-04-25T11:23:41.645439+0000","last_active":"2024-04-25T11:23:47.808522+0000","last_peered":"2024-04-25T11:23:47.808522+0000","last_clean":"2024-04-25T11:23:47.808522+0000","last_became_active":"2024-04-25T11:23:41.645056+0000","last_became_peered":"2024-04-25T11:23:41.645056+0000","last_unstale":"2024-04-25T11:23:47.808522+0000","last_undegraded":"2024-04-25T11:23:47.808522+0000","last_fullsized":"2024-04-25T11:23:47.808522+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:43.100321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807759+0000","last_change":"2024-04-25T11:23:45.672174+0000","last_active":"2024-04-25T11:23:47.807759+0000","last_peered":"2024-04-25T11:23:47.807759+0000","last_clean":"2024-04-25T11:23:47.807759+0000","last_became_active":"2024-04-25T11:23:45.670401+0000","last_became_peered":"2024-04-25T11:23:45.670401+0000","last_unstale":"2024-04-25T11:23:47.807759+0000","last_undegraded":"2024-04-25T11:23:47.807759+0000","last_fullsized":"2024-04-25T11:23:47.807759+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:59.259644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.809259+0000","last_change":"2024-04-25T11:23:40.636704+0000","last_active":"2024-04-25T11:23:47.809259+0000","last_peered":"2024-04-25T11:23:47.809259+0000","last_clean":"2024-04-25T11:23:47.809259+0000","last_became_active":"2024-04-25T11:23:40.636319+0000","last_became_peered":"2024-04-25T11:23:40.636319+0000","last_unstale":"2024-04-25T11:23:47.809259+0000","last_undegraded":"2024-04-25T11:23:47.809259+0000","last_fullsized":"2024-04-25T11:23:47.809259+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:17.199946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807858+0000","last_change":"2024-04-25T11:23:41.660510+0000","last_active":"2024-04-25T11:23:47.807858+0000","last_peered":"2024-04-25T11:23:47.807858+0000","last_clean":"2024-04-25T11:23:47.807858+0000","last_became_active":"2024-04-25T11:23:41.659837+0000","last_became_peered":"2024-04-25T11:23:41.659837+0000","last_unstale":"2024-04-25T11:23:47.807858+0000","last_undegraded":"2024-04-25T11:23:47.807858+0000","last_fullsized":"2024-04-25T11:23:47.807858+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:33.879487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807124+0000","last_change":"2024-04-25T11:23:46.383239+0000","last_active":"2024-04-25T11:23:47.807124+0000","last_peered":"2024-04-25T11:23:47.807124+0000","last_clean":"2024-04-25T11:23:47.807124+0000","last_became_active":"2024-04-25T11:23:46.382728+0000","last_became_peered":"2024-04-25T11:23:46.382728+0000","last_unstale":"2024-04-25T11:23:47.807124+0000","last_undegraded":"2024-04-25T11:23:47.807124+0000","last_fullsized":"2024-04-25T11:23:47.807124+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:25.087389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.667390+0000","last_change":"2024-04-25T11:23:41.654723+0000","last_active":"2024-04-25T11:23:47.667390+0000","last_peered":"2024-04-25T11:23:47.667390+0000","last_clean":"2024-04-25T11:23:47.667390+0000","last_became_active":"2024-04-25T11:23:41.654140+0000","last_became_peered":"2024-04-25T11:23:41.654140+0000","last_unstale":"2024-04-25T11:23:47.667390+0000","last_undegraded":"2024-04-25T11:23:47.667390+0000","last_fullsized":"2024-04-25T11:23:47.667390+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:41.981728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:24:09.060515+0000","last_change":"2024-04-25T11:23:40.641902+0000","last_active":"2024-04-25T11:24:09.060515+0000","last_peered":"2024-04-25T11:24:09.060515+0000","last_clean":"2024-04-25T11:24:09.060515+0000","last_became_active":"2024-04-25T11:23:40.641490+0000","last_became_peered":"2024-04-25T11:23:40.641490+0000","last_unstale":"2024-04-25T11:24:09.060515+0000","last_undegraded":"2024-04-25T11:24:09.060515+0000","last_fullsized":"2024-04-25T11:24:09.060515+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:03.939354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808902+0000","last_change":"2024-04-25T11:23:45.670485+0000","last_active":"2024-04-25T11:23:47.808902+0000","last_peered":"2024-04-25T11:23:47.808902+0000","last_clean":"2024-04-25T11:23:47.808902+0000","last_became_active":"2024-04-25T11:23:45.669842+0000","last_became_peered":"2024-04-25T11:23:45.669842+0000","last_unstale":"2024-04-25T11:23:47.808902+0000","last_undegraded":"2024-04-25T11:23:47.808902+0000","last_fullsized":"2024-04-25T11:23:47.808902+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:32.801217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.668341+0000","last_change":"2024-04-25T11:23:41.640583+0000","last_active":"2024-04-25T11:23:47.668341+0000","last_peered":"2024-04-25T11:23:47.668341+0000","last_clean":"2024-04-25T11:23:47.668341+0000","last_became_active":"2024-04-25T11:23:41.640262+0000","last_became_peered":"2024-04-25T11:23:41.640262+0000","last_unstale":"2024-04-25T11:23:47.668341+0000","last_undegraded":"2024-04-25T11:23:47.668341+0000","last_fullsized":"2024-04-25T11:23:47.668341+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:09.539504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:24:08.065847+0000","last_change":"2024-04-25T11:23:40.628424+0000","last_active":"2024-04-25T11:24:08.065847+0000","last_peered":"2024-04-25T11:24:08.065847+0000","last_clean":"2024-04-25T11:24:08.065847+0000","last_became_active":"2024-04-25T11:23:40.628064+0000","last_became_peered":"2024-04-25T11:23:40.628064+0000","last_unstale":"2024-04-25T11:24:08.065847+0000","last_undegraded":"2024-04-25T11:24:08.065847+0000","last_fullsized":"2024-04-25T11:24:08.065847+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:34.837291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":99,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807960+0000","last_change":"2024-04-25T11:22:25.135774+0000","last_active":"2024-04-25T11:23:47.807960+0000","last_peered":"2024-04-25T11:23:47.807960+0000","last_clean":"2024-04-25T11:23:47.807960+0000","last_became_active":"2024-04-25T11:22:25.131178+0000","last_became_peered":"2024-04-25T11:22:25.131178+0000","last_unstale":"2024-04-25T11:23:47.807960+0000","last_undegraded":"2024-04-25T11:23:47.807960+0000","last_fullsized":"2024-04-25T11:23:47.807960+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:18:57.891013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:18:57.891013+0000","last_clean_scrub_stamp":"2024-04-25T11:18:57.891013+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:29.317685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.667984+0000","last_change":"2024-04-25T11:23:46.385048+0000","last_active":"2024-04-25T11:23:47.667984+0000","last_peered":"2024-04-25T11:23:47.667984+0000","last_clean":"2024-04-25T11:23:47.667984+0000","last_became_active":"2024-04-25T11:23:46.384014+0000","last_became_peered":"2024-04-25T11:23:46.384014+0000","last_unstale":"2024-04-25T11:23:47.667984+0000","last_undegraded":"2024-04-25T11:23:47.667984+0000","last_fullsized":"2024-04-25T11:23:47.667984+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:26.508616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.806557+0000","last_change":"2024-04-25T11:23:41.650783+0000","last_active":"2024-04-25T11:23:47.806557+0000","last_peered":"2024-04-25T11:23:47.806557+0000","last_clean":"2024-04-25T11:23:47.806557+0000","last_became_active":"2024-04-25T11:23:41.650431+0000","last_became_peered":"2024-04-25T11:23:41.650431+0000","last_unstale":"2024-04-25T11:23:47.806557+0000","last_undegraded":"2024-04-25T11:23:47.806557+0000","last_fullsized":"2024-04-25T11:23:47.806557+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:38.925779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808088+0000","last_change":"2024-04-25T11:23:40.638879+0000","last_active":"2024-04-25T11:23:47.808088+0000","last_peered":"2024-04-25T11:23:47.808088+0000","last_clean":"2024-04-25T11:23:47.808088+0000","last_became_active":"2024-04-25T11:23:40.638475+0000","last_became_peered":"2024-04-25T11:23:40.638475+0000","last_unstale":"2024-04-25T11:23:47.808088+0000","last_undegraded":"2024-04-25T11:23:47.808088+0000","last_fullsized":"2024-04-25T11:23:47.808088+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:25.591831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808503+0000","last_change":"2024-04-25T11:23:45.679333+0000","last_active":"2024-04-25T11:23:47.808503+0000","last_peered":"2024-04-25T11:23:47.808503+0000","last_clean":"2024-04-25T11:23:47.808503+0000","last_became_active":"2024-04-25T11:23:45.678622+0000","last_became_peered":"2024-04-25T11:23:45.678622+0000","last_unstale":"2024-04-25T11:23:47.808503+0000","last_undegraded":"2024-04-25T11:23:47.808503+0000","last_fullsized":"2024-04-25T11:23:47.808503+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:08.951281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808065+0000","last_change":"2024-04-25T11:23:40.642975+0000","last_active":"2024-04-25T11:23:47.808065+0000","last_peered":"2024-04-25T11:23:47.808065+0000","last_clean":"2024-04-25T11:23:47.808065+0000","last_became_active":"2024-04-25T11:23:40.642605+0000","last_became_peered":"2024-04-25T11:23:40.642605+0000","last_unstale":"2024-04-25T11:23:47.808065+0000","last_undegraded":"2024-04-25T11:23:47.808065+0000","last_fullsized":"2024-04-25T11:23:47.808065+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:57.665763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807680+0000","last_change":"2024-04-25T11:23:41.628396+0000","last_active":"2024-04-25T11:23:47.807680+0000","last_peered":"2024-04-25T11:23:47.807680+0000","last_clean":"2024-04-25T11:23:47.807680+0000","last_became_active":"2024-04-25T11:23:41.627979+0000","last_became_peered":"2024-04-25T11:23:41.627979+0000","last_unstale":"2024-04-25T11:23:47.807680+0000","last_undegraded":"2024-04-25T11:23:47.807680+0000","last_fullsized":"2024-04-25T11:23:47.807680+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:48.471219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.668877+0000","last_change":"2024-04-25T11:23:46.384252+0000","last_active":"2024-04-25T11:23:47.668877+0000","last_peered":"2024-04-25T11:23:47.668877+0000","last_clean":"2024-04-25T11:23:47.668877+0000","last_became_active":"2024-04-25T11:23:46.383408+0000","last_became_peered":"2024-04-25T11:23:46.383408+0000","last_unstale":"2024-04-25T11:23:47.668877+0000","last_undegraded":"2024-04-25T11:23:47.668877+0000","last_fullsized":"2024-04-25T11:23:47.668877+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:19.337043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"80'8","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:24:07.916188+0000","last_change":"2024-04-25T11:23:40.643767+0000","last_active":"2024-04-25T11:24:07.916188+0000","last_peered":"2024-04-25T11:24:07.916188+0000","last_clean":"2024-04-25T11:24:07.916188+0000","last_became_active":"2024-04-25T11:23:40.642748+0000","last_became_peered":"2024-04-25T11:23:40.642748+0000","last_unstale":"2024-04-25T11:24:07.916188+0000","last_undegraded":"2024-04-25T11:24:07.916188+0000","last_fullsized":"2024-04-25T11:24:07.916188+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:12.322055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.668696+0000","last_change":"2024-04-25T11:23:41.642766+0000","last_active":"2024-04-25T11:23:47.668696+0000","last_peered":"2024-04-25T11:23:47.668696+0000","last_clean":"2024-04-25T11:23:47.668696+0000","last_became_active":"2024-04-25T11:23:41.642409+0000","last_became_peered":"2024-04-25T11:23:41.642409+0000","last_unstale":"2024-04-25T11:23:47.668696+0000","last_undegraded":"2024-04-25T11:23:47.668696+0000","last_fullsized":"2024-04-25T11:23:47.668696+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:36.139574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.809051+0000","last_change":"2024-04-25T11:23:45.670926+0000","last_active":"2024-04-25T11:23:47.809051+0000","last_peered":"2024-04-25T11:23:47.809051+0000","last_clean":"2024-04-25T11:23:47.809051+0000","last_became_active":"2024-04-25T11:23:45.670444+0000","last_became_peered":"2024-04-25T11:23:45.670444+0000","last_unstale":"2024-04-25T11:23:47.809051+0000","last_undegraded":"2024-04-25T11:23:47.809051+0000","last_fullsized":"2024-04-25T11:23:47.809051+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:40.538155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:24:08.065782+0000","last_change":"2024-04-25T11:23:40.647132+0000","last_active":"2024-04-25T11:24:08.065782+0000","last_peered":"2024-04-25T11:24:08.065782+0000","last_clean":"2024-04-25T11:24:08.065782+0000","last_became_active":"2024-04-25T11:23:40.646792+0000","last_became_peered":"2024-04-25T11:23:40.646792+0000","last_unstale":"2024-04-25T11:24:08.065782+0000","last_undegraded":"2024-04-25T11:24:08.065782+0000","last_fullsized":"2024-04-25T11:24:08.065782+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:50.935908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807656+0000","last_change":"2024-04-25T11:23:41.658368+0000","last_active":"2024-04-25T11:23:47.807656+0000","last_peered":"2024-04-25T11:23:47.807656+0000","last_clean":"2024-04-25T11:23:47.807656+0000","last_became_active":"2024-04-25T11:23:41.657972+0000","last_became_peered":"2024-04-25T11:23:41.657972+0000","last_unstale":"2024-04-25T11:23:47.807656+0000","last_undegraded":"2024-04-25T11:23:47.807656+0000","last_fullsized":"2024-04-25T11:23:47.807656+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:02.717294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.809318+0000","last_change":"2024-04-25T11:23:46.384683+0000","last_active":"2024-04-25T11:23:47.809318+0000","last_peered":"2024-04-25T11:23:47.809318+0000","last_clean":"2024-04-25T11:23:47.809318+0000","last_became_active":"2024-04-25T11:23:46.384230+0000","last_became_peered":"2024-04-25T11:23:46.384230+0000","last_unstale":"2024-04-25T11:23:47.809318+0000","last_undegraded":"2024-04-25T11:23:47.809318+0000","last_fullsized":"2024-04-25T11:23:47.809318+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:06.982061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:24:05.895647+0000","last_change":"2024-04-25T11:23:40.642289+0000","last_active":"2024-04-25T11:24:05.895647+0000","last_peered":"2024-04-25T11:24:05.895647+0000","last_clean":"2024-04-25T11:24:05.895647+0000","last_became_active":"2024-04-25T11:23:40.641948+0000","last_became_peered":"2024-04-25T11:23:40.641948+0000","last_unstale":"2024-04-25T11:24:05.895647+0000","last_undegraded":"2024-04-25T11:24:05.895647+0000","last_fullsized":"2024-04-25T11:24:05.895647+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:39.265090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807380+0000","last_change":"2024-04-25T11:23:41.651792+0000","last_active":"2024-04-25T11:23:47.807380+0000","last_peered":"2024-04-25T11:23:47.807380+0000","last_clean":"2024-04-25T11:23:47.807380+0000","last_became_active":"2024-04-25T11:23:41.651413+0000","last_became_peered":"2024-04-25T11:23:41.651413+0000","last_unstale":"2024-04-25T11:23:47.807380+0000","last_undegraded":"2024-04-25T11:23:47.807380+0000","last_fullsized":"2024-04-25T11:23:47.807380+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:50.805600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808651+0000","last_change":"2024-04-25T11:23:45.670509+0000","last_active":"2024-04-25T11:23:47.808651+0000","last_peered":"2024-04-25T11:23:47.808651+0000","last_clean":"2024-04-25T11:23:47.808651+0000","last_became_active":"2024-04-25T11:23:45.670145+0000","last_became_peered":"2024-04-25T11:23:45.670145+0000","last_unstale":"2024-04-25T11:23:47.808651+0000","last_undegraded":"2024-04-25T11:23:47.808651+0000","last_fullsized":"2024-04-25T11:23:47.808651+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:03.764278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:24:05.895435+0000","last_change":"2024-04-25T11:23:40.644123+0000","last_active":"2024-04-25T11:24:05.895435+0000","last_peered":"2024-04-25T11:24:05.895435+0000","last_clean":"2024-04-25T11:24:05.895435+0000","last_became_active":"2024-04-25T11:23:40.643750+0000","last_became_peered":"2024-04-25T11:23:40.643750+0000","last_unstale":"2024-04-25T11:24:05.895435+0000","last_undegraded":"2024-04-25T11:24:05.895435+0000","last_fullsized":"2024-04-25T11:24:05.895435+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:49.613669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808227+0000","last_change":"2024-04-25T11:23:41.643606+0000","last_active":"2024-04-25T11:23:47.808227+0000","last_peered":"2024-04-25T11:23:47.808227+0000","last_clean":"2024-04-25T11:23:47.808227+0000","last_became_active":"2024-04-25T11:23:41.643248+0000","last_became_peered":"2024-04-25T11:23:41.643248+0000","last_unstale":"2024-04-25T11:23:47.808227+0000","last_undegraded":"2024-04-25T11:23:47.808227+0000","last_fullsized":"2024-04-25T11:23:47.808227+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:46.200959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808435+0000","last_change":"2024-04-25T11:23:45.674138+0000","last_active":"2024-04-25T11:23:47.808435+0000","last_peered":"2024-04-25T11:23:47.808435+0000","last_clean":"2024-04-25T11:23:47.808435+0000","last_became_active":"2024-04-25T11:23:45.672549+0000","last_became_peered":"2024-04-25T11:23:45.672549+0000","last_unstale":"2024-04-25T11:23:47.808435+0000","last_undegraded":"2024-04-25T11:23:47.808435+0000","last_fullsized":"2024-04-25T11:23:47.808435+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:11.345176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"80'2","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:24:06.889336+0000","last_change":"2024-04-25T11:23:40.636602+0000","last_active":"2024-04-25T11:24:06.889336+0000","last_peered":"2024-04-25T11:24:06.889336+0000","last_clean":"2024-04-25T11:24:06.889336+0000","last_became_active":"2024-04-25T11:23:40.636187+0000","last_became_peered":"2024-04-25T11:23:40.636187+0000","last_unstale":"2024-04-25T11:24:06.889336+0000","last_undegraded":"2024-04-25T11:24:06.889336+0000","last_fullsized":"2024-04-25T11:24:06.889336+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:15.894714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.668747+0000","last_change":"2024-04-25T11:23:41.647345+0000","last_active":"2024-04-25T11:23:47.668747+0000","last_peered":"2024-04-25T11:23:47.668747+0000","last_clean":"2024-04-25T11:23:47.668747+0000","last_became_active":"2024-04-25T11:23:41.646981+0000","last_became_peered":"2024-04-25T11:23:41.646981+0000","last_unstale":"2024-04-25T11:23:47.668747+0000","last_undegraded":"2024-04-25T11:23:47.668747+0000","last_fullsized":"2024-04-25T11:23:47.668747+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:30.096856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.809026+0000","last_change":"2024-04-25T11:23:46.384132+0000","last_active":"2024-04-25T11:23:47.809026+0000","last_peered":"2024-04-25T11:23:47.809026+0000","last_clean":"2024-04-25T11:23:47.809026+0000","last_became_active":"2024-04-25T11:23:46.383501+0000","last_became_peered":"2024-04-25T11:23:46.383501+0000","last_unstale":"2024-04-25T11:23:47.809026+0000","last_undegraded":"2024-04-25T11:23:47.809026+0000","last_fullsized":"2024-04-25T11:23:47.809026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:22.486326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:24:05.893999+0000","last_change":"2024-04-25T11:23:40.634942+0000","last_active":"2024-04-25T11:24:05.893999+0000","last_peered":"2024-04-25T11:24:05.893999+0000","last_clean":"2024-04-25T11:24:05.893999+0000","last_became_active":"2024-04-25T11:23:40.634557+0000","last_became_peered":"2024-04-25T11:23:40.634557+0000","last_unstale":"2024-04-25T11:24:05.893999+0000","last_undegraded":"2024-04-25T11:24:05.893999+0000","last_fullsized":"2024-04-25T11:24:05.893999+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:08.612328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.809156+0000","last_change":"2024-04-25T11:23:41.648985+0000","last_active":"2024-04-25T11:23:47.809156+0000","last_peered":"2024-04-25T11:23:47.809156+0000","last_clean":"2024-04-25T11:23:47.809156+0000","last_became_active":"2024-04-25T11:23:41.645643+0000","last_became_peered":"2024-04-25T11:23:41.645643+0000","last_unstale":"2024-04-25T11:23:47.809156+0000","last_undegraded":"2024-04-25T11:23:47.809156+0000","last_fullsized":"2024-04-25T11:23:47.809156+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:22.289698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.667439+0000","last_change":"2024-04-25T11:23:45.671474+0000","last_active":"2024-04-25T11:23:47.667439+0000","last_peered":"2024-04-25T11:23:47.667439+0000","last_clean":"2024-04-25T11:23:47.667439+0000","last_became_active":"2024-04-25T11:23:45.670976+0000","last_became_peered":"2024-04-25T11:23:45.670976+0000","last_unstale":"2024-04-25T11:23:47.667439+0000","last_undegraded":"2024-04-25T11:23:47.667439+0000","last_fullsized":"2024-04-25T11:23:47.667439+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:13:29.771087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"80'4","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:24:06.903258+0000","last_change":"2024-04-25T11:23:40.647950+0000","last_active":"2024-04-25T11:24:06.903258+0000","last_peered":"2024-04-25T11:24:06.903258+0000","last_clean":"2024-04-25T11:24:06.903258+0000","last_became_active":"2024-04-25T11:23:40.647549+0000","last_became_peered":"2024-04-25T11:23:40.647549+0000","last_unstale":"2024-04-25T11:24:06.903258+0000","last_undegraded":"2024-04-25T11:24:06.903258+0000","last_fullsized":"2024-04-25T11:24:06.903258+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:47.975294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.809264+0000","last_change":"2024-04-25T11:23:41.647670+0000","last_active":"2024-04-25T11:23:47.809264+0000","last_peered":"2024-04-25T11:23:47.809264+0000","last_clean":"2024-04-25T11:23:47.809264+0000","last_became_active":"2024-04-25T11:23:41.646686+0000","last_became_peered":"2024-04-25T11:23:41.646686+0000","last_unstale":"2024-04-25T11:23:47.809264+0000","last_undegraded":"2024-04-25T11:23:47.809264+0000","last_fullsized":"2024-04-25T11:23:47.809264+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:21.260685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.806946+0000","last_change":"2024-04-25T11:23:45.676721+0000","last_active":"2024-04-25T11:23:47.806946+0000","last_peered":"2024-04-25T11:23:47.806946+0000","last_clean":"2024-04-25T11:23:47.806946+0000","last_became_active":"2024-04-25T11:23:45.676325+0000","last_became_peered":"2024-04-25T11:23:45.676325+0000","last_unstale":"2024-04-25T11:23:47.806946+0000","last_undegraded":"2024-04-25T11:23:47.806946+0000","last_fullsized":"2024-04-25T11:23:47.806946+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:12.023474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"80'2","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:24:07.914780+0000","last_change":"2024-04-25T11:23:40.642974+0000","last_active":"2024-04-25T11:24:07.914780+0000","last_peered":"2024-04-25T11:24:07.914780+0000","last_clean":"2024-04-25T11:24:07.914780+0000","last_became_active":"2024-04-25T11:23:40.642628+0000","last_became_peered":"2024-04-25T11:23:40.642628+0000","last_unstale":"2024-04-25T11:24:07.914780+0000","last_undegraded":"2024-04-25T11:24:07.914780+0000","last_fullsized":"2024-04-25T11:24:07.914780+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:03.870867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.809465+0000","last_change":"2024-04-25T11:23:41.643519+0000","last_active":"2024-04-25T11:23:47.809465+0000","last_peered":"2024-04-25T11:23:47.809465+0000","last_clean":"2024-04-25T11:23:47.809465+0000","last_became_active":"2024-04-25T11:23:41.642621+0000","last_became_peered":"2024-04-25T11:23:41.642621+0000","last_unstale":"2024-04-25T11:23:47.809465+0000","last_undegraded":"2024-04-25T11:23:47.809465+0000","last_fullsized":"2024-04-25T11:23:47.809465+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:38.699914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.667080+0000","last_change":"2024-04-25T11:23:45.671017+0000","last_active":"2024-04-25T11:23:47.667080+0000","last_peered":"2024-04-25T11:23:47.667080+0000","last_clean":"2024-04-25T11:23:47.667080+0000","last_became_active":"2024-04-25T11:23:45.670603+0000","last_became_peered":"2024-04-25T11:23:45.670603+0000","last_unstale":"2024-04-25T11:23:47.667080+0000","last_undegraded":"2024-04-25T11:23:47.667080+0000","last_fullsized":"2024-04-25T11:23:47.667080+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:32.385176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:24:08.064752+0000","last_change":"2024-04-25T11:23:40.635711+0000","last_active":"2024-04-25T11:24:08.064752+0000","last_peered":"2024-04-25T11:24:08.064752+0000","last_clean":"2024-04-25T11:24:08.064752+0000","last_became_active":"2024-04-25T11:23:40.635311+0000","last_became_peered":"2024-04-25T11:23:40.635311+0000","last_unstale":"2024-04-25T11:24:08.064752+0000","last_undegraded":"2024-04-25T11:24:08.064752+0000","last_fullsized":"2024-04-25T11:24:08.064752+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:30:01.135579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.809121+0000","last_change":"2024-04-25T11:23:41.634946+0000","last_active":"2024-04-25T11:23:47.809121+0000","last_peered":"2024-04-25T11:23:47.809121+0000","last_clean":"2024-04-25T11:23:47.809121+0000","last_became_active":"2024-04-25T11:23:41.634505+0000","last_became_peered":"2024-04-25T11:23:41.634505+0000","last_unstale":"2024-04-25T11:23:47.809121+0000","last_undegraded":"2024-04-25T11:23:47.809121+0000","last_fullsized":"2024-04-25T11:23:47.809121+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:52.793207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808398+0000","last_change":"2024-04-25T11:23:41.642079+0000","last_active":"2024-04-25T11:23:47.808398+0000","last_peered":"2024-04-25T11:23:47.808398+0000","last_clean":"2024-04-25T11:23:47.808398+0000","last_became_active":"2024-04-25T11:23:41.641721+0000","last_became_peered":"2024-04-25T11:23:41.641721+0000","last_unstale":"2024-04-25T11:23:47.808398+0000","last_undegraded":"2024-04-25T11:23:47.808398+0000","last_fullsized":"2024-04-25T11:23:47.808398+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:59.968749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.809166+0000","last_change":"2024-04-25T11:23:41.650503+0000","last_active":"2024-04-25T11:23:47.809166+0000","last_peered":"2024-04-25T11:23:47.809166+0000","last_clean":"2024-04-25T11:23:47.809166+0000","last_became_active":"2024-04-25T11:23:41.650158+0000","last_became_peered":"2024-04-25T11:23:41.650158+0000","last_unstale":"2024-04-25T11:23:47.809166+0000","last_undegraded":"2024-04-25T11:23:47.809166+0000","last_fullsized":"2024-04-25T11:23:47.809166+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:07.228378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807856+0000","last_change":"2024-04-25T11:23:41.641108+0000","last_active":"2024-04-25T11:23:47.807856+0000","last_peered":"2024-04-25T11:23:47.807856+0000","last_clean":"2024-04-25T11:23:47.807856+0000","last_became_active":"2024-04-25T11:23:41.640665+0000","last_became_peered":"2024-04-25T11:23:41.640665+0000","last_unstale":"2024-04-25T11:23:47.807856+0000","last_undegraded":"2024-04-25T11:23:47.807856+0000","last_fullsized":"2024-04-25T11:23:47.807856+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:48.075466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807285+0000","last_change":"2024-04-25T11:23:41.652322+0000","last_active":"2024-04-25T11:23:47.807285+0000","last_peered":"2024-04-25T11:23:47.807285+0000","last_clean":"2024-04-25T11:23:47.807285+0000","last_became_active":"2024-04-25T11:23:41.651681+0000","last_became_peered":"2024-04-25T11:23:41.651681+0000","last_unstale":"2024-04-25T11:23:47.807285+0000","last_undegraded":"2024-04-25T11:23:47.807285+0000","last_fullsized":"2024-04-25T11:23:47.807285+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:17.574861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807783+0000","last_change":"2024-04-25T11:23:41.651701+0000","last_active":"2024-04-25T11:23:47.807783+0000","last_peered":"2024-04-25T11:23:47.807783+0000","last_clean":"2024-04-25T11:23:47.807783+0000","last_became_active":"2024-04-25T11:23:41.651362+0000","last_became_peered":"2024-04-25T11:23:41.651362+0000","last_unstale":"2024-04-25T11:23:47.807783+0000","last_undegraded":"2024-04-25T11:23:47.807783+0000","last_fullsized":"2024-04-25T11:23:47.807783+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:50.777396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.809360+0000","last_change":"2024-04-25T11:23:41.640385+0000","last_active":"2024-04-25T11:23:47.809360+0000","last_peered":"2024-04-25T11:23:47.809360+0000","last_clean":"2024-04-25T11:23:47.809360+0000","last_became_active":"2024-04-25T11:23:41.639986+0000","last_became_peered":"2024-04-25T11:23:41.639986+0000","last_unstale":"2024-04-25T11:23:47.809360+0000","last_undegraded":"2024-04-25T11:23:47.809360+0000","last_fullsized":"2024-04-25T11:23:47.809360+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:13.376858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808557+0000","last_change":"2024-04-25T11:23:41.648075+0000","last_active":"2024-04-25T11:23:47.808557+0000","last_peered":"2024-04-25T11:23:47.808557+0000","last_clean":"2024-04-25T11:23:47.808557+0000","last_became_active":"2024-04-25T11:23:41.647681+0000","last_became_peered":"2024-04-25T11:23:41.647681+0000","last_unstale":"2024-04-25T11:23:47.808557+0000","last_undegraded":"2024-04-25T11:23:47.808557+0000","last_fullsized":"2024-04-25T11:23:47.808557+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:44.916762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.668797+0000","last_change":"2024-04-25T11:23:41.641788+0000","last_active":"2024-04-25T11:23:47.668797+0000","last_peered":"2024-04-25T11:23:47.668797+0000","last_clean":"2024-04-25T11:23:47.668797+0000","last_became_active":"2024-04-25T11:23:41.641407+0000","last_became_peered":"2024-04-25T11:23:41.641407+0000","last_unstale":"2024-04-25T11:23:47.668797+0000","last_undegraded":"2024-04-25T11:23:47.668797+0000","last_fullsized":"2024-04-25T11:23:47.668797+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:33.609749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.667335+0000","last_change":"2024-04-25T11:23:41.647616+0000","last_active":"2024-04-25T11:23:47.667335+0000","last_peered":"2024-04-25T11:23:47.667335+0000","last_clean":"2024-04-25T11:23:47.667335+0000","last_became_active":"2024-04-25T11:23:41.645396+0000","last_became_peered":"2024-04-25T11:23:41.645396+0000","last_unstale":"2024-04-25T11:23:47.667335+0000","last_undegraded":"2024-04-25T11:23:47.667335+0000","last_fullsized":"2024-04-25T11:23:47.667335+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:47.842574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.668544+0000","last_change":"2024-04-25T11:23:41.638407+0000","last_active":"2024-04-25T11:23:47.668544+0000","last_peered":"2024-04-25T11:23:47.668544+0000","last_clean":"2024-04-25T11:23:47.668544+0000","last_became_active":"2024-04-25T11:23:41.638054+0000","last_became_peered":"2024-04-25T11:23:41.638054+0000","last_unstale":"2024-04-25T11:23:47.668544+0000","last_undegraded":"2024-04-25T11:23:47.668544+0000","last_fullsized":"2024-04-25T11:23:47.668544+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:34.306493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.667520+0000","last_change":"2024-04-25T11:23:41.650140+0000","last_active":"2024-04-25T11:23:47.667520+0000","last_peered":"2024-04-25T11:23:47.667520+0000","last_clean":"2024-04-25T11:23:47.667520+0000","last_became_active":"2024-04-25T11:23:41.649487+0000","last_became_peered":"2024-04-25T11:23:41.649487+0000","last_unstale":"2024-04-25T11:23:47.667520+0000","last_undegraded":"2024-04-25T11:23:47.667520+0000","last_fullsized":"2024-04-25T11:23:47.667520+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:55.652921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807201+0000","last_change":"2024-04-25T11:23:41.640253+0000","last_active":"2024-04-25T11:23:47.807201+0000","last_peered":"2024-04-25T11:23:47.807201+0000","last_clean":"2024-04-25T11:23:47.807201+0000","last_became_active":"2024-04-25T11:23:41.639926+0000","last_became_peered":"2024-04-25T11:23:41.639926+0000","last_unstale":"2024-04-25T11:23:47.807201+0000","last_undegraded":"2024-04-25T11:23:47.807201+0000","last_fullsized":"2024-04-25T11:23:47.807201+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:01.407198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.809145+0000","last_change":"2024-04-25T11:23:41.653244+0000","last_active":"2024-04-25T11:23:47.809145+0000","last_peered":"2024-04-25T11:23:47.809145+0000","last_clean":"2024-04-25T11:23:47.809145+0000","last_became_active":"2024-04-25T11:23:41.652875+0000","last_became_peered":"2024-04-25T11:23:41.652875+0000","last_unstale":"2024-04-25T11:23:47.809145+0000","last_undegraded":"2024-04-25T11:23:47.809145+0000","last_fullsized":"2024-04-25T11:23:47.809145+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:06.897766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":27603,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":25,"num_read_kb":24,"num_write":166,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":47,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":319488,"data_stored":61948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710738,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":431863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906967,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28524,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716628,"statfs":{"total":95995035648,"available":95965827072,"internally_reserved":0,"allocated":1544192,"data_stored":1034409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070493,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":921600,"data_stored":432985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266721,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":892928,"data_stored":432467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462949,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28468,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716684,"statfs":{"total":95995035648,"available":95965884416,"internally_reserved":0,"allocated":1482752,"data_stored":1022813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626477,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":901120,"data_stored":432585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":444057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738442,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":888832,"data_stored":431953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411393,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":437055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182459,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":432147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986231,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":432563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822705,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36820,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708332,"statfs":{"total":95995035648,"available":95957331968,"internally_reserved":0,"allocated":1519616,"data_stored":1024469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":24893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":24877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:24:10.728 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:24:10.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.875+0000 7f4cb76aa640 1 -- 172.21.15.69:0/2905704094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4cb0108940 msgr2=0x7f4cb010ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:10.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.875+0000 7f4cb76aa640 1 --2- 172.21.15.69:0/2905704094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4cb0108940 0x7f4cb010ad60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4ca8009ca0 tx=0x7f4ca802a8c0 comp rx=0 tx=0).stop 2024-04-25T11:24:10.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.876+0000 7f4cb76aa640 1 -- 172.21.15.69:0/2905704094 shutdown_connections 2024-04-25T11:24:10.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.876+0000 7f4cb76aa640 1 --2- 172.21.15.69:0/2905704094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4cb0108940 0x7f4cb010ad60 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:10.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.876+0000 7f4cb76aa640 1 --2- 172.21.15.69:0/2905704094 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cb0103c40 0x7f4cb0104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:10.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.876+0000 7f4cb76aa640 1 --2- 172.21.15.69:0/2905704094 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4cb0102a00 0x7f4cb0103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:10.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.876+0000 7f4cb76aa640 1 -- 172.21.15.69:0/2905704094 >> 172.21.15.69:0/2905704094 conn(0x7f4cb00fdbf0 msgr2=0x7f4cb01000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:10.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.876+0000 7f4cb76aa640 1 -- 172.21.15.69:0/2905704094 shutdown_connections 2024-04-25T11:24:10.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.876+0000 7f4cb76aa640 1 -- 172.21.15.69:0/2905704094 wait complete. 2024-04-25T11:24:10.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.876+0000 7f4cb76aa640 1 Processor -- start 2024-04-25T11:24:10.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.876+0000 7f4cb76aa640 1 -- start start 2024-04-25T11:24:10.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.876+0000 7f4cb76aa640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4cb0102a00 0x7f4cb0199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:10.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.876+0000 7f4cb76aa640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4cb0103c40 0x7f4cb0199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:10.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.876+0000 7f4cb76aa640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cb0108940 0x7f4cb01a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:10.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.876+0000 7f4cb76aa640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb010e430 con 0x7f4cb0103c40 2024-04-25T11:24:10.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.876+0000 7f4cb76aa640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb010e2b0 con 0x7f4cb0102a00 2024-04-25T11:24:10.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.876+0000 7f4cb76aa640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb010e5b0 con 0x7f4cb0108940 2024-04-25T11:24:10.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.877+0000 7f4cb4c45640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4cb0103c40 0x7f4cb0199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:10.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.877+0000 7f4cb4c45640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4cb0103c40 0x7f4cb0199540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45582/0 (socket says 172.21.15.69:45582) 2024-04-25T11:24:10.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.877+0000 7f4cb4c45640 1 -- 172.21.15.69:0/1227979694 learned_addr learned my addr 172.21.15.69:0/1227979694 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:24:10.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.877+0000 7f4cb5c47640 1 --2- 172.21.15.69:0/1227979694 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cb0108940 0x7f4cb01a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:10.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.877+0000 7f4cb5446640 1 --2- 172.21.15.69:0/1227979694 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4cb0102a00 0x7f4cb0199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:10.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.877+0000 7f4cb4c45640 1 -- 172.21.15.69:0/1227979694 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cb0108940 msgr2=0x7f4cb01a0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:10.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.877+0000 7f4cb4c45640 1 --2- 172.21.15.69:0/1227979694 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cb0108940 0x7f4cb01a0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:10.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.877+0000 7f4cb4c45640 1 -- 172.21.15.69:0/1227979694 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4cb0102a00 msgr2=0x7f4cb0199000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:10.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.877+0000 7f4cb4c45640 1 --2- 172.21.15.69:0/1227979694 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4cb0102a00 0x7f4cb0199000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:10.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.877+0000 7f4cb4c45640 1 -- 172.21.15.69:0/1227979694 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cb01a0ad0 con 0x7f4cb0103c40 2024-04-25T11:24:10.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.877+0000 7f4cb4c45640 1 --2- 172.21.15.69:0/1227979694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4cb0103c40 0x7f4cb0199540 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0009860 tx=0x7f4ca0009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:10.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.877+0000 7f4ca67fc640 1 -- 172.21.15.69:0/1227979694 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca0005020 con 0x7f4cb0103c40 2024-04-25T11:24:10.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.877+0000 7f4ca67fc640 1 -- 172.21.15.69:0/1227979694 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4ca00047a0 con 0x7f4cb0103c40 2024-04-25T11:24:10.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.877+0000 7f4cb76aa640 1 -- 172.21.15.69:0/1227979694 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cb0074e30 con 0x7f4cb0103c40 2024-04-25T11:24:10.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.877+0000 7f4ca67fc640 1 -- 172.21.15.69:0/1227979694 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca0007a00 con 0x7f4cb0103c40 2024-04-25T11:24:10.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.877+0000 7f4cb76aa640 1 -- 172.21.15.69:0/1227979694 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cb0075330 con 0x7f4cb0103c40 2024-04-25T11:24:10.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.878+0000 7f4cb76aa640 1 -- 172.21.15.69:0/1227979694 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cb006a190 con 0x7f4cb0103c40 2024-04-25T11:24:10.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.880+0000 7f4ca67fc640 1 -- 172.21.15.69:0/1227979694 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f4ca0004950 con 0x7f4cb0103c40 2024-04-25T11:24:10.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.880+0000 7f4ca67fc640 1 --2- 172.21.15.69:0/1227979694 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f4c9c0afb80 0x7f4c9c0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:10.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.880+0000 7f4cb5446640 1 --2- 172.21.15.69:0/1227979694 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f4c9c0afb80 0x7f4c9c0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:10.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.880+0000 7f4ca67fc640 1 -- 172.21.15.69:0/1227979694 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7f4ca00e0c30 con 0x7f4cb0103c40 2024-04-25T11:24:10.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.881+0000 7f4cb5446640 1 --2- 172.21.15.69:0/1227979694 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f4c9c0afb80 0x7f4c9c0b2090 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0103600 tx=0x7f4cac00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:10.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:10.884+0000 7f4ca67fc640 1 -- 172.21.15.69:0/1227979694 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4ca00a8e20 con 0x7f4cb0103c40 2024-04-25T11:24:11.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:11.061+0000 7f4cb76aa640 1 -- 172.21.15.69:0/1227979694 --> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4cb0075590 con 0x7f4c9c0afb80 2024-04-25T11:24:11.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:11.070+0000 7f4ca67fc640 1 -- 172.21.15.69:0/1227979694 <== mgr.14152 v2:172.21.15.69:6800/4220065409 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340316 (secure 0 0 0) 0x7f4cb0075590 con 0x7f4c9c0afb80 2024-04-25T11:24:11.067 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:24:11.069 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-04-25T11:24:11.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:11.074+0000 7f4cb76aa640 1 -- 172.21.15.69:0/1227979694 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f4c9c0afb80 msgr2=0x7f4c9c0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:11.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:11.074+0000 7f4cb76aa640 1 --2- 172.21.15.69:0/1227979694 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f4c9c0afb80 0x7f4c9c0b2090 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0103600 tx=0x7f4cac00b390 comp rx=0 tx=0).stop 2024-04-25T11:24:11.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:11.074+0000 7f4cb76aa640 1 -- 172.21.15.69:0/1227979694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4cb0103c40 msgr2=0x7f4cb0199540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:11.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:11.074+0000 7f4cb76aa640 1 --2- 172.21.15.69:0/1227979694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4cb0103c40 0x7f4cb0199540 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0009860 tx=0x7f4ca0009d20 comp rx=0 tx=0).stop 2024-04-25T11:24:11.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:11.074+0000 7f4cb76aa640 1 -- 172.21.15.69:0/1227979694 shutdown_connections 2024-04-25T11:24:11.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:11.074+0000 7f4cb76aa640 1 --2- 172.21.15.69:0/1227979694 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f4c9c0afb80 0x7f4c9c0b2090 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:11.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:11.074+0000 7f4cb76aa640 1 --2- 172.21.15.69:0/1227979694 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cb0108940 0x7f4cb01a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:11.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:11.074+0000 7f4cb76aa640 1 --2- 172.21.15.69:0/1227979694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4cb0103c40 0x7f4cb0199540 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:11.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:11.074+0000 7f4cb76aa640 1 --2- 172.21.15.69:0/1227979694 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4cb0102a00 0x7f4cb0199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:11.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:11.074+0000 7f4cb76aa640 1 -- 172.21.15.69:0/1227979694 >> 172.21.15.69:0/1227979694 conn(0x7f4cb00fdbf0 msgr2=0x7f4cb0104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:11.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:11.074+0000 7f4cb76aa640 1 -- 172.21.15.69:0/1227979694 shutdown_connections 2024-04-25T11:24:11.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:11.074+0000 7f4cb76aa640 1 -- 172.21.15.69:0/1227979694 wait complete. 2024-04-25T11:24:11.078 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:10 smithi069 ceph-mon[29188]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:24:11.078 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:10 smithi069 ceph-mon[29188]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:24:11.078 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:10 smithi069 ceph-mon[29188]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:24:11.078 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:10 smithi069 ceph-mon[29188]: pgmap v299: 129 pgs: 129 active+clean; 603 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 5.1 KiB/s wr, 11 op/s 2024-04-25T11:24:11.078 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:10 smithi069 ceph-mon[29188]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T11:24:11.078 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:10 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2448485462' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:24:11.078 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:10 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/1155999108' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:24:11.079 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":299,"stamp":"2024-04-25T11:24:09.786537+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":617971,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":101,"num_read_kb":88,"num_write":289,"num_write_kb":2228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":47,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":235,"ondisk_log_size":235,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365720,"kb_used_data":12384,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576104,"statfs":{"total":1151940427776,"available":1151565930496,"internally_reserved":0,"allocated":12681216,"data_stored":6989366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23104,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":25,"num_read_kb":24,"num_write":119,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003462"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.809037+0000","last_change":"2024-04-25T11:23:41.646759+0000","last_active":"2024-04-25T11:23:47.809037+0000","last_peered":"2024-04-25T11:23:47.809037+0000","last_clean":"2024-04-25T11:23:47.809037+0000","last_became_active":"2024-04-25T11:23:41.645882+0000","last_became_peered":"2024-04-25T11:23:41.645882+0000","last_unstale":"2024-04-25T11:23:47.809037+0000","last_undegraded":"2024-04-25T11:23:47.809037+0000","last_fullsized":"2024-04-25T11:23:47.809037+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:05.776414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807560+0000","last_change":"2024-04-25T11:23:41.648561+0000","last_active":"2024-04-25T11:23:47.807560+0000","last_peered":"2024-04-25T11:23:47.807560+0000","last_clean":"2024-04-25T11:23:47.807560+0000","last_became_active":"2024-04-25T11:23:41.647735+0000","last_became_peered":"2024-04-25T11:23:41.647735+0000","last_unstale":"2024-04-25T11:23:47.807560+0000","last_undegraded":"2024-04-25T11:23:47.807560+0000","last_fullsized":"2024-04-25T11:23:47.807560+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:20.303934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807099+0000","last_change":"2024-04-25T11:23:41.646656+0000","last_active":"2024-04-25T11:23:47.807099+0000","last_peered":"2024-04-25T11:23:47.807099+0000","last_clean":"2024-04-25T11:23:47.807099+0000","last_became_active":"2024-04-25T11:23:41.645487+0000","last_became_peered":"2024-04-25T11:23:41.645487+0000","last_unstale":"2024-04-25T11:23:47.807099+0000","last_undegraded":"2024-04-25T11:23:47.807099+0000","last_fullsized":"2024-04-25T11:23:47.807099+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:32.747478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.809763+0000","last_change":"2024-04-25T11:23:41.651994+0000","last_active":"2024-04-25T11:23:47.809763+0000","last_peered":"2024-04-25T11:23:47.809763+0000","last_clean":"2024-04-25T11:23:47.809763+0000","last_became_active":"2024-04-25T11:23:41.651640+0000","last_became_peered":"2024-04-25T11:23:41.651640+0000","last_unstale":"2024-04-25T11:23:47.809763+0000","last_undegraded":"2024-04-25T11:23:47.809763+0000","last_fullsized":"2024-04-25T11:23:47.809763+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:09.376398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808522+0000","last_change":"2024-04-25T11:23:41.650954+0000","last_active":"2024-04-25T11:23:47.808522+0000","last_peered":"2024-04-25T11:23:47.808522+0000","last_clean":"2024-04-25T11:23:47.808522+0000","last_became_active":"2024-04-25T11:23:41.650553+0000","last_became_peered":"2024-04-25T11:23:41.650553+0000","last_unstale":"2024-04-25T11:23:47.808522+0000","last_undegraded":"2024-04-25T11:23:47.808522+0000","last_fullsized":"2024-04-25T11:23:47.808522+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:52:34.348370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808283+0000","last_change":"2024-04-25T11:23:41.650770+0000","last_active":"2024-04-25T11:23:47.808283+0000","last_peered":"2024-04-25T11:23:47.808283+0000","last_clean":"2024-04-25T11:23:47.808283+0000","last_became_active":"2024-04-25T11:23:41.649794+0000","last_became_peered":"2024-04-25T11:23:41.649794+0000","last_unstale":"2024-04-25T11:23:47.808283+0000","last_undegraded":"2024-04-25T11:23:47.808283+0000","last_fullsized":"2024-04-25T11:23:47.808283+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:04.032001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807801+0000","last_change":"2024-04-25T11:23:41.643753+0000","last_active":"2024-04-25T11:23:47.807801+0000","last_peered":"2024-04-25T11:23:47.807801+0000","last_clean":"2024-04-25T11:23:47.807801+0000","last_became_active":"2024-04-25T11:23:41.642854+0000","last_became_peered":"2024-04-25T11:23:41.642854+0000","last_unstale":"2024-04-25T11:23:47.807801+0000","last_undegraded":"2024-04-25T11:23:47.807801+0000","last_fullsized":"2024-04-25T11:23:47.807801+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:40.868312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.668451+0000","last_change":"2024-04-25T11:23:41.646501+0000","last_active":"2024-04-25T11:23:47.668451+0000","last_peered":"2024-04-25T11:23:47.668451+0000","last_clean":"2024-04-25T11:23:47.668451+0000","last_became_active":"2024-04-25T11:23:41.646178+0000","last_became_peered":"2024-04-25T11:23:41.646178+0000","last_unstale":"2024-04-25T11:23:47.668451+0000","last_undegraded":"2024-04-25T11:23:47.668451+0000","last_fullsized":"2024-04-25T11:23:47.668451+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:34.699979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.667783+0000","last_change":"2024-04-25T11:23:41.647025+0000","last_active":"2024-04-25T11:23:47.667783+0000","last_peered":"2024-04-25T11:23:47.667783+0000","last_clean":"2024-04-25T11:23:47.667783+0000","last_became_active":"2024-04-25T11:23:41.646645+0000","last_became_peered":"2024-04-25T11:23:41.646645+0000","last_unstale":"2024-04-25T11:23:47.667783+0000","last_undegraded":"2024-04-25T11:23:47.667783+0000","last_fullsized":"2024-04-25T11:23:47.667783+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:24:57.108535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808683+0000","last_change":"2024-04-25T11:23:41.660957+0000","last_active":"2024-04-25T11:23:47.808683+0000","last_peered":"2024-04-25T11:23:47.808683+0000","last_clean":"2024-04-25T11:23:47.808683+0000","last_became_active":"2024-04-25T11:23:41.660529+0000","last_became_peered":"2024-04-25T11:23:41.660529+0000","last_unstale":"2024-04-25T11:23:47.808683+0000","last_undegraded":"2024-04-25T11:23:47.808683+0000","last_fullsized":"2024-04-25T11:23:47.808683+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:51.113991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.809101+0000","last_change":"2024-04-25T11:23:41.648777+0000","last_active":"2024-04-25T11:23:47.809101+0000","last_peered":"2024-04-25T11:23:47.809101+0000","last_clean":"2024-04-25T11:23:47.809101+0000","last_became_active":"2024-04-25T11:23:41.647282+0000","last_became_peered":"2024-04-25T11:23:41.647282+0000","last_unstale":"2024-04-25T11:23:47.809101+0000","last_undegraded":"2024-04-25T11:23:47.809101+0000","last_fullsized":"2024-04-25T11:23:47.809101+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:14.396228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808728+0000","last_change":"2024-04-25T11:23:41.648855+0000","last_active":"2024-04-25T11:23:47.808728+0000","last_peered":"2024-04-25T11:23:47.808728+0000","last_clean":"2024-04-25T11:23:47.808728+0000","last_became_active":"2024-04-25T11:23:41.648527+0000","last_became_peered":"2024-04-25T11:23:41.648527+0000","last_unstale":"2024-04-25T11:23:47.808728+0000","last_undegraded":"2024-04-25T11:23:47.808728+0000","last_fullsized":"2024-04-25T11:23:47.808728+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:13.569916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807392+0000","last_change":"2024-04-25T11:23:41.646377+0000","last_active":"2024-04-25T11:23:47.807392+0000","last_peered":"2024-04-25T11:23:47.807392+0000","last_clean":"2024-04-25T11:23:47.807392+0000","last_became_active":"2024-04-25T11:23:41.646063+0000","last_became_peered":"2024-04-25T11:23:41.646063+0000","last_unstale":"2024-04-25T11:23:47.807392+0000","last_undegraded":"2024-04-25T11:23:47.807392+0000","last_fullsized":"2024-04-25T11:23:47.807392+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:50.365317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.806904+0000","last_change":"2024-04-25T11:23:41.648147+0000","last_active":"2024-04-25T11:23:47.806904+0000","last_peered":"2024-04-25T11:23:47.806904+0000","last_clean":"2024-04-25T11:23:47.806904+0000","last_became_active":"2024-04-25T11:23:41.647771+0000","last_became_peered":"2024-04-25T11:23:41.647771+0000","last_unstale":"2024-04-25T11:23:47.806904+0000","last_undegraded":"2024-04-25T11:23:47.806904+0000","last_fullsized":"2024-04-25T11:23:47.806904+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:20.157971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807368+0000","last_change":"2024-04-25T11:23:41.652784+0000","last_active":"2024-04-25T11:23:47.807368+0000","last_peered":"2024-04-25T11:23:47.807368+0000","last_clean":"2024-04-25T11:23:47.807368+0000","last_became_active":"2024-04-25T11:23:41.652434+0000","last_became_peered":"2024-04-25T11:23:41.652434+0000","last_unstale":"2024-04-25T11:23:47.807368+0000","last_undegraded":"2024-04-25T11:23:47.807368+0000","last_fullsized":"2024-04-25T11:23:47.807368+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:48.468138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.667177+0000","last_change":"2024-04-25T11:23:41.648218+0000","last_active":"2024-04-25T11:23:47.667177+0000","last_peered":"2024-04-25T11:23:47.667177+0000","last_clean":"2024-04-25T11:23:47.667177+0000","last_became_active":"2024-04-25T11:23:41.647871+0000","last_became_peered":"2024-04-25T11:23:41.647871+0000","last_unstale":"2024-04-25T11:23:47.667177+0000","last_undegraded":"2024-04-25T11:23:47.667177+0000","last_fullsized":"2024-04-25T11:23:47.667177+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:49.919090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808862+0000","last_change":"2024-04-25T11:23:41.647087+0000","last_active":"2024-04-25T11:23:47.808862+0000","last_peered":"2024-04-25T11:23:47.808862+0000","last_clean":"2024-04-25T11:23:47.808862+0000","last_became_active":"2024-04-25T11:23:41.646359+0000","last_became_peered":"2024-04-25T11:23:41.646359+0000","last_unstale":"2024-04-25T11:23:47.808862+0000","last_undegraded":"2024-04-25T11:23:47.808862+0000","last_fullsized":"2024-04-25T11:23:47.808862+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:32.129538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808431+0000","last_change":"2024-04-25T11:23:41.644143+0000","last_active":"2024-04-25T11:23:47.808431+0000","last_peered":"2024-04-25T11:23:47.808431+0000","last_clean":"2024-04-25T11:23:47.808431+0000","last_became_active":"2024-04-25T11:23:41.643738+0000","last_became_peered":"2024-04-25T11:23:41.643738+0000","last_unstale":"2024-04-25T11:23:47.808431+0000","last_undegraded":"2024-04-25T11:23:47.808431+0000","last_fullsized":"2024-04-25T11:23:47.808431+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:19.786933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.809443+0000","last_change":"2024-04-25T11:23:41.636421+0000","last_active":"2024-04-25T11:23:47.809443+0000","last_peered":"2024-04-25T11:23:47.809443+0000","last_clean":"2024-04-25T11:23:47.809443+0000","last_became_active":"2024-04-25T11:23:41.635392+0000","last_became_peered":"2024-04-25T11:23:41.635392+0000","last_unstale":"2024-04-25T11:23:47.809443+0000","last_undegraded":"2024-04-25T11:23:47.809443+0000","last_fullsized":"2024-04-25T11:23:47.809443+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:59.859340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.809291+0000","last_change":"2024-04-25T11:23:45.675875+0000","last_active":"2024-04-25T11:23:47.809291+0000","last_peered":"2024-04-25T11:23:47.809291+0000","last_clean":"2024-04-25T11:23:47.809291+0000","last_became_active":"2024-04-25T11:23:45.675405+0000","last_became_peered":"2024-04-25T11:23:45.675405+0000","last_unstale":"2024-04-25T11:23:47.809291+0000","last_undegraded":"2024-04-25T11:23:47.809291+0000","last_fullsized":"2024-04-25T11:23:47.809291+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:06.732509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":38,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:24:08.066349+0000","last_change":"2024-04-25T11:23:40.630862+0000","last_active":"2024-04-25T11:24:08.066349+0000","last_peered":"2024-04-25T11:24:08.066349+0000","last_clean":"2024-04-25T11:24:08.066349+0000","last_became_active":"2024-04-25T11:23:40.630498+0000","last_became_peered":"2024-04-25T11:23:40.630498+0000","last_unstale":"2024-04-25T11:24:08.066349+0000","last_undegraded":"2024-04-25T11:24:08.066349+0000","last_fullsized":"2024-04-25T11:24:08.066349+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:07:44.028640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.668492+0000","last_change":"2024-04-25T11:23:41.639879+0000","last_active":"2024-04-25T11:23:47.668492+0000","last_peered":"2024-04-25T11:23:47.668492+0000","last_clean":"2024-04-25T11:23:47.668492+0000","last_became_active":"2024-04-25T11:23:41.639359+0000","last_became_peered":"2024-04-25T11:23:41.639359+0000","last_unstale":"2024-04-25T11:23:47.668492+0000","last_undegraded":"2024-04-25T11:23:47.668492+0000","last_fullsized":"2024-04-25T11:23:47.668492+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:47.525089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.809141+0000","last_change":"2024-04-25T11:23:45.672219+0000","last_active":"2024-04-25T11:23:47.809141+0000","last_peered":"2024-04-25T11:23:47.809141+0000","last_clean":"2024-04-25T11:23:47.809141+0000","last_became_active":"2024-04-25T11:23:45.671788+0000","last_became_peered":"2024-04-25T11:23:45.671788+0000","last_unstale":"2024-04-25T11:23:47.809141+0000","last_undegraded":"2024-04-25T11:23:47.809141+0000","last_fullsized":"2024-04-25T11:23:47.809141+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:36.228061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.668792+0000","last_change":"2024-04-25T11:23:40.643435+0000","last_active":"2024-04-25T11:23:47.668792+0000","last_peered":"2024-04-25T11:23:47.668792+0000","last_clean":"2024-04-25T11:23:47.668792+0000","last_became_active":"2024-04-25T11:23:40.642990+0000","last_became_peered":"2024-04-25T11:23:40.642990+0000","last_unstale":"2024-04-25T11:23:47.668792+0000","last_undegraded":"2024-04-25T11:23:47.668792+0000","last_fullsized":"2024-04-25T11:23:47.668792+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:42.723486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807765+0000","last_change":"2024-04-25T11:23:41.645977+0000","last_active":"2024-04-25T11:23:47.807765+0000","last_peered":"2024-04-25T11:23:47.807765+0000","last_clean":"2024-04-25T11:23:47.807765+0000","last_became_active":"2024-04-25T11:23:41.645562+0000","last_became_peered":"2024-04-25T11:23:41.645562+0000","last_unstale":"2024-04-25T11:23:47.807765+0000","last_undegraded":"2024-04-25T11:23:47.807765+0000","last_fullsized":"2024-04-25T11:23:47.807765+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:04.027324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807920+0000","last_change":"2024-04-25T11:23:45.672843+0000","last_active":"2024-04-25T11:23:47.807920+0000","last_peered":"2024-04-25T11:23:47.807920+0000","last_clean":"2024-04-25T11:23:47.807920+0000","last_became_active":"2024-04-25T11:23:45.672429+0000","last_became_peered":"2024-04-25T11:23:45.672429+0000","last_unstale":"2024-04-25T11:23:47.807920+0000","last_undegraded":"2024-04-25T11:23:47.807920+0000","last_fullsized":"2024-04-25T11:23:47.807920+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:48.818624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"74'2","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808083+0000","last_change":"2024-04-25T11:23:40.634393+0000","last_active":"2024-04-25T11:23:47.808083+0000","last_peered":"2024-04-25T11:23:47.808083+0000","last_clean":"2024-04-25T11:23:47.808083+0000","last_became_active":"2024-04-25T11:23:40.633518+0000","last_became_peered":"2024-04-25T11:23:40.633518+0000","last_unstale":"2024-04-25T11:23:47.808083+0000","last_undegraded":"2024-04-25T11:23:47.808083+0000","last_fullsized":"2024-04-25T11:23:47.808083+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:59.623135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.667422+0000","last_change":"2024-04-25T11:23:41.656152+0000","last_active":"2024-04-25T11:23:47.667422+0000","last_peered":"2024-04-25T11:23:47.667422+0000","last_clean":"2024-04-25T11:23:47.667422+0000","last_became_active":"2024-04-25T11:23:41.655344+0000","last_became_peered":"2024-04-25T11:23:41.655344+0000","last_unstale":"2024-04-25T11:23:47.667422+0000","last_undegraded":"2024-04-25T11:23:47.667422+0000","last_fullsized":"2024-04-25T11:23:47.667422+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:07.649534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.809772+0000","last_change":"2024-04-25T11:23:45.668602+0000","last_active":"2024-04-25T11:23:47.809772+0000","last_peered":"2024-04-25T11:23:47.809772+0000","last_clean":"2024-04-25T11:23:47.809772+0000","last_became_active":"2024-04-25T11:23:45.668143+0000","last_became_peered":"2024-04-25T11:23:45.668143+0000","last_unstale":"2024-04-25T11:23:47.809772+0000","last_undegraded":"2024-04-25T11:23:47.809772+0000","last_fullsized":"2024-04-25T11:23:47.809772+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:13.463604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"74'2","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807985+0000","last_change":"2024-04-25T11:23:40.639644+0000","last_active":"2024-04-25T11:23:47.807985+0000","last_peered":"2024-04-25T11:23:47.807985+0000","last_clean":"2024-04-25T11:23:47.807985+0000","last_became_active":"2024-04-25T11:23:40.639295+0000","last_became_peered":"2024-04-25T11:23:40.639295+0000","last_unstale":"2024-04-25T11:23:47.807985+0000","last_undegraded":"2024-04-25T11:23:47.807985+0000","last_fullsized":"2024-04-25T11:23:47.807985+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:43.849692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.667803+0000","last_change":"2024-04-25T11:23:41.657112+0000","last_active":"2024-04-25T11:23:47.667803+0000","last_peered":"2024-04-25T11:23:47.667803+0000","last_clean":"2024-04-25T11:23:47.667803+0000","last_became_active":"2024-04-25T11:23:41.656623+0000","last_became_peered":"2024-04-25T11:23:41.656623+0000","last_unstale":"2024-04-25T11:23:47.667803+0000","last_undegraded":"2024-04-25T11:23:47.667803+0000","last_fullsized":"2024-04-25T11:23:47.667803+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:41.644825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.809246+0000","last_change":"2024-04-25T11:23:45.673373+0000","last_active":"2024-04-25T11:23:47.809246+0000","last_peered":"2024-04-25T11:23:47.809246+0000","last_clean":"2024-04-25T11:23:47.809246+0000","last_became_active":"2024-04-25T11:23:45.672928+0000","last_became_peered":"2024-04-25T11:23:45.672928+0000","last_unstale":"2024-04-25T11:23:47.809246+0000","last_undegraded":"2024-04-25T11:23:47.809246+0000","last_fullsized":"2024-04-25T11:23:47.809246+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:10.553189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"80'5","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:24:08.059490+0000","last_change":"2024-04-25T11:23:40.635821+0000","last_active":"2024-04-25T11:24:08.059490+0000","last_peered":"2024-04-25T11:24:08.059490+0000","last_clean":"2024-04-25T11:24:08.059490+0000","last_became_active":"2024-04-25T11:23:40.635438+0000","last_became_peered":"2024-04-25T11:23:40.635438+0000","last_unstale":"2024-04-25T11:24:08.059490+0000","last_undegraded":"2024-04-25T11:24:08.059490+0000","last_fullsized":"2024-04-25T11:24:08.059490+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:55.750408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808248+0000","last_change":"2024-04-25T11:23:41.648771+0000","last_active":"2024-04-25T11:23:47.808248+0000","last_peered":"2024-04-25T11:23:47.808248+0000","last_clean":"2024-04-25T11:23:47.808248+0000","last_became_active":"2024-04-25T11:23:41.648059+0000","last_became_peered":"2024-04-25T11:23:41.648059+0000","last_unstale":"2024-04-25T11:23:47.808248+0000","last_undegraded":"2024-04-25T11:23:47.808248+0000","last_fullsized":"2024-04-25T11:23:47.808248+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:27.873310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.809609+0000","last_change":"2024-04-25T11:23:45.672676+0000","last_active":"2024-04-25T11:23:47.809609+0000","last_peered":"2024-04-25T11:23:47.809609+0000","last_clean":"2024-04-25T11:23:47.809609+0000","last_became_active":"2024-04-25T11:23:45.672229+0000","last_became_peered":"2024-04-25T11:23:45.672229+0000","last_unstale":"2024-04-25T11:23:47.809609+0000","last_undegraded":"2024-04-25T11:23:47.809609+0000","last_fullsized":"2024-04-25T11:23:47.809609+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:16.648809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:24:07.916201+0000","last_change":"2024-04-25T11:23:40.635988+0000","last_active":"2024-04-25T11:24:07.916201+0000","last_peered":"2024-04-25T11:24:07.916201+0000","last_clean":"2024-04-25T11:24:07.916201+0000","last_became_active":"2024-04-25T11:23:40.635366+0000","last_became_peered":"2024-04-25T11:23:40.635366+0000","last_unstale":"2024-04-25T11:24:07.916201+0000","last_undegraded":"2024-04-25T11:24:07.916201+0000","last_fullsized":"2024-04-25T11:24:07.916201+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:56.336232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.667736+0000","last_change":"2024-04-25T11:23:41.656031+0000","last_active":"2024-04-25T11:23:47.667736+0000","last_peered":"2024-04-25T11:23:47.667736+0000","last_clean":"2024-04-25T11:23:47.667736+0000","last_became_active":"2024-04-25T11:23:41.655656+0000","last_became_peered":"2024-04-25T11:23:41.655656+0000","last_unstale":"2024-04-25T11:23:47.667736+0000","last_undegraded":"2024-04-25T11:23:47.667736+0000","last_fullsized":"2024-04-25T11:23:47.667736+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:37.991435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808673+0000","last_change":"2024-04-25T11:23:45.675380+0000","last_active":"2024-04-25T11:23:47.808673+0000","last_peered":"2024-04-25T11:23:47.808673+0000","last_clean":"2024-04-25T11:23:47.808673+0000","last_became_active":"2024-04-25T11:23:45.674914+0000","last_became_peered":"2024-04-25T11:23:45.674914+0000","last_unstale":"2024-04-25T11:23:47.808673+0000","last_undegraded":"2024-04-25T11:23:47.808673+0000","last_fullsized":"2024-04-25T11:23:47.808673+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:26.231510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:24:06.903481+0000","last_change":"2024-04-25T11:23:40.631581+0000","last_active":"2024-04-25T11:24:06.903481+0000","last_peered":"2024-04-25T11:24:06.903481+0000","last_clean":"2024-04-25T11:24:06.903481+0000","last_became_active":"2024-04-25T11:23:40.631169+0000","last_became_peered":"2024-04-25T11:23:40.631169+0000","last_unstale":"2024-04-25T11:24:06.903481+0000","last_undegraded":"2024-04-25T11:24:06.903481+0000","last_fullsized":"2024-04-25T11:24:06.903481+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:15.737402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.668234+0000","last_change":"2024-04-25T11:23:41.633740+0000","last_active":"2024-04-25T11:23:47.668234+0000","last_peered":"2024-04-25T11:23:47.668234+0000","last_clean":"2024-04-25T11:23:47.668234+0000","last_became_active":"2024-04-25T11:23:41.633445+0000","last_became_peered":"2024-04-25T11:23:41.633445+0000","last_unstale":"2024-04-25T11:23:47.668234+0000","last_undegraded":"2024-04-25T11:23:47.668234+0000","last_fullsized":"2024-04-25T11:23:47.668234+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:46.103646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807455+0000","last_change":"2024-04-25T11:23:45.678899+0000","last_active":"2024-04-25T11:23:47.807455+0000","last_peered":"2024-04-25T11:23:47.807455+0000","last_clean":"2024-04-25T11:23:47.807455+0000","last_became_active":"2024-04-25T11:23:45.675305+0000","last_became_peered":"2024-04-25T11:23:45.675305+0000","last_unstale":"2024-04-25T11:23:47.807455+0000","last_undegraded":"2024-04-25T11:23:47.807455+0000","last_fullsized":"2024-04-25T11:23:47.807455+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:49.456551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"80'2","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:24:05.895935+0000","last_change":"2024-04-25T11:23:40.638090+0000","last_active":"2024-04-25T11:24:05.895935+0000","last_peered":"2024-04-25T11:24:05.895935+0000","last_clean":"2024-04-25T11:24:05.895935+0000","last_became_active":"2024-04-25T11:23:40.637707+0000","last_became_peered":"2024-04-25T11:23:40.637707+0000","last_unstale":"2024-04-25T11:24:05.895935+0000","last_undegraded":"2024-04-25T11:24:05.895935+0000","last_fullsized":"2024-04-25T11:24:05.895935+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:38:43.413273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808127+0000","last_change":"2024-04-25T11:23:41.641580+0000","last_active":"2024-04-25T11:23:47.808127+0000","last_peered":"2024-04-25T11:23:47.808127+0000","last_clean":"2024-04-25T11:23:47.808127+0000","last_became_active":"2024-04-25T11:23:41.641199+0000","last_became_peered":"2024-04-25T11:23:41.641199+0000","last_unstale":"2024-04-25T11:23:47.808127+0000","last_undegraded":"2024-04-25T11:23:47.808127+0000","last_fullsized":"2024-04-25T11:23:47.808127+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:34:10.471476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.668708+0000","last_change":"2024-04-25T11:23:45.675573+0000","last_active":"2024-04-25T11:23:47.668708+0000","last_peered":"2024-04-25T11:23:47.668708+0000","last_clean":"2024-04-25T11:23:47.668708+0000","last_became_active":"2024-04-25T11:23:45.675197+0000","last_became_peered":"2024-04-25T11:23:45.675197+0000","last_unstale":"2024-04-25T11:23:47.668708+0000","last_undegraded":"2024-04-25T11:23:47.668708+0000","last_fullsized":"2024-04-25T11:23:47.668708+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:39.740949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:24:08.058947+0000","last_change":"2024-04-25T11:23:40.635020+0000","last_active":"2024-04-25T11:24:08.058947+0000","last_peered":"2024-04-25T11:24:08.058947+0000","last_clean":"2024-04-25T11:24:08.058947+0000","last_became_active":"2024-04-25T11:23:40.634549+0000","last_became_peered":"2024-04-25T11:23:40.634549+0000","last_unstale":"2024-04-25T11:24:08.058947+0000","last_undegraded":"2024-04-25T11:24:08.058947+0000","last_fullsized":"2024-04-25T11:24:08.058947+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:15:16.289370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.667738+0000","last_change":"2024-04-25T11:23:41.654903+0000","last_active":"2024-04-25T11:23:47.667738+0000","last_peered":"2024-04-25T11:23:47.667738+0000","last_clean":"2024-04-25T11:23:47.667738+0000","last_became_active":"2024-04-25T11:23:41.654564+0000","last_became_peered":"2024-04-25T11:23:41.654564+0000","last_unstale":"2024-04-25T11:23:47.667738+0000","last_undegraded":"2024-04-25T11:23:47.667738+0000","last_fullsized":"2024-04-25T11:23:47.667738+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:29.025240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808068+0000","last_change":"2024-04-25T11:23:45.677683+0000","last_active":"2024-04-25T11:23:47.808068+0000","last_peered":"2024-04-25T11:23:47.808068+0000","last_clean":"2024-04-25T11:23:47.808068+0000","last_became_active":"2024-04-25T11:23:45.677286+0000","last_became_peered":"2024-04-25T11:23:45.677286+0000","last_unstale":"2024-04-25T11:23:47.808068+0000","last_undegraded":"2024-04-25T11:23:47.808068+0000","last_fullsized":"2024-04-25T11:23:47.808068+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:43.236060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808590+0000","last_change":"2024-04-25T11:23:40.640844+0000","last_active":"2024-04-25T11:23:47.808590+0000","last_peered":"2024-04-25T11:23:47.808590+0000","last_clean":"2024-04-25T11:23:47.808590+0000","last_became_active":"2024-04-25T11:23:40.640434+0000","last_became_peered":"2024-04-25T11:23:40.640434+0000","last_unstale":"2024-04-25T11:23:47.808590+0000","last_undegraded":"2024-04-25T11:23:47.808590+0000","last_fullsized":"2024-04-25T11:23:47.808590+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:16.701418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808035+0000","last_change":"2024-04-25T11:23:41.651962+0000","last_active":"2024-04-25T11:23:47.808035+0000","last_peered":"2024-04-25T11:23:47.808035+0000","last_clean":"2024-04-25T11:23:47.808035+0000","last_became_active":"2024-04-25T11:23:41.650870+0000","last_became_peered":"2024-04-25T11:23:41.650870+0000","last_unstale":"2024-04-25T11:23:47.808035+0000","last_undegraded":"2024-04-25T11:23:47.808035+0000","last_fullsized":"2024-04-25T11:23:47.808035+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:45.623722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808282+0000","last_change":"2024-04-25T11:23:45.668287+0000","last_active":"2024-04-25T11:23:47.808282+0000","last_peered":"2024-04-25T11:23:47.808282+0000","last_clean":"2024-04-25T11:23:47.808282+0000","last_became_active":"2024-04-25T11:23:45.667727+0000","last_became_peered":"2024-04-25T11:23:45.667727+0000","last_unstale":"2024-04-25T11:23:47.808282+0000","last_undegraded":"2024-04-25T11:23:47.808282+0000","last_fullsized":"2024-04-25T11:23:47.808282+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:44:07.699179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:24:07.912927+0000","last_change":"2024-04-25T11:23:40.642112+0000","last_active":"2024-04-25T11:24:07.912927+0000","last_peered":"2024-04-25T11:24:07.912927+0000","last_clean":"2024-04-25T11:24:07.912927+0000","last_became_active":"2024-04-25T11:23:40.641745+0000","last_became_peered":"2024-04-25T11:23:40.641745+0000","last_unstale":"2024-04-25T11:24:07.912927+0000","last_undegraded":"2024-04-25T11:24:07.912927+0000","last_fullsized":"2024-04-25T11:24:07.912927+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:14.899432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.667580+0000","last_change":"2024-04-25T11:23:41.656460+0000","last_active":"2024-04-25T11:23:47.667580+0000","last_peered":"2024-04-25T11:23:47.667580+0000","last_clean":"2024-04-25T11:23:47.667580+0000","last_became_active":"2024-04-25T11:23:41.655800+0000","last_became_peered":"2024-04-25T11:23:41.655800+0000","last_unstale":"2024-04-25T11:23:47.667580+0000","last_undegraded":"2024-04-25T11:23:47.667580+0000","last_fullsized":"2024-04-25T11:23:47.667580+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:42.505142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808104+0000","last_change":"2024-04-25T11:23:45.671650+0000","last_active":"2024-04-25T11:23:47.808104+0000","last_peered":"2024-04-25T11:23:47.808104+0000","last_clean":"2024-04-25T11:23:47.808104+0000","last_became_active":"2024-04-25T11:23:45.671254+0000","last_became_peered":"2024-04-25T11:23:45.671254+0000","last_unstale":"2024-04-25T11:23:47.808104+0000","last_undegraded":"2024-04-25T11:23:47.808104+0000","last_fullsized":"2024-04-25T11:23:47.808104+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:29.404898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"80'21","reported_seq":50,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:24:08.642197+0000","last_change":"2024-04-25T11:23:40.633712+0000","last_active":"2024-04-25T11:24:08.642197+0000","last_peered":"2024-04-25T11:24:08.642197+0000","last_clean":"2024-04-25T11:24:08.642197+0000","last_became_active":"2024-04-25T11:23:40.633274+0000","last_became_peered":"2024-04-25T11:23:40.633274+0000","last_unstale":"2024-04-25T11:24:08.642197+0000","last_undegraded":"2024-04-25T11:24:08.642197+0000","last_fullsized":"2024-04-25T11:24:08.642197+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:47.576213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23141,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":6,"num_write":23,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807871+0000","last_change":"2024-04-25T11:23:41.649007+0000","last_active":"2024-04-25T11:23:47.807871+0000","last_peered":"2024-04-25T11:23:47.807871+0000","last_clean":"2024-04-25T11:23:47.807871+0000","last_became_active":"2024-04-25T11:23:41.647778+0000","last_became_peered":"2024-04-25T11:23:41.647778+0000","last_unstale":"2024-04-25T11:23:47.807871+0000","last_undegraded":"2024-04-25T11:23:47.807871+0000","last_fullsized":"2024-04-25T11:23:47.807871+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:33.097482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.668274+0000","last_change":"2024-04-25T11:23:45.671645+0000","last_active":"2024-04-25T11:23:47.668274+0000","last_peered":"2024-04-25T11:23:47.668274+0000","last_clean":"2024-04-25T11:23:47.668274+0000","last_became_active":"2024-04-25T11:23:45.670005+0000","last_became_peered":"2024-04-25T11:23:45.670005+0000","last_unstale":"2024-04-25T11:23:47.668274+0000","last_undegraded":"2024-04-25T11:23:47.668274+0000","last_fullsized":"2024-04-25T11:23:47.668274+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:36.047222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"80'7","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:24:06.902822+0000","last_change":"2024-04-25T11:23:40.639922+0000","last_active":"2024-04-25T11:24:06.902822+0000","last_peered":"2024-04-25T11:24:06.902822+0000","last_clean":"2024-04-25T11:24:06.902822+0000","last_became_active":"2024-04-25T11:23:40.639544+0000","last_became_peered":"2024-04-25T11:23:40.639544+0000","last_unstale":"2024-04-25T11:24:06.902822+0000","last_undegraded":"2024-04-25T11:24:06.902822+0000","last_fullsized":"2024-04-25T11:24:06.902822+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:06.281281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807072+0000","last_change":"2024-04-25T11:23:41.629257+0000","last_active":"2024-04-25T11:23:47.807072+0000","last_peered":"2024-04-25T11:23:47.807072+0000","last_clean":"2024-04-25T11:23:47.807072+0000","last_became_active":"2024-04-25T11:23:41.628933+0000","last_became_peered":"2024-04-25T11:23:41.628933+0000","last_unstale":"2024-04-25T11:23:47.807072+0000","last_undegraded":"2024-04-25T11:23:47.807072+0000","last_fullsized":"2024-04-25T11:23:47.807072+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:04.031468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808813+0000","last_change":"2024-04-25T11:23:45.680460+0000","last_active":"2024-04-25T11:23:47.808813+0000","last_peered":"2024-04-25T11:23:47.808813+0000","last_clean":"2024-04-25T11:23:47.808813+0000","last_became_active":"2024-04-25T11:23:45.679963+0000","last_became_peered":"2024-04-25T11:23:45.679963+0000","last_unstale":"2024-04-25T11:23:47.808813+0000","last_undegraded":"2024-04-25T11:23:47.808813+0000","last_fullsized":"2024-04-25T11:23:47.808813+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:11.199855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"80'4","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:24:05.896179+0000","last_change":"2024-04-25T11:23:40.644026+0000","last_active":"2024-04-25T11:24:05.896179+0000","last_peered":"2024-04-25T11:24:05.896179+0000","last_clean":"2024-04-25T11:24:05.896179+0000","last_became_active":"2024-04-25T11:23:40.643665+0000","last_became_peered":"2024-04-25T11:23:40.643665+0000","last_unstale":"2024-04-25T11:24:05.896179+0000","last_undegraded":"2024-04-25T11:24:05.896179+0000","last_fullsized":"2024-04-25T11:24:05.896179+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:32.950387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808598+0000","last_change":"2024-04-25T11:23:41.642285+0000","last_active":"2024-04-25T11:23:47.808598+0000","last_peered":"2024-04-25T11:23:47.808598+0000","last_clean":"2024-04-25T11:23:47.808598+0000","last_became_active":"2024-04-25T11:23:41.641712+0000","last_became_peered":"2024-04-25T11:23:41.641712+0000","last_unstale":"2024-04-25T11:23:47.808598+0000","last_undegraded":"2024-04-25T11:23:47.808598+0000","last_fullsized":"2024-04-25T11:23:47.808598+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:22.813546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808373+0000","last_change":"2024-04-25T11:23:45.677632+0000","last_active":"2024-04-25T11:23:47.808373+0000","last_peered":"2024-04-25T11:23:47.808373+0000","last_clean":"2024-04-25T11:23:47.808373+0000","last_became_active":"2024-04-25T11:23:45.677244+0000","last_became_peered":"2024-04-25T11:23:45.677244+0000","last_unstale":"2024-04-25T11:23:47.808373+0000","last_undegraded":"2024-04-25T11:23:47.808373+0000","last_fullsized":"2024-04-25T11:23:47.808373+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:03.982435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"80'5","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:24:08.059729+0000","last_change":"2024-04-25T11:23:40.634661+0000","last_active":"2024-04-25T11:24:08.059729+0000","last_peered":"2024-04-25T11:24:08.059729+0000","last_clean":"2024-04-25T11:24:08.059729+0000","last_became_active":"2024-04-25T11:23:40.634211+0000","last_became_peered":"2024-04-25T11:23:40.634211+0000","last_unstale":"2024-04-25T11:24:08.059729+0000","last_undegraded":"2024-04-25T11:24:08.059729+0000","last_fullsized":"2024-04-25T11:24:08.059729+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:54.928339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808206+0000","last_change":"2024-04-25T11:23:41.644147+0000","last_active":"2024-04-25T11:23:47.808206+0000","last_peered":"2024-04-25T11:23:47.808206+0000","last_clean":"2024-04-25T11:23:47.808206+0000","last_became_active":"2024-04-25T11:23:41.643103+0000","last_became_peered":"2024-04-25T11:23:41.643103+0000","last_unstale":"2024-04-25T11:23:47.808206+0000","last_undegraded":"2024-04-25T11:23:47.808206+0000","last_fullsized":"2024-04-25T11:23:47.808206+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:00.956551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807838+0000","last_change":"2024-04-25T11:23:45.671504+0000","last_active":"2024-04-25T11:23:47.807838+0000","last_peered":"2024-04-25T11:23:47.807838+0000","last_clean":"2024-04-25T11:23:47.807838+0000","last_became_active":"2024-04-25T11:23:45.671148+0000","last_became_peered":"2024-04-25T11:23:45.671148+0000","last_unstale":"2024-04-25T11:23:47.807838+0000","last_undegraded":"2024-04-25T11:23:47.807838+0000","last_fullsized":"2024-04-25T11:23:47.807838+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:21.795839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"80'10","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:24:05.892611+0000","last_change":"2024-04-25T11:23:40.642021+0000","last_active":"2024-04-25T11:24:05.892611+0000","last_peered":"2024-04-25T11:24:05.892611+0000","last_clean":"2024-04-25T11:24:05.892611+0000","last_became_active":"2024-04-25T11:23:40.641639+0000","last_became_peered":"2024-04-25T11:23:40.641639+0000","last_unstale":"2024-04-25T11:24:05.892611+0000","last_undegraded":"2024-04-25T11:24:05.892611+0000","last_fullsized":"2024-04-25T11:24:05.892611+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:27.101032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.667187+0000","last_change":"2024-04-25T11:23:41.658283+0000","last_active":"2024-04-25T11:23:47.667187+0000","last_peered":"2024-04-25T11:23:47.667187+0000","last_clean":"2024-04-25T11:23:47.667187+0000","last_became_active":"2024-04-25T11:23:41.657898+0000","last_became_peered":"2024-04-25T11:23:41.657898+0000","last_unstale":"2024-04-25T11:23:47.667187+0000","last_undegraded":"2024-04-25T11:23:47.667187+0000","last_fullsized":"2024-04-25T11:23:47.667187+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:52:55.941232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.667374+0000","last_change":"2024-04-25T11:23:45.677362+0000","last_active":"2024-04-25T11:23:47.667374+0000","last_peered":"2024-04-25T11:23:47.667374+0000","last_clean":"2024-04-25T11:23:47.667374+0000","last_became_active":"2024-04-25T11:23:45.677024+0000","last_became_peered":"2024-04-25T11:23:45.677024+0000","last_unstale":"2024-04-25T11:23:47.667374+0000","last_undegraded":"2024-04-25T11:23:47.667374+0000","last_fullsized":"2024-04-25T11:23:47.667374+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:19.154764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"74'1","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807777+0000","last_change":"2024-04-25T11:23:40.639442+0000","last_active":"2024-04-25T11:23:47.807777+0000","last_peered":"2024-04-25T11:23:47.807777+0000","last_clean":"2024-04-25T11:23:47.807777+0000","last_became_active":"2024-04-25T11:23:40.639068+0000","last_became_peered":"2024-04-25T11:23:40.639068+0000","last_unstale":"2024-04-25T11:23:47.807777+0000","last_undegraded":"2024-04-25T11:23:47.807777+0000","last_fullsized":"2024-04-25T11:23:47.807777+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:21.491233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808780+0000","last_change":"2024-04-25T11:23:41.652309+0000","last_active":"2024-04-25T11:23:47.808780+0000","last_peered":"2024-04-25T11:23:47.808780+0000","last_clean":"2024-04-25T11:23:47.808780+0000","last_became_active":"2024-04-25T11:23:41.651927+0000","last_became_peered":"2024-04-25T11:23:41.651927+0000","last_unstale":"2024-04-25T11:23:47.808780+0000","last_undegraded":"2024-04-25T11:23:47.808780+0000","last_fullsized":"2024-04-25T11:23:47.808780+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:51.002480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808249+0000","last_change":"2024-04-25T11:23:46.384203+0000","last_active":"2024-04-25T11:23:47.808249+0000","last_peered":"2024-04-25T11:23:47.808249+0000","last_clean":"2024-04-25T11:23:47.808249+0000","last_became_active":"2024-04-25T11:23:46.383532+0000","last_became_peered":"2024-04-25T11:23:46.383532+0000","last_unstale":"2024-04-25T11:23:47.808249+0000","last_undegraded":"2024-04-25T11:23:47.808249+0000","last_fullsized":"2024-04-25T11:23:47.808249+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:25.265484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808435+0000","last_change":"2024-04-25T11:23:40.640082+0000","last_active":"2024-04-25T11:23:47.808435+0000","last_peered":"2024-04-25T11:23:47.808435+0000","last_clean":"2024-04-25T11:23:47.808435+0000","last_became_active":"2024-04-25T11:23:40.639725+0000","last_became_peered":"2024-04-25T11:23:40.639725+0000","last_unstale":"2024-04-25T11:23:47.808435+0000","last_undegraded":"2024-04-25T11:23:47.808435+0000","last_fullsized":"2024-04-25T11:23:47.808435+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:01.244116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808522+0000","last_change":"2024-04-25T11:23:41.645439+0000","last_active":"2024-04-25T11:23:47.808522+0000","last_peered":"2024-04-25T11:23:47.808522+0000","last_clean":"2024-04-25T11:23:47.808522+0000","last_became_active":"2024-04-25T11:23:41.645056+0000","last_became_peered":"2024-04-25T11:23:41.645056+0000","last_unstale":"2024-04-25T11:23:47.808522+0000","last_undegraded":"2024-04-25T11:23:47.808522+0000","last_fullsized":"2024-04-25T11:23:47.808522+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:43.100321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807759+0000","last_change":"2024-04-25T11:23:45.672174+0000","last_active":"2024-04-25T11:23:47.807759+0000","last_peered":"2024-04-25T11:23:47.807759+0000","last_clean":"2024-04-25T11:23:47.807759+0000","last_became_active":"2024-04-25T11:23:45.670401+0000","last_became_peered":"2024-04-25T11:23:45.670401+0000","last_unstale":"2024-04-25T11:23:47.807759+0000","last_undegraded":"2024-04-25T11:23:47.807759+0000","last_fullsized":"2024-04-25T11:23:47.807759+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:59.259644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.809259+0000","last_change":"2024-04-25T11:23:40.636704+0000","last_active":"2024-04-25T11:23:47.809259+0000","last_peered":"2024-04-25T11:23:47.809259+0000","last_clean":"2024-04-25T11:23:47.809259+0000","last_became_active":"2024-04-25T11:23:40.636319+0000","last_became_peered":"2024-04-25T11:23:40.636319+0000","last_unstale":"2024-04-25T11:23:47.809259+0000","last_undegraded":"2024-04-25T11:23:47.809259+0000","last_fullsized":"2024-04-25T11:23:47.809259+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:17.199946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807858+0000","last_change":"2024-04-25T11:23:41.660510+0000","last_active":"2024-04-25T11:23:47.807858+0000","last_peered":"2024-04-25T11:23:47.807858+0000","last_clean":"2024-04-25T11:23:47.807858+0000","last_became_active":"2024-04-25T11:23:41.659837+0000","last_became_peered":"2024-04-25T11:23:41.659837+0000","last_unstale":"2024-04-25T11:23:47.807858+0000","last_undegraded":"2024-04-25T11:23:47.807858+0000","last_fullsized":"2024-04-25T11:23:47.807858+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:33.879487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807124+0000","last_change":"2024-04-25T11:23:46.383239+0000","last_active":"2024-04-25T11:23:47.807124+0000","last_peered":"2024-04-25T11:23:47.807124+0000","last_clean":"2024-04-25T11:23:47.807124+0000","last_became_active":"2024-04-25T11:23:46.382728+0000","last_became_peered":"2024-04-25T11:23:46.382728+0000","last_unstale":"2024-04-25T11:23:47.807124+0000","last_undegraded":"2024-04-25T11:23:47.807124+0000","last_fullsized":"2024-04-25T11:23:47.807124+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:25.087389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.667390+0000","last_change":"2024-04-25T11:23:41.654723+0000","last_active":"2024-04-25T11:23:47.667390+0000","last_peered":"2024-04-25T11:23:47.667390+0000","last_clean":"2024-04-25T11:23:47.667390+0000","last_became_active":"2024-04-25T11:23:41.654140+0000","last_became_peered":"2024-04-25T11:23:41.654140+0000","last_unstale":"2024-04-25T11:23:47.667390+0000","last_undegraded":"2024-04-25T11:23:47.667390+0000","last_fullsized":"2024-04-25T11:23:47.667390+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:41.981728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:24:09.060515+0000","last_change":"2024-04-25T11:23:40.641902+0000","last_active":"2024-04-25T11:24:09.060515+0000","last_peered":"2024-04-25T11:24:09.060515+0000","last_clean":"2024-04-25T11:24:09.060515+0000","last_became_active":"2024-04-25T11:23:40.641490+0000","last_became_peered":"2024-04-25T11:23:40.641490+0000","last_unstale":"2024-04-25T11:24:09.060515+0000","last_undegraded":"2024-04-25T11:24:09.060515+0000","last_fullsized":"2024-04-25T11:24:09.060515+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:03.939354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808902+0000","last_change":"2024-04-25T11:23:45.670485+0000","last_active":"2024-04-25T11:23:47.808902+0000","last_peered":"2024-04-25T11:23:47.808902+0000","last_clean":"2024-04-25T11:23:47.808902+0000","last_became_active":"2024-04-25T11:23:45.669842+0000","last_became_peered":"2024-04-25T11:23:45.669842+0000","last_unstale":"2024-04-25T11:23:47.808902+0000","last_undegraded":"2024-04-25T11:23:47.808902+0000","last_fullsized":"2024-04-25T11:23:47.808902+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:32.801217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.668341+0000","last_change":"2024-04-25T11:23:41.640583+0000","last_active":"2024-04-25T11:23:47.668341+0000","last_peered":"2024-04-25T11:23:47.668341+0000","last_clean":"2024-04-25T11:23:47.668341+0000","last_became_active":"2024-04-25T11:23:41.640262+0000","last_became_peered":"2024-04-25T11:23:41.640262+0000","last_unstale":"2024-04-25T11:23:47.668341+0000","last_undegraded":"2024-04-25T11:23:47.668341+0000","last_fullsized":"2024-04-25T11:23:47.668341+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:09.539504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:24:08.065847+0000","last_change":"2024-04-25T11:23:40.628424+0000","last_active":"2024-04-25T11:24:08.065847+0000","last_peered":"2024-04-25T11:24:08.065847+0000","last_clean":"2024-04-25T11:24:08.065847+0000","last_became_active":"2024-04-25T11:23:40.628064+0000","last_became_peered":"2024-04-25T11:23:40.628064+0000","last_unstale":"2024-04-25T11:24:08.065847+0000","last_undegraded":"2024-04-25T11:24:08.065847+0000","last_fullsized":"2024-04-25T11:24:08.065847+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:34.837291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":99,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807960+0000","last_change":"2024-04-25T11:22:25.135774+0000","last_active":"2024-04-25T11:23:47.807960+0000","last_peered":"2024-04-25T11:23:47.807960+0000","last_clean":"2024-04-25T11:23:47.807960+0000","last_became_active":"2024-04-25T11:22:25.131178+0000","last_became_peered":"2024-04-25T11:22:25.131178+0000","last_unstale":"2024-04-25T11:23:47.807960+0000","last_undegraded":"2024-04-25T11:23:47.807960+0000","last_fullsized":"2024-04-25T11:23:47.807960+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:18:57.891013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:18:57.891013+0000","last_clean_scrub_stamp":"2024-04-25T11:18:57.891013+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:29.317685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.667984+0000","last_change":"2024-04-25T11:23:46.385048+0000","last_active":"2024-04-25T11:23:47.667984+0000","last_peered":"2024-04-25T11:23:47.667984+0000","last_clean":"2024-04-25T11:23:47.667984+0000","last_became_active":"2024-04-25T11:23:46.384014+0000","last_became_peered":"2024-04-25T11:23:46.384014+0000","last_unstale":"2024-04-25T11:23:47.667984+0000","last_undegraded":"2024-04-25T11:23:47.667984+0000","last_fullsized":"2024-04-25T11:23:47.667984+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:26.508616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.806557+0000","last_change":"2024-04-25T11:23:41.650783+0000","last_active":"2024-04-25T11:23:47.806557+0000","last_peered":"2024-04-25T11:23:47.806557+0000","last_clean":"2024-04-25T11:23:47.806557+0000","last_became_active":"2024-04-25T11:23:41.650431+0000","last_became_peered":"2024-04-25T11:23:41.650431+0000","last_unstale":"2024-04-25T11:23:47.806557+0000","last_undegraded":"2024-04-25T11:23:47.806557+0000","last_fullsized":"2024-04-25T11:23:47.806557+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:38.925779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808088+0000","last_change":"2024-04-25T11:23:40.638879+0000","last_active":"2024-04-25T11:23:47.808088+0000","last_peered":"2024-04-25T11:23:47.808088+0000","last_clean":"2024-04-25T11:23:47.808088+0000","last_became_active":"2024-04-25T11:23:40.638475+0000","last_became_peered":"2024-04-25T11:23:40.638475+0000","last_unstale":"2024-04-25T11:23:47.808088+0000","last_undegraded":"2024-04-25T11:23:47.808088+0000","last_fullsized":"2024-04-25T11:23:47.808088+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:25.591831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808503+0000","last_change":"2024-04-25T11:23:45.679333+0000","last_active":"2024-04-25T11:23:47.808503+0000","last_peered":"2024-04-25T11:23:47.808503+0000","last_clean":"2024-04-25T11:23:47.808503+0000","last_became_active":"2024-04-25T11:23:45.678622+0000","last_became_peered":"2024-04-25T11:23:45.678622+0000","last_unstale":"2024-04-25T11:23:47.808503+0000","last_undegraded":"2024-04-25T11:23:47.808503+0000","last_fullsized":"2024-04-25T11:23:47.808503+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:08.951281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808065+0000","last_change":"2024-04-25T11:23:40.642975+0000","last_active":"2024-04-25T11:23:47.808065+0000","last_peered":"2024-04-25T11:23:47.808065+0000","last_clean":"2024-04-25T11:23:47.808065+0000","last_became_active":"2024-04-25T11:23:40.642605+0000","last_became_peered":"2024-04-25T11:23:40.642605+0000","last_unstale":"2024-04-25T11:23:47.808065+0000","last_undegraded":"2024-04-25T11:23:47.808065+0000","last_fullsized":"2024-04-25T11:23:47.808065+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:57.665763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807680+0000","last_change":"2024-04-25T11:23:41.628396+0000","last_active":"2024-04-25T11:23:47.807680+0000","last_peered":"2024-04-25T11:23:47.807680+0000","last_clean":"2024-04-25T11:23:47.807680+0000","last_became_active":"2024-04-25T11:23:41.627979+0000","last_became_peered":"2024-04-25T11:23:41.627979+0000","last_unstale":"2024-04-25T11:23:47.807680+0000","last_undegraded":"2024-04-25T11:23:47.807680+0000","last_fullsized":"2024-04-25T11:23:47.807680+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:48.471219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.668877+0000","last_change":"2024-04-25T11:23:46.384252+0000","last_active":"2024-04-25T11:23:47.668877+0000","last_peered":"2024-04-25T11:23:47.668877+0000","last_clean":"2024-04-25T11:23:47.668877+0000","last_became_active":"2024-04-25T11:23:46.383408+0000","last_became_peered":"2024-04-25T11:23:46.383408+0000","last_unstale":"2024-04-25T11:23:47.668877+0000","last_undegraded":"2024-04-25T11:23:47.668877+0000","last_fullsized":"2024-04-25T11:23:47.668877+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:19.337043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"80'8","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:24:07.916188+0000","last_change":"2024-04-25T11:23:40.643767+0000","last_active":"2024-04-25T11:24:07.916188+0000","last_peered":"2024-04-25T11:24:07.916188+0000","last_clean":"2024-04-25T11:24:07.916188+0000","last_became_active":"2024-04-25T11:23:40.642748+0000","last_became_peered":"2024-04-25T11:23:40.642748+0000","last_unstale":"2024-04-25T11:24:07.916188+0000","last_undegraded":"2024-04-25T11:24:07.916188+0000","last_fullsized":"2024-04-25T11:24:07.916188+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:12.322055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.668696+0000","last_change":"2024-04-25T11:23:41.642766+0000","last_active":"2024-04-25T11:23:47.668696+0000","last_peered":"2024-04-25T11:23:47.668696+0000","last_clean":"2024-04-25T11:23:47.668696+0000","last_became_active":"2024-04-25T11:23:41.642409+0000","last_became_peered":"2024-04-25T11:23:41.642409+0000","last_unstale":"2024-04-25T11:23:47.668696+0000","last_undegraded":"2024-04-25T11:23:47.668696+0000","last_fullsized":"2024-04-25T11:23:47.668696+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:36.139574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.809051+0000","last_change":"2024-04-25T11:23:45.670926+0000","last_active":"2024-04-25T11:23:47.809051+0000","last_peered":"2024-04-25T11:23:47.809051+0000","last_clean":"2024-04-25T11:23:47.809051+0000","last_became_active":"2024-04-25T11:23:45.670444+0000","last_became_peered":"2024-04-25T11:23:45.670444+0000","last_unstale":"2024-04-25T11:23:47.809051+0000","last_undegraded":"2024-04-25T11:23:47.809051+0000","last_fullsized":"2024-04-25T11:23:47.809051+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:40.538155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:24:08.065782+0000","last_change":"2024-04-25T11:23:40.647132+0000","last_active":"2024-04-25T11:24:08.065782+0000","last_peered":"2024-04-25T11:24:08.065782+0000","last_clean":"2024-04-25T11:24:08.065782+0000","last_became_active":"2024-04-25T11:23:40.646792+0000","last_became_peered":"2024-04-25T11:23:40.646792+0000","last_unstale":"2024-04-25T11:24:08.065782+0000","last_undegraded":"2024-04-25T11:24:08.065782+0000","last_fullsized":"2024-04-25T11:24:08.065782+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:50.935908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807656+0000","last_change":"2024-04-25T11:23:41.658368+0000","last_active":"2024-04-25T11:23:47.807656+0000","last_peered":"2024-04-25T11:23:47.807656+0000","last_clean":"2024-04-25T11:23:47.807656+0000","last_became_active":"2024-04-25T11:23:41.657972+0000","last_became_peered":"2024-04-25T11:23:41.657972+0000","last_unstale":"2024-04-25T11:23:47.807656+0000","last_undegraded":"2024-04-25T11:23:47.807656+0000","last_fullsized":"2024-04-25T11:23:47.807656+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:02.717294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.809318+0000","last_change":"2024-04-25T11:23:46.384683+0000","last_active":"2024-04-25T11:23:47.809318+0000","last_peered":"2024-04-25T11:23:47.809318+0000","last_clean":"2024-04-25T11:23:47.809318+0000","last_became_active":"2024-04-25T11:23:46.384230+0000","last_became_peered":"2024-04-25T11:23:46.384230+0000","last_unstale":"2024-04-25T11:23:47.809318+0000","last_undegraded":"2024-04-25T11:23:47.809318+0000","last_fullsized":"2024-04-25T11:23:47.809318+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:06.982061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:24:05.895647+0000","last_change":"2024-04-25T11:23:40.642289+0000","last_active":"2024-04-25T11:24:05.895647+0000","last_peered":"2024-04-25T11:24:05.895647+0000","last_clean":"2024-04-25T11:24:05.895647+0000","last_became_active":"2024-04-25T11:23:40.641948+0000","last_became_peered":"2024-04-25T11:23:40.641948+0000","last_unstale":"2024-04-25T11:24:05.895647+0000","last_undegraded":"2024-04-25T11:24:05.895647+0000","last_fullsized":"2024-04-25T11:24:05.895647+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:39.265090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807380+0000","last_change":"2024-04-25T11:23:41.651792+0000","last_active":"2024-04-25T11:23:47.807380+0000","last_peered":"2024-04-25T11:23:47.807380+0000","last_clean":"2024-04-25T11:23:47.807380+0000","last_became_active":"2024-04-25T11:23:41.651413+0000","last_became_peered":"2024-04-25T11:23:41.651413+0000","last_unstale":"2024-04-25T11:23:47.807380+0000","last_undegraded":"2024-04-25T11:23:47.807380+0000","last_fullsized":"2024-04-25T11:23:47.807380+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:50.805600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808651+0000","last_change":"2024-04-25T11:23:45.670509+0000","last_active":"2024-04-25T11:23:47.808651+0000","last_peered":"2024-04-25T11:23:47.808651+0000","last_clean":"2024-04-25T11:23:47.808651+0000","last_became_active":"2024-04-25T11:23:45.670145+0000","last_became_peered":"2024-04-25T11:23:45.670145+0000","last_unstale":"2024-04-25T11:23:47.808651+0000","last_undegraded":"2024-04-25T11:23:47.808651+0000","last_fullsized":"2024-04-25T11:23:47.808651+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:03.764278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:24:05.895435+0000","last_change":"2024-04-25T11:23:40.644123+0000","last_active":"2024-04-25T11:24:05.895435+0000","last_peered":"2024-04-25T11:24:05.895435+0000","last_clean":"2024-04-25T11:24:05.895435+0000","last_became_active":"2024-04-25T11:23:40.643750+0000","last_became_peered":"2024-04-25T11:23:40.643750+0000","last_unstale":"2024-04-25T11:24:05.895435+0000","last_undegraded":"2024-04-25T11:24:05.895435+0000","last_fullsized":"2024-04-25T11:24:05.895435+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:49.613669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808227+0000","last_change":"2024-04-25T11:23:41.643606+0000","last_active":"2024-04-25T11:23:47.808227+0000","last_peered":"2024-04-25T11:23:47.808227+0000","last_clean":"2024-04-25T11:23:47.808227+0000","last_became_active":"2024-04-25T11:23:41.643248+0000","last_became_peered":"2024-04-25T11:23:41.643248+0000","last_unstale":"2024-04-25T11:23:47.808227+0000","last_undegraded":"2024-04-25T11:23:47.808227+0000","last_fullsized":"2024-04-25T11:23:47.808227+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:46.200959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808435+0000","last_change":"2024-04-25T11:23:45.674138+0000","last_active":"2024-04-25T11:23:47.808435+0000","last_peered":"2024-04-25T11:23:47.808435+0000","last_clean":"2024-04-25T11:23:47.808435+0000","last_became_active":"2024-04-25T11:23:45.672549+0000","last_became_peered":"2024-04-25T11:23:45.672549+0000","last_unstale":"2024-04-25T11:23:47.808435+0000","last_undegraded":"2024-04-25T11:23:47.808435+0000","last_fullsized":"2024-04-25T11:23:47.808435+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:11.345176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"80'2","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:24:06.889336+0000","last_change":"2024-04-25T11:23:40.636602+0000","last_active":"2024-04-25T11:24:06.889336+0000","last_peered":"2024-04-25T11:24:06.889336+0000","last_clean":"2024-04-25T11:24:06.889336+0000","last_became_active":"2024-04-25T11:23:40.636187+0000","last_became_peered":"2024-04-25T11:23:40.636187+0000","last_unstale":"2024-04-25T11:24:06.889336+0000","last_undegraded":"2024-04-25T11:24:06.889336+0000","last_fullsized":"2024-04-25T11:24:06.889336+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:15.894714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.668747+0000","last_change":"2024-04-25T11:23:41.647345+0000","last_active":"2024-04-25T11:23:47.668747+0000","last_peered":"2024-04-25T11:23:47.668747+0000","last_clean":"2024-04-25T11:23:47.668747+0000","last_became_active":"2024-04-25T11:23:41.646981+0000","last_became_peered":"2024-04-25T11:23:41.646981+0000","last_unstale":"2024-04-25T11:23:47.668747+0000","last_undegraded":"2024-04-25T11:23:47.668747+0000","last_fullsized":"2024-04-25T11:23:47.668747+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:30.096856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.809026+0000","last_change":"2024-04-25T11:23:46.384132+0000","last_active":"2024-04-25T11:23:47.809026+0000","last_peered":"2024-04-25T11:23:47.809026+0000","last_clean":"2024-04-25T11:23:47.809026+0000","last_became_active":"2024-04-25T11:23:46.383501+0000","last_became_peered":"2024-04-25T11:23:46.383501+0000","last_unstale":"2024-04-25T11:23:47.809026+0000","last_undegraded":"2024-04-25T11:23:47.809026+0000","last_fullsized":"2024-04-25T11:23:47.809026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:22.486326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:24:05.893999+0000","last_change":"2024-04-25T11:23:40.634942+0000","last_active":"2024-04-25T11:24:05.893999+0000","last_peered":"2024-04-25T11:24:05.893999+0000","last_clean":"2024-04-25T11:24:05.893999+0000","last_became_active":"2024-04-25T11:23:40.634557+0000","last_became_peered":"2024-04-25T11:23:40.634557+0000","last_unstale":"2024-04-25T11:24:05.893999+0000","last_undegraded":"2024-04-25T11:24:05.893999+0000","last_fullsized":"2024-04-25T11:24:05.893999+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:08.612328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.809156+0000","last_change":"2024-04-25T11:23:41.648985+0000","last_active":"2024-04-25T11:23:47.809156+0000","last_peered":"2024-04-25T11:23:47.809156+0000","last_clean":"2024-04-25T11:23:47.809156+0000","last_became_active":"2024-04-25T11:23:41.645643+0000","last_became_peered":"2024-04-25T11:23:41.645643+0000","last_unstale":"2024-04-25T11:23:47.809156+0000","last_undegraded":"2024-04-25T11:23:47.809156+0000","last_fullsized":"2024-04-25T11:23:47.809156+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:22.289698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.667439+0000","last_change":"2024-04-25T11:23:45.671474+0000","last_active":"2024-04-25T11:23:47.667439+0000","last_peered":"2024-04-25T11:23:47.667439+0000","last_clean":"2024-04-25T11:23:47.667439+0000","last_became_active":"2024-04-25T11:23:45.670976+0000","last_became_peered":"2024-04-25T11:23:45.670976+0000","last_unstale":"2024-04-25T11:23:47.667439+0000","last_undegraded":"2024-04-25T11:23:47.667439+0000","last_fullsized":"2024-04-25T11:23:47.667439+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:13:29.771087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"80'4","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:24:06.903258+0000","last_change":"2024-04-25T11:23:40.647950+0000","last_active":"2024-04-25T11:24:06.903258+0000","last_peered":"2024-04-25T11:24:06.903258+0000","last_clean":"2024-04-25T11:24:06.903258+0000","last_became_active":"2024-04-25T11:23:40.647549+0000","last_became_peered":"2024-04-25T11:23:40.647549+0000","last_unstale":"2024-04-25T11:24:06.903258+0000","last_undegraded":"2024-04-25T11:24:06.903258+0000","last_fullsized":"2024-04-25T11:24:06.903258+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:47.975294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.809264+0000","last_change":"2024-04-25T11:23:41.647670+0000","last_active":"2024-04-25T11:23:47.809264+0000","last_peered":"2024-04-25T11:23:47.809264+0000","last_clean":"2024-04-25T11:23:47.809264+0000","last_became_active":"2024-04-25T11:23:41.646686+0000","last_became_peered":"2024-04-25T11:23:41.646686+0000","last_unstale":"2024-04-25T11:23:47.809264+0000","last_undegraded":"2024-04-25T11:23:47.809264+0000","last_fullsized":"2024-04-25T11:23:47.809264+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:21.260685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.806946+0000","last_change":"2024-04-25T11:23:45.676721+0000","last_active":"2024-04-25T11:23:47.806946+0000","last_peered":"2024-04-25T11:23:47.806946+0000","last_clean":"2024-04-25T11:23:47.806946+0000","last_became_active":"2024-04-25T11:23:45.676325+0000","last_became_peered":"2024-04-25T11:23:45.676325+0000","last_unstale":"2024-04-25T11:23:47.806946+0000","last_undegraded":"2024-04-25T11:23:47.806946+0000","last_fullsized":"2024-04-25T11:23:47.806946+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:12.023474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"80'2","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:24:07.914780+0000","last_change":"2024-04-25T11:23:40.642974+0000","last_active":"2024-04-25T11:24:07.914780+0000","last_peered":"2024-04-25T11:24:07.914780+0000","last_clean":"2024-04-25T11:24:07.914780+0000","last_became_active":"2024-04-25T11:23:40.642628+0000","last_became_peered":"2024-04-25T11:23:40.642628+0000","last_unstale":"2024-04-25T11:24:07.914780+0000","last_undegraded":"2024-04-25T11:24:07.914780+0000","last_fullsized":"2024-04-25T11:24:07.914780+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:03.870867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.809465+0000","last_change":"2024-04-25T11:23:41.643519+0000","last_active":"2024-04-25T11:23:47.809465+0000","last_peered":"2024-04-25T11:23:47.809465+0000","last_clean":"2024-04-25T11:23:47.809465+0000","last_became_active":"2024-04-25T11:23:41.642621+0000","last_became_peered":"2024-04-25T11:23:41.642621+0000","last_unstale":"2024-04-25T11:23:47.809465+0000","last_undegraded":"2024-04-25T11:23:47.809465+0000","last_fullsized":"2024-04-25T11:23:47.809465+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:38.699914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.667080+0000","last_change":"2024-04-25T11:23:45.671017+0000","last_active":"2024-04-25T11:23:47.667080+0000","last_peered":"2024-04-25T11:23:47.667080+0000","last_clean":"2024-04-25T11:23:47.667080+0000","last_became_active":"2024-04-25T11:23:45.670603+0000","last_became_peered":"2024-04-25T11:23:45.670603+0000","last_unstale":"2024-04-25T11:23:47.667080+0000","last_undegraded":"2024-04-25T11:23:47.667080+0000","last_fullsized":"2024-04-25T11:23:47.667080+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:32.385176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:24:08.064752+0000","last_change":"2024-04-25T11:23:40.635711+0000","last_active":"2024-04-25T11:24:08.064752+0000","last_peered":"2024-04-25T11:24:08.064752+0000","last_clean":"2024-04-25T11:24:08.064752+0000","last_became_active":"2024-04-25T11:23:40.635311+0000","last_became_peered":"2024-04-25T11:23:40.635311+0000","last_unstale":"2024-04-25T11:24:08.064752+0000","last_undegraded":"2024-04-25T11:24:08.064752+0000","last_fullsized":"2024-04-25T11:24:08.064752+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:30:01.135579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.809121+0000","last_change":"2024-04-25T11:23:41.634946+0000","last_active":"2024-04-25T11:23:47.809121+0000","last_peered":"2024-04-25T11:23:47.809121+0000","last_clean":"2024-04-25T11:23:47.809121+0000","last_became_active":"2024-04-25T11:23:41.634505+0000","last_became_peered":"2024-04-25T11:23:41.634505+0000","last_unstale":"2024-04-25T11:23:47.809121+0000","last_undegraded":"2024-04-25T11:23:47.809121+0000","last_fullsized":"2024-04-25T11:23:47.809121+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:52.793207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808398+0000","last_change":"2024-04-25T11:23:41.642079+0000","last_active":"2024-04-25T11:23:47.808398+0000","last_peered":"2024-04-25T11:23:47.808398+0000","last_clean":"2024-04-25T11:23:47.808398+0000","last_became_active":"2024-04-25T11:23:41.641721+0000","last_became_peered":"2024-04-25T11:23:41.641721+0000","last_unstale":"2024-04-25T11:23:47.808398+0000","last_undegraded":"2024-04-25T11:23:47.808398+0000","last_fullsized":"2024-04-25T11:23:47.808398+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:59.968749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.809166+0000","last_change":"2024-04-25T11:23:41.650503+0000","last_active":"2024-04-25T11:23:47.809166+0000","last_peered":"2024-04-25T11:23:47.809166+0000","last_clean":"2024-04-25T11:23:47.809166+0000","last_became_active":"2024-04-25T11:23:41.650158+0000","last_became_peered":"2024-04-25T11:23:41.650158+0000","last_unstale":"2024-04-25T11:23:47.809166+0000","last_undegraded":"2024-04-25T11:23:47.809166+0000","last_fullsized":"2024-04-25T11:23:47.809166+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:07.228378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807856+0000","last_change":"2024-04-25T11:23:41.641108+0000","last_active":"2024-04-25T11:23:47.807856+0000","last_peered":"2024-04-25T11:23:47.807856+0000","last_clean":"2024-04-25T11:23:47.807856+0000","last_became_active":"2024-04-25T11:23:41.640665+0000","last_became_peered":"2024-04-25T11:23:41.640665+0000","last_unstale":"2024-04-25T11:23:47.807856+0000","last_undegraded":"2024-04-25T11:23:47.807856+0000","last_fullsized":"2024-04-25T11:23:47.807856+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:48.075466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807285+0000","last_change":"2024-04-25T11:23:41.652322+0000","last_active":"2024-04-25T11:23:47.807285+0000","last_peered":"2024-04-25T11:23:47.807285+0000","last_clean":"2024-04-25T11:23:47.807285+0000","last_became_active":"2024-04-25T11:23:41.651681+0000","last_became_peered":"2024-04-25T11:23:41.651681+0000","last_unstale":"2024-04-25T11:23:47.807285+0000","last_undegraded":"2024-04-25T11:23:47.807285+0000","last_fullsized":"2024-04-25T11:23:47.807285+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:17.574861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807783+0000","last_change":"2024-04-25T11:23:41.651701+0000","last_active":"2024-04-25T11:23:47.807783+0000","last_peered":"2024-04-25T11:23:47.807783+0000","last_clean":"2024-04-25T11:23:47.807783+0000","last_became_active":"2024-04-25T11:23:41.651362+0000","last_became_peered":"2024-04-25T11:23:41.651362+0000","last_unstale":"2024-04-25T11:23:47.807783+0000","last_undegraded":"2024-04-25T11:23:47.807783+0000","last_fullsized":"2024-04-25T11:23:47.807783+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:50.777396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.809360+0000","last_change":"2024-04-25T11:23:41.640385+0000","last_active":"2024-04-25T11:23:47.809360+0000","last_peered":"2024-04-25T11:23:47.809360+0000","last_clean":"2024-04-25T11:23:47.809360+0000","last_became_active":"2024-04-25T11:23:41.639986+0000","last_became_peered":"2024-04-25T11:23:41.639986+0000","last_unstale":"2024-04-25T11:23:47.809360+0000","last_undegraded":"2024-04-25T11:23:47.809360+0000","last_fullsized":"2024-04-25T11:23:47.809360+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:13.376858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.808557+0000","last_change":"2024-04-25T11:23:41.648075+0000","last_active":"2024-04-25T11:23:47.808557+0000","last_peered":"2024-04-25T11:23:47.808557+0000","last_clean":"2024-04-25T11:23:47.808557+0000","last_became_active":"2024-04-25T11:23:41.647681+0000","last_became_peered":"2024-04-25T11:23:41.647681+0000","last_unstale":"2024-04-25T11:23:47.808557+0000","last_undegraded":"2024-04-25T11:23:47.808557+0000","last_fullsized":"2024-04-25T11:23:47.808557+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:44.916762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.668797+0000","last_change":"2024-04-25T11:23:41.641788+0000","last_active":"2024-04-25T11:23:47.668797+0000","last_peered":"2024-04-25T11:23:47.668797+0000","last_clean":"2024-04-25T11:23:47.668797+0000","last_became_active":"2024-04-25T11:23:41.641407+0000","last_became_peered":"2024-04-25T11:23:41.641407+0000","last_unstale":"2024-04-25T11:23:47.668797+0000","last_undegraded":"2024-04-25T11:23:47.668797+0000","last_fullsized":"2024-04-25T11:23:47.668797+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:33.609749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.667335+0000","last_change":"2024-04-25T11:23:41.647616+0000","last_active":"2024-04-25T11:23:47.667335+0000","last_peered":"2024-04-25T11:23:47.667335+0000","last_clean":"2024-04-25T11:23:47.667335+0000","last_became_active":"2024-04-25T11:23:41.645396+0000","last_became_peered":"2024-04-25T11:23:41.645396+0000","last_unstale":"2024-04-25T11:23:47.667335+0000","last_undegraded":"2024-04-25T11:23:47.667335+0000","last_fullsized":"2024-04-25T11:23:47.667335+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:47.842574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.668544+0000","last_change":"2024-04-25T11:23:41.638407+0000","last_active":"2024-04-25T11:23:47.668544+0000","last_peered":"2024-04-25T11:23:47.668544+0000","last_clean":"2024-04-25T11:23:47.668544+0000","last_became_active":"2024-04-25T11:23:41.638054+0000","last_became_peered":"2024-04-25T11:23:41.638054+0000","last_unstale":"2024-04-25T11:23:47.668544+0000","last_undegraded":"2024-04-25T11:23:47.668544+0000","last_fullsized":"2024-04-25T11:23:47.668544+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:34.306493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.667520+0000","last_change":"2024-04-25T11:23:41.650140+0000","last_active":"2024-04-25T11:23:47.667520+0000","last_peered":"2024-04-25T11:23:47.667520+0000","last_clean":"2024-04-25T11:23:47.667520+0000","last_became_active":"2024-04-25T11:23:41.649487+0000","last_became_peered":"2024-04-25T11:23:41.649487+0000","last_unstale":"2024-04-25T11:23:47.667520+0000","last_undegraded":"2024-04-25T11:23:47.667520+0000","last_fullsized":"2024-04-25T11:23:47.667520+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:55.652921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.807201+0000","last_change":"2024-04-25T11:23:41.640253+0000","last_active":"2024-04-25T11:23:47.807201+0000","last_peered":"2024-04-25T11:23:47.807201+0000","last_clean":"2024-04-25T11:23:47.807201+0000","last_became_active":"2024-04-25T11:23:41.639926+0000","last_became_peered":"2024-04-25T11:23:41.639926+0000","last_unstale":"2024-04-25T11:23:47.807201+0000","last_undegraded":"2024-04-25T11:23:47.807201+0000","last_fullsized":"2024-04-25T11:23:47.807201+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:01.407198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:23:47.809145+0000","last_change":"2024-04-25T11:23:41.653244+0000","last_active":"2024-04-25T11:23:47.809145+0000","last_peered":"2024-04-25T11:23:47.809145+0000","last_clean":"2024-04-25T11:23:47.809145+0000","last_became_active":"2024-04-25T11:23:41.652875+0000","last_became_peered":"2024-04-25T11:23:41.652875+0000","last_unstale":"2024-04-25T11:23:47.809145+0000","last_undegraded":"2024-04-25T11:23:47.809145+0000","last_fullsized":"2024-04-25T11:23:47.809145+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:06.897766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":27603,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":25,"num_read_kb":24,"num_write":166,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":47,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":319488,"data_stored":61948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710738,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":431863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906967,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28524,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716628,"statfs":{"total":95995035648,"available":95965827072,"internally_reserved":0,"allocated":1544192,"data_stored":1034409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070493,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":921600,"data_stored":432985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266721,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":892928,"data_stored":432467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462949,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28468,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716684,"statfs":{"total":95995035648,"available":95965884416,"internally_reserved":0,"allocated":1482752,"data_stored":1022813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626477,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":901120,"data_stored":432585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":444057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738442,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":888832,"data_stored":431953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411393,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":437055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182459,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":432147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986231,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":432563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822705,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36820,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708332,"statfs":{"total":95995035648,"available":95957331968,"internally_reserved":0,"allocated":1519616,"data_stored":1024469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":24893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":24877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:24:11.084 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T11:24:11.084 INFO:tasks.cephadm:Setting up client nodes... 2024-04-25T11:24:11.085 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-25T11:24:11.154 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:10 smithi152 ceph-mon[32701]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:24:11.154 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:10 smithi152 ceph-mon[32701]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:24:11.154 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:10 smithi152 ceph-mon[32701]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:24:11.155 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:10 smithi152 ceph-mon[32701]: pgmap v299: 129 pgs: 129 active+clean; 603 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 5.1 KiB/s wr, 11 op/s 2024-04-25T11:24:11.155 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:10 smithi152 ceph-mon[32701]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T11:24:11.155 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:10 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/2448485462' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:24:11.155 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:10 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/1155999108' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:24:11.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:10 smithi146 ceph-mon[32821]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:24:11.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:10 smithi146 ceph-mon[32821]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:24:11.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:10 smithi146 ceph-mon[32821]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:24:11.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:10 smithi146 ceph-mon[32821]: pgmap v299: 129 pgs: 129 active+clean; 603 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 5.1 KiB/s wr, 11 op/s 2024-04-25T11:24:11.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:10 smithi146 ceph-mon[32821]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T11:24:11.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:10 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/2448485462' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:24:11.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:10 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/1155999108' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:24:11.357 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:24:12.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:11 smithi069 ceph-mon[29188]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T11:24:12.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:11 smithi152 ceph-mon[32701]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T11:24:12.250 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:11 smithi146 ceph-mon[32821]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T11:24:13.142 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:12 smithi069 ceph-mon[29188]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T11:24:13.142 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:12 smithi069 ceph-mon[29188]: pgmap v300: 129 pgs: 129 active+clean; 605 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5.9 KiB/s wr, 16 op/s 2024-04-25T11:24:13.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.162+0000 7f4cf3df7700 1 -- 172.21.15.69:0/2869527939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4cec105780 msgr2=0x7f4cec105b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:13.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.162+0000 7f4cf3df7700 1 --2- 172.21.15.69:0/2869527939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4cec105780 0x7f4cec105b90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8009990 tx=0x7f4ce8009ca0 comp rx=0 tx=0).stop 2024-04-25T11:24:13.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.163+0000 7f4cf3df7700 1 -- 172.21.15.69:0/2869527939 shutdown_connections 2024-04-25T11:24:13.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.163+0000 7f4cf3df7700 1 --2- 172.21.15.69:0/2869527939 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cec102b20 0x7f4cec10c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:13.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.163+0000 7f4cf3df7700 1 --2- 172.21.15.69:0/2869527939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4cec105780 0x7f4cec105b90 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:13.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.163+0000 7f4cf3df7700 1 --2- 172.21.15.69:0/2869527939 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4cec1021d0 0x7f4cec1025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:13.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.163+0000 7f4cf3df7700 1 -- 172.21.15.69:0/2869527939 >> 172.21.15.69:0/2869527939 conn(0x7f4cec0fde80 msgr2=0x7f4cec100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:13.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.163+0000 7f4cf3df7700 1 -- 172.21.15.69:0/2869527939 shutdown_connections 2024-04-25T11:24:13.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.163+0000 7f4cf3df7700 1 -- 172.21.15.69:0/2869527939 wait complete. 2024-04-25T11:24:13.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.163+0000 7f4cf3df7700 1 Processor -- start 2024-04-25T11:24:13.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.163+0000 7f4cf3df7700 1 -- start start 2024-04-25T11:24:13.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.163+0000 7f4cf3df7700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4cec1021d0 0x7f4cec19a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:13.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.164+0000 7f4cf3df7700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4cec102b20 0x7f4cec19ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:13.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.164+0000 7f4cf3df7700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cec105780 0x7f4cec1a1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:13.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.164+0000 7f4cf3df7700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4cec10fae0 con 0x7f4cec102b20 2024-04-25T11:24:13.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.164+0000 7f4cf3df7700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4cec10f960 con 0x7f4cec1021d0 2024-04-25T11:24:13.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.164+0000 7f4cf3df7700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4cec10fc60 con 0x7f4cec105780 2024-04-25T11:24:13.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.164+0000 7f4cf1392700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4cec102b20 0x7f4cec19ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:13.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.164+0000 7f4cf1392700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4cec102b20 0x7f4cec19ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54828/0 (socket says 172.21.15.69:54828) 2024-04-25T11:24:13.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.164+0000 7f4cf1392700 1 -- 172.21.15.69:0/1302464724 learned_addr learned my addr 172.21.15.69:0/1302464724 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:24:13.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.164+0000 7f4cf1b93700 1 --2- 172.21.15.69:0/1302464724 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4cec1021d0 0x7f4cec19a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:13.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.164+0000 7f4cf2394700 1 --2- 172.21.15.69:0/1302464724 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cec105780 0x7f4cec1a1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:13.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.164+0000 7f4cf1392700 1 -- 172.21.15.69:0/1302464724 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cec105780 msgr2=0x7f4cec1a1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:13.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.164+0000 7f4cf1392700 1 --2- 172.21.15.69:0/1302464724 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cec105780 0x7f4cec1a1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:13.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.164+0000 7f4cf1392700 1 -- 172.21.15.69:0/1302464724 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4cec1021d0 msgr2=0x7f4cec19a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:13.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.164+0000 7f4cf1392700 1 --2- 172.21.15.69:0/1302464724 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4cec1021d0 0x7f4cec19a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:13.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.164+0000 7f4cf1392700 1 -- 172.21.15.69:0/1302464724 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cec1a21b0 con 0x7f4cec102b20 2024-04-25T11:24:13.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.164+0000 7f4cf1392700 1 --2- 172.21.15.69:0/1302464724 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4cec102b20 0x7f4cec19ac20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc00d950 tx=0x7f4cdc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:13.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.165+0000 7f4ce2ffd700 1 -- 172.21.15.69:0/1302464724 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc016070 con 0x7f4cec102b20 2024-04-25T11:24:13.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.165+0000 7f4ce2ffd700 1 -- 172.21.15.69:0/1302464724 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4cdc00fd20 con 0x7f4cec102b20 2024-04-25T11:24:13.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.165+0000 7f4cf3df7700 1 -- 172.21.15.69:0/1302464724 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cec1a24a0 con 0x7f4cec102b20 2024-04-25T11:24:13.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.165+0000 7f4ce2ffd700 1 -- 172.21.15.69:0/1302464724 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc00e3b0 con 0x7f4cec102b20 2024-04-25T11:24:13.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.165+0000 7f4cf3df7700 1 -- 172.21.15.69:0/1302464724 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cec1a29e0 con 0x7f4cec102b20 2024-04-25T11:24:13.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.166+0000 7f4cf3df7700 1 -- 172.21.15.69:0/1302464724 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cec051190 con 0x7f4cec102b20 2024-04-25T11:24:13.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.167+0000 7f4ce2ffd700 1 -- 172.21.15.69:0/1302464724 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f4cdc019070 con 0x7f4cec102b20 2024-04-25T11:24:13.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.168+0000 7f4ce2ffd700 1 --2- 172.21.15.69:0/1302464724 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f4cd80afc10 0x7f4cd80b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:13.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.168+0000 7f4cf1b93700 1 --2- 172.21.15.69:0/1302464724 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f4cd80afc10 0x7f4cd80b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:13.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.168+0000 7f4ce2ffd700 1 -- 172.21.15.69:0/1302464724 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7f4cdc0deb20 con 0x7f4cec102b20 2024-04-25T11:24:13.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.169+0000 7f4cf1b93700 1 --2- 172.21.15.69:0/1302464724 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f4cd80afc10 0x7f4cd80b2120 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4ce80098e0 tx=0x7f4ce801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:13.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.172+0000 7f4ce2ffd700 1 -- 172.21.15.69:0/1302464724 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4cdc0a7300 con 0x7f4cec102b20 2024-04-25T11:24:13.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:12 smithi152 ceph-mon[32701]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T11:24:13.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:12 smithi152 ceph-mon[32701]: pgmap v300: 129 pgs: 129 active+clean; 605 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5.9 KiB/s wr, 16 op/s 2024-04-25T11:24:13.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:12 smithi146 ceph-mon[32821]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T11:24:13.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:12 smithi146 ceph-mon[32821]: pgmap v300: 129 pgs: 129 active+clean; 605 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5.9 KiB/s wr, 16 op/s 2024-04-25T11:24:13.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.456+0000 7f4cf3df7700 1 -- 172.21.15.69:0/1302464724 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f4cec02e510 con 0x7f4cec102b20 2024-04-25T11:24:13.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.459+0000 7f4ce2ffd700 1 -- 172.21.15.69:0/1302464724 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f4cdc0a6c00 con 0x7f4cec102b20 2024-04-25T11:24:13.453 INFO:teuthology.orchestra.run.smithi069.stdout:[client.0] 2024-04-25T11:24:13.453 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQBdPSpmjUlSGxAAsiZkmEiJl0Yl7VAKeAIQ1Q== 2024-04-25T11:24:13.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.461+0000 7f4cf3df7700 1 -- 172.21.15.69:0/1302464724 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f4cd80afc10 msgr2=0x7f4cd80b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:13.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.461+0000 7f4cf3df7700 1 --2- 172.21.15.69:0/1302464724 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f4cd80afc10 0x7f4cd80b2120 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4ce80098e0 tx=0x7f4ce801a040 comp rx=0 tx=0).stop 2024-04-25T11:24:13.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.461+0000 7f4cf3df7700 1 -- 172.21.15.69:0/1302464724 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4cec102b20 msgr2=0x7f4cec19ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:13.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.461+0000 7f4cf3df7700 1 --2- 172.21.15.69:0/1302464724 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4cec102b20 0x7f4cec19ac20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc00d950 tx=0x7f4cdc00dc60 comp rx=0 tx=0).stop 2024-04-25T11:24:13.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.461+0000 7f4cf3df7700 1 -- 172.21.15.69:0/1302464724 shutdown_connections 2024-04-25T11:24:13.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.461+0000 7f4cf3df7700 1 --2- 172.21.15.69:0/1302464724 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f4cd80afc10 0x7f4cd80b2120 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:13.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.461+0000 7f4cf3df7700 1 --2- 172.21.15.69:0/1302464724 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cec105780 0x7f4cec1a1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:13.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.462+0000 7f4cf3df7700 1 --2- 172.21.15.69:0/1302464724 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4cec1021d0 0x7f4cec19a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:13.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.462+0000 7f4cf3df7700 1 --2- 172.21.15.69:0/1302464724 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4cec102b20 0x7f4cec19ac20 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:13.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.462+0000 7f4cf3df7700 1 -- 172.21.15.69:0/1302464724 >> 172.21.15.69:0/1302464724 conn(0x7f4cec0fde80 msgr2=0x7f4cec108720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:13.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.462+0000 7f4cf3df7700 1 -- 172.21.15.69:0/1302464724 shutdown_connections 2024-04-25T11:24:13.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:13.462+0000 7f4cf3df7700 1 -- 172.21.15.69:0/1302464724 wait complete. 2024-04-25T11:24:14.080 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:13 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/1302464724' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T11:24:14.081 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:13 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/1302464724' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T11:24:14.081 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:13 smithi069 ceph-mon[29188]: pgmap v301: 129 pgs: 129 active+clean; 606 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6.2 KiB/s wr, 18 op/s 2024-04-25T11:24:14.108 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T11:24:14.108 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-25T11:24:14.108 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T11:24:14.144 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T11:24:14.145 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-25T11:24:14.145 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-25T11:24:14.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:13 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/1302464724' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T11:24:14.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:13 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/1302464724' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T11:24:14.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:13 smithi152 ceph-mon[32701]: pgmap v301: 129 pgs: 129 active+clean; 606 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6.2 KiB/s wr, 18 op/s 2024-04-25T11:24:14.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:13 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/1302464724' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T11:24:14.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:13 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/1302464724' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T11:24:14.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:13 smithi146 ceph-mon[32821]: pgmap v301: 129 pgs: 129 active+clean; 606 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6.2 KiB/s wr, 18 op/s 2024-04-25T11:24:14.421 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:24:16.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.392+0000 7f99788cb700 1 -- 172.21.15.69:0/1281893029 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9974108de0 msgr2=0x7f99741091f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:16.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.392+0000 7f99788cb700 1 --2- 172.21.15.69:0/1281893029 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9974108de0 0x7f99741091f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f99640099c0 tx=0x7f9964009cd0 comp rx=0 tx=0).stop 2024-04-25T11:24:16.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.392+0000 7f99788cb700 1 -- 172.21.15.69:0/1281893029 shutdown_connections 2024-04-25T11:24:16.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.392+0000 7f99788cb700 1 --2- 172.21.15.69:0/1281893029 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9974109730 0x7f9974111090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:16.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.392+0000 7f99788cb700 1 --2- 172.21.15.69:0/1281893029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f99741044b0 0x7f99741048c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:16.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.392+0000 7f99788cb700 1 --2- 172.21.15.69:0/1281893029 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9974108de0 0x7f99741091f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:16.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.392+0000 7f99788cb700 1 -- 172.21.15.69:0/1281893029 >> 172.21.15.69:0/1281893029 conn(0x7f99740ffdc0 msgr2=0x7f9974102210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:16.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.392+0000 7f99788cb700 1 -- 172.21.15.69:0/1281893029 shutdown_connections 2024-04-25T11:24:16.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.392+0000 7f99788cb700 1 -- 172.21.15.69:0/1281893029 wait complete. 2024-04-25T11:24:16.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.393+0000 7f99788cb700 1 Processor -- start 2024-04-25T11:24:16.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.393+0000 7f99788cb700 1 -- start start 2024-04-25T11:24:16.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.393+0000 7f99788cb700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99741044b0 0x7f9974076aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:16.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.393+0000 7f99788cb700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9974108de0 0x7f9974076fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:16.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.393+0000 7f99788cb700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9974109730 0x7f9974077520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:16.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.393+0000 7f99788cb700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9974114260 con 0x7f99741044b0 2024-04-25T11:24:16.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.393+0000 7f99788cb700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f99741140e0 con 0x7f9974109730 2024-04-25T11:24:16.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.393+0000 7f99788cb700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f99741143e0 con 0x7f9974108de0 2024-04-25T11:24:16.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.393+0000 7f997259c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99741044b0 0x7f9974076aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:16.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.393+0000 7f997259c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99741044b0 0x7f9974076aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54848/0 (socket says 172.21.15.69:54848) 2024-04-25T11:24:16.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.393+0000 7f997259c700 1 -- 172.21.15.69:0/3572296411 learned_addr learned my addr 172.21.15.69:0/3572296411 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:24:16.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.393+0000 7f9972d9d700 1 --2- 172.21.15.69:0/3572296411 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9974109730 0x7f9974077520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:16.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.393+0000 7f9971d9b700 1 --2- 172.21.15.69:0/3572296411 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9974108de0 0x7f9974076fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:16.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.393+0000 7f997259c700 1 -- 172.21.15.69:0/3572296411 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9974108de0 msgr2=0x7f9974076fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:16.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.393+0000 7f997259c700 1 --2- 172.21.15.69:0/3572296411 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9974108de0 0x7f9974076fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:16.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.393+0000 7f997259c700 1 -- 172.21.15.69:0/3572296411 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9974109730 msgr2=0x7f9974077520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:16.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.393+0000 7f997259c700 1 --2- 172.21.15.69:0/3572296411 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9974109730 0x7f9974077520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:16.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.393+0000 7f997259c700 1 -- 172.21.15.69:0/3572296411 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99741a8fe0 con 0x7f99741044b0 2024-04-25T11:24:16.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.394+0000 7f997259c700 1 --2- 172.21.15.69:0/3572296411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99741044b0 0x7f9974076aa0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f996800b530 tx=0x7f996800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:16.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.394+0000 7f99637fe700 1 -- 172.21.15.69:0/3572296411 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9968015020 con 0x7f99741044b0 2024-04-25T11:24:16.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.394+0000 7f99637fe700 1 -- 172.21.15.69:0/3572296411 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9968011cc0 con 0x7f99741044b0 2024-04-25T11:24:16.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.394+0000 7f99788cb700 1 -- 172.21.15.69:0/3572296411 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99741a92d0 con 0x7f99741044b0 2024-04-25T11:24:16.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.394+0000 7f99637fe700 1 -- 172.21.15.69:0/3572296411 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9968010d30 con 0x7f99741044b0 2024-04-25T11:24:16.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.394+0000 7f99788cb700 1 -- 172.21.15.69:0/3572296411 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99741a9810 con 0x7f99741044b0 2024-04-25T11:24:16.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.395+0000 7f99788cb700 1 -- 172.21.15.69:0/3572296411 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9974051190 con 0x7f99741044b0 2024-04-25T11:24:16.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.396+0000 7f99637fe700 1 -- 172.21.15.69:0/3572296411 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f9968018020 con 0x7f99741044b0 2024-04-25T11:24:16.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.397+0000 7f99637fe700 1 --2- 172.21.15.69:0/3572296411 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f995c0afc10 0x7f995c0b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:16.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.397+0000 7f9971d9b700 1 --2- 172.21.15.69:0/3572296411 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f995c0afc10 0x7f995c0b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:16.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.397+0000 7f99637fe700 1 -- 172.21.15.69:0/3572296411 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7f99680ded10 con 0x7f99741044b0 2024-04-25T11:24:16.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.397+0000 7f9971d9b700 1 --2- 172.21.15.69:0/3572296411 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f995c0afc10 0x7f995c0b2120 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f996400b580 tx=0x7f9964005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:16.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.401+0000 7f99637fe700 1 -- 172.21.15.69:0/3572296411 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f996801f020 con 0x7f99741044b0 2024-04-25T11:24:16.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.655+0000 7f99788cb700 1 -- 172.21.15.69:0/3572296411 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f997402e510 con 0x7f99741044b0 2024-04-25T11:24:16.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.660+0000 7f99637fe700 1 -- 172.21.15.69:0/3572296411 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+284479 (secure 0 0 0) 0x7f99680a7660 con 0x7f99741044b0 2024-04-25T11:24:16.658 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:24:16.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.664+0000 7f99788cb700 1 -- 172.21.15.69:0/3572296411 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f995c0afc10 msgr2=0x7f995c0b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:16.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.664+0000 7f99788cb700 1 --2- 172.21.15.69:0/3572296411 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f995c0afc10 0x7f995c0b2120 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f996400b580 tx=0x7f9964005fb0 comp rx=0 tx=0).stop 2024-04-25T11:24:16.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.664+0000 7f99788cb700 1 -- 172.21.15.69:0/3572296411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99741044b0 msgr2=0x7f9974076aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:16.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.664+0000 7f99788cb700 1 --2- 172.21.15.69:0/3572296411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99741044b0 0x7f9974076aa0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f996800b530 tx=0x7f996800b8f0 comp rx=0 tx=0).stop 2024-04-25T11:24:16.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.664+0000 7f99788cb700 1 -- 172.21.15.69:0/3572296411 shutdown_connections 2024-04-25T11:24:16.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.664+0000 7f99788cb700 1 --2- 172.21.15.69:0/3572296411 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f995c0afc10 0x7f995c0b2120 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:16.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.664+0000 7f99788cb700 1 --2- 172.21.15.69:0/3572296411 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9974109730 0x7f9974077520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:16.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.664+0000 7f99788cb700 1 --2- 172.21.15.69:0/3572296411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99741044b0 0x7f9974076aa0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:16.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.664+0000 7f99788cb700 1 --2- 172.21.15.69:0/3572296411 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9974108de0 0x7f9974076fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:16.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.664+0000 7f99788cb700 1 -- 172.21.15.69:0/3572296411 >> 172.21.15.69:0/3572296411 conn(0x7f99740ffdc0 msgr2=0x7f99741021e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:16.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.664+0000 7f99788cb700 1 -- 172.21.15.69:0/3572296411 shutdown_connections 2024-04-25T11:24:16.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:16.664+0000 7f99788cb700 1 -- 172.21.15.69:0/3572296411 wait complete. 2024-04-25T11:24:16.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:16 smithi069 ceph-mon[29188]: pgmap v302: 129 pgs: 129 active+clean; 628 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.4 KiB/s wr, 23 op/s 2024-04-25T11:24:16.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:16 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/3572296411' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T11:24:17.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:16 smithi146 ceph-mon[32821]: pgmap v302: 129 pgs: 129 active+clean; 628 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.4 KiB/s wr, 23 op/s 2024-04-25T11:24:17.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:16 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/3572296411' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T11:24:17.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:16 smithi152 ceph-mon[32701]: pgmap v302: 129 pgs: 129 active+clean; 628 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.4 KiB/s wr, 23 op/s 2024-04-25T11:24:17.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:16 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/3572296411' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T11:24:17.493 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":4220065409},{"type":"v1","addr":"172.21.15.69:6801","nonce":4220065409}]},"active_addr":"172.21.15.69:6801/4220065409","active_change":"2024-04-25T11:16:31.688322+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.69:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.69:0","nonce":2942075650}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.69:0","nonce":711059222}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.69:0","nonce":3467705820}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.69:0","nonce":1139337151}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.69:0","nonce":2572666992}]}]} 2024-04-25T11:24:17.500 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-25T11:24:17.500 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-25T11:24:17.500 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T11:24:17.738 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:24:18.942 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:18 smithi069 ceph-mon[29188]: pgmap v303: 129 pgs: 129 active+clean; 628 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.4 KiB/s wr, 23 op/s 2024-04-25T11:24:19.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:18 smithi146 ceph-mon[32821]: pgmap v303: 129 pgs: 129 active+clean; 628 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.4 KiB/s wr, 23 op/s 2024-04-25T11:24:19.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:18 smithi152 ceph-mon[32701]: pgmap v303: 129 pgs: 129 active+clean; 628 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.4 KiB/s wr, 23 op/s 2024-04-25T11:24:19.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.547+0000 7f425501a700 1 -- 172.21.15.69:0/1449074517 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4250108640 msgr2=0x7f425010ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:19.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.547+0000 7f425501a700 1 --2- 172.21.15.69:0/1449074517 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4250108640 0x7f425010ffa0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f424800b0d0 tx=0x7f424800b3e0 comp rx=0 tx=0).stop 2024-04-25T11:24:19.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.547+0000 7f425501a700 1 -- 172.21.15.69:0/1449074517 shutdown_connections 2024-04-25T11:24:19.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.547+0000 7f425501a700 1 --2- 172.21.15.69:0/1449074517 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4250108640 0x7f425010ffa0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:19.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.547+0000 7f425501a700 1 --2- 172.21.15.69:0/1449074517 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42501033c0 0x7f42501037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:19.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.547+0000 7f425501a700 1 --2- 172.21.15.69:0/1449074517 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4250107cf0 0x7f4250108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:19.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.547+0000 7f425501a700 1 -- 172.21.15.69:0/1449074517 >> 172.21.15.69:0/1449074517 conn(0x7f42500fedb0 msgr2=0x7f42501011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:19.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.547+0000 7f425501a700 1 -- 172.21.15.69:0/1449074517 shutdown_connections 2024-04-25T11:24:19.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.547+0000 7f425501a700 1 -- 172.21.15.69:0/1449074517 wait complete. 2024-04-25T11:24:19.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.548+0000 7f425501a700 1 Processor -- start 2024-04-25T11:24:19.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.548+0000 7f425501a700 1 -- start start 2024-04-25T11:24:19.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.549+0000 7f425501a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42501033c0 0x7f425019f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:19.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.549+0000 7f425501a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4250107cf0 0x7f425019fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:19.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.549+0000 7f425501a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4250108640 0x7f42501998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:19.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.549+0000 7f425501a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4250113120 con 0x7f42501033c0 2024-04-25T11:24:19.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.549+0000 7f425501a700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4250112fa0 con 0x7f4250107cf0 2024-04-25T11:24:19.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.549+0000 7f425501a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f42501132a0 con 0x7f4250108640 2024-04-25T11:24:19.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.549+0000 7f424ed9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42501033c0 0x7f425019f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:19.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.549+0000 7f424ed9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42501033c0 0x7f425019f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54876/0 (socket says 172.21.15.69:54876) 2024-04-25T11:24:19.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.549+0000 7f424ed9d700 1 -- 172.21.15.69:0/1650910746 learned_addr learned my addr 172.21.15.69:0/1650910746 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:24:19.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.549+0000 7f424e59c700 1 --2- 172.21.15.69:0/1650910746 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4250107cf0 0x7f425019fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:19.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.549+0000 7f424f59e700 1 --2- 172.21.15.69:0/1650910746 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4250108640 0x7f42501998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:19.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.549+0000 7f424ed9d700 1 -- 172.21.15.69:0/1650910746 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4250108640 msgr2=0x7f42501998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:19.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.549+0000 7f424ed9d700 1 --2- 172.21.15.69:0/1650910746 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4250108640 0x7f42501998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:19.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.549+0000 7f424ed9d700 1 -- 172.21.15.69:0/1650910746 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4250107cf0 msgr2=0x7f425019fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:19.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.549+0000 7f424ed9d700 1 --2- 172.21.15.69:0/1650910746 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4250107cf0 0x7f425019fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:19.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.549+0000 7f424ed9d700 1 -- 172.21.15.69:0/1650910746 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4250199f70 con 0x7f42501033c0 2024-04-25T11:24:19.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.549+0000 7f424e59c700 1 --2- 172.21.15.69:0/1650910746 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4250107cf0 0x7f425019fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:24:19.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.549+0000 7f424f59e700 1 --2- 172.21.15.69:0/1650910746 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4250108640 0x7f42501998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:24:19.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.549+0000 7f424ed9d700 1 --2- 172.21.15.69:0/1650910746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42501033c0 0x7f425019f6a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f424000c920 tx=0x7f424000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:19.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.549+0000 7f4237fff700 1 -- 172.21.15.69:0/1650910746 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4240007b00 con 0x7f42501033c0 2024-04-25T11:24:19.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.549+0000 7f425501a700 1 -- 172.21.15.69:0/1650910746 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f425019a260 con 0x7f42501033c0 2024-04-25T11:24:19.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.549+0000 7f425501a700 1 -- 172.21.15.69:0/1650910746 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42501a6530 con 0x7f42501033c0 2024-04-25T11:24:19.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.550+0000 7f4237fff700 1 -- 172.21.15.69:0/1650910746 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4240004cc0 con 0x7f42501033c0 2024-04-25T11:24:19.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.550+0000 7f4237fff700 1 -- 172.21.15.69:0/1650910746 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f424001d060 con 0x7f42501033c0 2024-04-25T11:24:19.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.551+0000 7f425501a700 1 -- 172.21.15.69:0/1650910746 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4250051190 con 0x7f42501033c0 2024-04-25T11:24:19.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.552+0000 7f4237fff700 1 -- 172.21.15.69:0/1650910746 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f4240007ca0 con 0x7f42501033c0 2024-04-25T11:24:19.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.553+0000 7f4237fff700 1 --2- 172.21.15.69:0/1650910746 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f42380b1e20 0x7f42380b4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:19.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.553+0000 7f424e59c700 1 --2- 172.21.15.69:0/1650910746 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f42380b1e20 0x7f42380b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:19.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.553+0000 7f4237fff700 1 -- 172.21.15.69:0/1650910746 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7f42400de8b0 con 0x7f42501033c0 2024-04-25T11:24:19.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.553+0000 7f424e59c700 1 --2- 172.21.15.69:0/1650910746 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f42380b1e20 0x7f42380b4330 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4244005e10 tx=0x7f4244005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:19.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.557+0000 7f4237fff700 1 -- 172.21.15.69:0/1650910746 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4240014030 con 0x7f42501033c0 2024-04-25T11:24:19.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.756+0000 7f425501a700 1 -- 172.21.15.69:0/1650910746 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f425010b0d0 con 0x7f42501033c0 2024-04-25T11:24:19.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.758+0000 7f4237fff700 1 -- 172.21.15.69:0/1650910746 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+24130 (secure 0 0 0) 0x7f42400a70f0 con 0x7f42501033c0 2024-04-25T11:24:19.752 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:24:19.752 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":80,"fsid":"f7ed73dc-02f4-11ef-bc93-c7b262605968","created":"2024-04-25T11:15:37.965281+0000","modified":"2024-04-25T11:23:47.662955+0000","last_up_change":"2024-04-25T11:22:50.691869+0000","last_in_change":"2024-04-25T11:22:30.271959+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:18:57.781263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:23:38.685110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:23:39.943803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T11:23:43.650521+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6a0d179-4998-4754-bf82-2ea3dac54e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6803","nonce":3756025235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6805","nonce":3756025235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6809","nonce":3756025235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6807","nonce":3756025235}]},"public_addr":"172.21.15.69:6803/3756025235","cluster_addr":"172.21.15.69:6805/3756025235","heartbeat_back_addr":"172.21.15.69:6809/3756025235","heartbeat_front_addr":"172.21.15.69:6807/3756025235","state":["exists","up"]},{"osd":1,"uuid":"c0c66930-4233-493f-b2bb-99c473994b59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6801","nonce":3166305213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6803","nonce":3166305213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6807","nonce":3166305213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6805","nonce":3166305213}]},"public_addr":"172.21.15.146:6801/3166305213","cluster_addr":"172.21.15.146:6803/3166305213","heartbeat_back_addr":"172.21.15.146:6807/3166305213","heartbeat_front_addr":"172.21.15.146:6805/3166305213","state":["exists","up"]},{"osd":2,"uuid":"d4ef735b-9786-4cb9-83ef-7c3f0691a729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6801","nonce":1474319175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6803","nonce":1474319175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6807","nonce":1474319175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6805","nonce":1474319175}]},"public_addr":"172.21.15.152:6801/1474319175","cluster_addr":"172.21.15.152:6803/1474319175","heartbeat_back_addr":"172.21.15.152:6807/1474319175","heartbeat_front_addr":"172.21.15.152:6805/1474319175","state":["exists","up"]},{"osd":3,"uuid":"0ab85fb8-5bbb-4ffc-bddf-cae7644a64ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6811","nonce":1809071296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6813","nonce":1809071296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6817","nonce":1809071296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6815","nonce":1809071296}]},"public_addr":"172.21.15.69:6811/1809071296","cluster_addr":"172.21.15.69:6813/1809071296","heartbeat_back_addr":"172.21.15.69:6817/1809071296","heartbeat_front_addr":"172.21.15.69:6815/1809071296","state":["exists","up"]},{"osd":4,"uuid":"273e0592-bbbc-4188-8eeb-46a3879cf632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6809","nonce":1898520972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6811","nonce":1898520972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6815","nonce":1898520972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6813","nonce":1898520972}]},"public_addr":"172.21.15.146:6809/1898520972","cluster_addr":"172.21.15.146:6811/1898520972","heartbeat_back_addr":"172.21.15.146:6815/1898520972","heartbeat_front_addr":"172.21.15.146:6813/1898520972","state":["exists","up"]},{"osd":5,"uuid":"bbeae945-9fa2-48f9-a1d6-8bf18dcaef19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6809","nonce":1996555108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6811","nonce":1996555108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6815","nonce":1996555108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6813","nonce":1996555108}]},"public_addr":"172.21.15.152:6809/1996555108","cluster_addr":"172.21.15.152:6811/1996555108","heartbeat_back_addr":"172.21.15.152:6815/1996555108","heartbeat_front_addr":"172.21.15.152:6813/1996555108","state":["exists","up"]},{"osd":6,"uuid":"dbb61253-30bc-418e-bd75-118c80bbf931","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6819","nonce":491215762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6821","nonce":491215762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6825","nonce":491215762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6823","nonce":491215762}]},"public_addr":"172.21.15.69:6819/491215762","cluster_addr":"172.21.15.69:6821/491215762","heartbeat_back_addr":"172.21.15.69:6825/491215762","heartbeat_front_addr":"172.21.15.69:6823/491215762","state":["exists","up"]},{"osd":7,"uuid":"75f23854-e283-4e3e-94cf-7b4079d83876","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6817","nonce":2330294690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6819","nonce":2330294690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6823","nonce":2330294690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6821","nonce":2330294690}]},"public_addr":"172.21.15.146:6817/2330294690","cluster_addr":"172.21.15.146:6819/2330294690","heartbeat_back_addr":"172.21.15.146:6823/2330294690","heartbeat_front_addr":"172.21.15.146:6821/2330294690","state":["exists","up"]},{"osd":8,"uuid":"1aa1060c-0c14-4abd-be56-5ebfeb78759c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6817","nonce":950669381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6819","nonce":950669381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6823","nonce":950669381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6821","nonce":950669381}]},"public_addr":"172.21.15.152:6817/950669381","cluster_addr":"172.21.15.152:6819/950669381","heartbeat_back_addr":"172.21.15.152:6823/950669381","heartbeat_front_addr":"172.21.15.152:6821/950669381","state":["exists","up"]},{"osd":9,"uuid":"3f4bf800-4090-4138-9bb3-d001f13f450c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6827","nonce":2414186758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6829","nonce":2414186758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6833","nonce":2414186758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6831","nonce":2414186758}]},"public_addr":"172.21.15.69:6827/2414186758","cluster_addr":"172.21.15.69:6829/2414186758","heartbeat_back_addr":"172.21.15.69:6833/2414186758","heartbeat_front_addr":"172.21.15.69:6831/2414186758","state":["exists","up"]},{"osd":10,"uuid":"19e967e5-51ce-4506-80c9-0a95441baf99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6825","nonce":707256400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6827","nonce":707256400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6831","nonce":707256400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6829","nonce":707256400}]},"public_addr":"172.21.15.146:6825/707256400","cluster_addr":"172.21.15.146:6827/707256400","heartbeat_back_addr":"172.21.15.146:6831/707256400","heartbeat_front_addr":"172.21.15.146:6829/707256400","state":["exists","up"]},{"osd":11,"uuid":"7788c2de-6930-4808-be0f-f1fd1ae0cb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6825","nonce":2934336833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6827","nonce":2934336833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6831","nonce":2934336833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6829","nonce":2934336833}]},"public_addr":"172.21.15.152:6825/2934336833","cluster_addr":"172.21.15.152:6827/2934336833","heartbeat_back_addr":"172.21.15.152:6831/2934336833","heartbeat_front_addr":"172.21.15.152:6829/2934336833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:07.497448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:31.227494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:55.027978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:20.713437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:46.214704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:20:11.199983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:20:37.177070+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:02.951240+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:28.963680+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:56.095807+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:22:22.221744+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:22:48.929334+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/1372727546":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6801/1097124842":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6800/1508198215":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3103465990":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3540132960":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3899912190":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6800/1097124842":"2024-04-26T11:16:31.688159+0000","172.21.15.69:0/1199907202":"2024-04-26T11:16:04.860792+0000","172.21.15.69:6801/1508198215":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/2879102529":"2024-04-26T11:16:31.688159+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:24:19.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.760+0000 7f425501a700 1 -- 172.21.15.69:0/1650910746 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f42380b1e20 msgr2=0x7f42380b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:19.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.760+0000 7f425501a700 1 --2- 172.21.15.69:0/1650910746 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f42380b1e20 0x7f42380b4330 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4244005e10 tx=0x7f4244005d60 comp rx=0 tx=0).stop 2024-04-25T11:24:19.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.760+0000 7f425501a700 1 -- 172.21.15.69:0/1650910746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42501033c0 msgr2=0x7f425019f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:19.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.760+0000 7f425501a700 1 --2- 172.21.15.69:0/1650910746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42501033c0 0x7f425019f6a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f424000c920 tx=0x7f424000cce0 comp rx=0 tx=0).stop 2024-04-25T11:24:19.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.760+0000 7f425501a700 1 -- 172.21.15.69:0/1650910746 shutdown_connections 2024-04-25T11:24:19.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.760+0000 7f425501a700 1 --2- 172.21.15.69:0/1650910746 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f42380b1e20 0x7f42380b4330 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:19.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.760+0000 7f425501a700 1 --2- 172.21.15.69:0/1650910746 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4250108640 0x7f42501998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:19.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.760+0000 7f425501a700 1 --2- 172.21.15.69:0/1650910746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42501033c0 0x7f425019f6a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:19.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.760+0000 7f425501a700 1 --2- 172.21.15.69:0/1650910746 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4250107cf0 0x7f425019fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:19.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.760+0000 7f425501a700 1 -- 172.21.15.69:0/1650910746 >> 172.21.15.69:0/1650910746 conn(0x7f42500fedb0 msgr2=0x7f4250107410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:19.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.761+0000 7f425501a700 1 -- 172.21.15.69:0/1650910746 shutdown_connections 2024-04-25T11:24:19.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:19.761+0000 7f425501a700 1 -- 172.21.15.69:0/1650910746 wait complete. 2024-04-25T11:24:20.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:19 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/1650910746' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:24:20.154 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:19 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/1650910746' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:24:20.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:19 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/1650910746' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:24:20.582 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-25T11:24:20.582 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T11:24:20.822 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:24:21.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:20 smithi146 ceph-mon[32821]: pgmap v304: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.5 KiB/s wr, 27 op/s 2024-04-25T11:24:21.192 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:20 smithi069 ceph-mon[29188]: pgmap v304: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.5 KiB/s wr, 27 op/s 2024-04-25T11:24:21.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:20 smithi152 ceph-mon[32701]: pgmap v304: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.5 KiB/s wr, 27 op/s 2024-04-25T11:24:22.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.615+0000 7f67cd64f700 1 -- 172.21.15.69:0/2009436645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67c8103390 msgr2=0x7f67c81037a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:22.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.615+0000 7f67cd64f700 1 --2- 172.21.15.69:0/2009436645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67c8103390 0x7f67c81037a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f67b80099c0 tx=0x7f67b8009cd0 comp rx=0 tx=0).stop 2024-04-25T11:24:22.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.615+0000 7f67cd64f700 1 -- 172.21.15.69:0/2009436645 shutdown_connections 2024-04-25T11:24:22.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.615+0000 7f67cd64f700 1 --2- 172.21.15.69:0/2009436645 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f67c8108610 0x7f67c810ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:22.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.615+0000 7f67cd64f700 1 --2- 172.21.15.69:0/2009436645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67c8103390 0x7f67c81037a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:22.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.615+0000 7f67cd64f700 1 --2- 172.21.15.69:0/2009436645 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67c8107cc0 0x7f67c81080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:22.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.615+0000 7f67cd64f700 1 -- 172.21.15.69:0/2009436645 >> 172.21.15.69:0/2009436645 conn(0x7f67c80fec90 msgr2=0x7f67c81010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:22.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.615+0000 7f67cd64f700 1 -- 172.21.15.69:0/2009436645 shutdown_connections 2024-04-25T11:24:22.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.615+0000 7f67cd64f700 1 -- 172.21.15.69:0/2009436645 wait complete. 2024-04-25T11:24:22.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.616+0000 7f67cd64f700 1 Processor -- start 2024-04-25T11:24:22.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.616+0000 7f67cd64f700 1 -- start start 2024-04-25T11:24:22.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.616+0000 7f67cd64f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67c8103390 0x7f67c819f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:22.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.616+0000 7f67cd64f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67c8107cc0 0x7f67c819fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:22.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.617+0000 7f67cd64f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f67c8108610 0x7f67c8199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:22.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.617+0000 7f67cd64f700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f67c81130f0 con 0x7f67c8103390 2024-04-25T11:24:22.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.617+0000 7f67cd64f700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f67c8112f70 con 0x7f67c8108610 2024-04-25T11:24:22.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.617+0000 7f67cd64f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f67c8113270 con 0x7f67c8107cc0 2024-04-25T11:24:22.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.617+0000 7f67c6ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67c8103390 0x7f67c819f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:22.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.617+0000 7f67c6ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67c8103390 0x7f67c819f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35006/0 (socket says 172.21.15.69:35006) 2024-04-25T11:24:22.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.617+0000 7f67c6ffd700 1 -- 172.21.15.69:0/2762725589 learned_addr learned my addr 172.21.15.69:0/2762725589 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:24:22.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.617+0000 7f67c67fc700 1 --2- 172.21.15.69:0/2762725589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67c8107cc0 0x7f67c819fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:22.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.617+0000 7f67c77fe700 1 --2- 172.21.15.69:0/2762725589 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f67c8108610 0x7f67c8199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:22.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.617+0000 7f67c6ffd700 1 -- 172.21.15.69:0/2762725589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67c8107cc0 msgr2=0x7f67c819fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:22.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.617+0000 7f67c6ffd700 1 --2- 172.21.15.69:0/2762725589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67c8107cc0 0x7f67c819fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:22.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.617+0000 7f67c6ffd700 1 -- 172.21.15.69:0/2762725589 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f67c8108610 msgr2=0x7f67c8199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:22.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.617+0000 7f67c6ffd700 1 --2- 172.21.15.69:0/2762725589 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f67c8108610 0x7f67c8199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:22.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.617+0000 7f67c6ffd700 1 -- 172.21.15.69:0/2762725589 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67c8199fc0 con 0x7f67c8103390 2024-04-25T11:24:22.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.617+0000 7f67c6ffd700 1 --2- 172.21.15.69:0/2762725589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67c8103390 0x7f67c819f6f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f67b8005f50 tx=0x7f67b80057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:22.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.617+0000 7f67affff700 1 -- 172.21.15.69:0/2762725589 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b8027070 con 0x7f67c8103390 2024-04-25T11:24:22.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.617+0000 7f67affff700 1 -- 172.21.15.69:0/2762725589 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f67b800fd60 con 0x7f67c8103390 2024-04-25T11:24:22.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.617+0000 7f67affff700 1 -- 172.21.15.69:0/2762725589 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b8022e30 con 0x7f67c8103390 2024-04-25T11:24:22.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.617+0000 7f67cd64f700 1 -- 172.21.15.69:0/2762725589 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67c819a250 con 0x7f67c8103390 2024-04-25T11:24:22.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.617+0000 7f67cd64f700 1 -- 172.21.15.69:0/2762725589 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67c819a730 con 0x7f67c8103390 2024-04-25T11:24:22.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.618+0000 7f67cd64f700 1 -- 172.21.15.69:0/2762725589 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67c8051190 con 0x7f67c8103390 2024-04-25T11:24:22.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.620+0000 7f67affff700 1 -- 172.21.15.69:0/2762725589 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f67b802a070 con 0x7f67c8103390 2024-04-25T11:24:22.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.620+0000 7f67affff700 1 --2- 172.21.15.69:0/2762725589 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f67b00afc10 0x7f67b00b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:22.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.621+0000 7f67c67fc700 1 --2- 172.21.15.69:0/2762725589 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f67b00afc10 0x7f67b00b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:22.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.621+0000 7f67affff700 1 -- 172.21.15.69:0/2762725589 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8802+0+0 (secure 0 0 0) 0x7f67b80dff20 con 0x7f67c8103390 2024-04-25T11:24:22.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.621+0000 7f67c67fc700 1 --2- 172.21.15.69:0/2762725589 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f67b00afc10 0x7f67b00b2120 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f67bc005fd0 tx=0x7f67bc005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:22.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.624+0000 7f67affff700 1 -- 172.21.15.69:0/2762725589 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f67b8013070 con 0x7f67c8103390 2024-04-25T11:24:22.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.821+0000 7f67cd64f700 1 -- 172.21.15.69:0/2762725589 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f67c810afb0 con 0x7f67c8103390 2024-04-25T11:24:22.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.823+0000 7f67affff700 1 -- 172.21.15.69:0/2762725589 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+24130 (secure 0 0 0) 0x7f67b80a87f0 con 0x7f67c8103390 2024-04-25T11:24:22.817 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:24:22.818 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":80,"fsid":"f7ed73dc-02f4-11ef-bc93-c7b262605968","created":"2024-04-25T11:15:37.965281+0000","modified":"2024-04-25T11:23:47.662955+0000","last_up_change":"2024-04-25T11:22:50.691869+0000","last_in_change":"2024-04-25T11:22:30.271959+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:18:57.781263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:23:38.685110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:23:39.943803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T11:23:43.650521+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6a0d179-4998-4754-bf82-2ea3dac54e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6803","nonce":3756025235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6805","nonce":3756025235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6809","nonce":3756025235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6807","nonce":3756025235}]},"public_addr":"172.21.15.69:6803/3756025235","cluster_addr":"172.21.15.69:6805/3756025235","heartbeat_back_addr":"172.21.15.69:6809/3756025235","heartbeat_front_addr":"172.21.15.69:6807/3756025235","state":["exists","up"]},{"osd":1,"uuid":"c0c66930-4233-493f-b2bb-99c473994b59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6801","nonce":3166305213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6803","nonce":3166305213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6807","nonce":3166305213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6805","nonce":3166305213}]},"public_addr":"172.21.15.146:6801/3166305213","cluster_addr":"172.21.15.146:6803/3166305213","heartbeat_back_addr":"172.21.15.146:6807/3166305213","heartbeat_front_addr":"172.21.15.146:6805/3166305213","state":["exists","up"]},{"osd":2,"uuid":"d4ef735b-9786-4cb9-83ef-7c3f0691a729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6801","nonce":1474319175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6803","nonce":1474319175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6807","nonce":1474319175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6805","nonce":1474319175}]},"public_addr":"172.21.15.152:6801/1474319175","cluster_addr":"172.21.15.152:6803/1474319175","heartbeat_back_addr":"172.21.15.152:6807/1474319175","heartbeat_front_addr":"172.21.15.152:6805/1474319175","state":["exists","up"]},{"osd":3,"uuid":"0ab85fb8-5bbb-4ffc-bddf-cae7644a64ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6811","nonce":1809071296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6813","nonce":1809071296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6817","nonce":1809071296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6815","nonce":1809071296}]},"public_addr":"172.21.15.69:6811/1809071296","cluster_addr":"172.21.15.69:6813/1809071296","heartbeat_back_addr":"172.21.15.69:6817/1809071296","heartbeat_front_addr":"172.21.15.69:6815/1809071296","state":["exists","up"]},{"osd":4,"uuid":"273e0592-bbbc-4188-8eeb-46a3879cf632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6809","nonce":1898520972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6811","nonce":1898520972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6815","nonce":1898520972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6813","nonce":1898520972}]},"public_addr":"172.21.15.146:6809/1898520972","cluster_addr":"172.21.15.146:6811/1898520972","heartbeat_back_addr":"172.21.15.146:6815/1898520972","heartbeat_front_addr":"172.21.15.146:6813/1898520972","state":["exists","up"]},{"osd":5,"uuid":"bbeae945-9fa2-48f9-a1d6-8bf18dcaef19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6809","nonce":1996555108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6811","nonce":1996555108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6815","nonce":1996555108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6813","nonce":1996555108}]},"public_addr":"172.21.15.152:6809/1996555108","cluster_addr":"172.21.15.152:6811/1996555108","heartbeat_back_addr":"172.21.15.152:6815/1996555108","heartbeat_front_addr":"172.21.15.152:6813/1996555108","state":["exists","up"]},{"osd":6,"uuid":"dbb61253-30bc-418e-bd75-118c80bbf931","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6819","nonce":491215762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6821","nonce":491215762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6825","nonce":491215762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6823","nonce":491215762}]},"public_addr":"172.21.15.69:6819/491215762","cluster_addr":"172.21.15.69:6821/491215762","heartbeat_back_addr":"172.21.15.69:6825/491215762","heartbeat_front_addr":"172.21.15.69:6823/491215762","state":["exists","up"]},{"osd":7,"uuid":"75f23854-e283-4e3e-94cf-7b4079d83876","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6817","nonce":2330294690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6819","nonce":2330294690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6823","nonce":2330294690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6821","nonce":2330294690}]},"public_addr":"172.21.15.146:6817/2330294690","cluster_addr":"172.21.15.146:6819/2330294690","heartbeat_back_addr":"172.21.15.146:6823/2330294690","heartbeat_front_addr":"172.21.15.146:6821/2330294690","state":["exists","up"]},{"osd":8,"uuid":"1aa1060c-0c14-4abd-be56-5ebfeb78759c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6817","nonce":950669381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6819","nonce":950669381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6823","nonce":950669381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6821","nonce":950669381}]},"public_addr":"172.21.15.152:6817/950669381","cluster_addr":"172.21.15.152:6819/950669381","heartbeat_back_addr":"172.21.15.152:6823/950669381","heartbeat_front_addr":"172.21.15.152:6821/950669381","state":["exists","up"]},{"osd":9,"uuid":"3f4bf800-4090-4138-9bb3-d001f13f450c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6827","nonce":2414186758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6829","nonce":2414186758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6833","nonce":2414186758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6831","nonce":2414186758}]},"public_addr":"172.21.15.69:6827/2414186758","cluster_addr":"172.21.15.69:6829/2414186758","heartbeat_back_addr":"172.21.15.69:6833/2414186758","heartbeat_front_addr":"172.21.15.69:6831/2414186758","state":["exists","up"]},{"osd":10,"uuid":"19e967e5-51ce-4506-80c9-0a95441baf99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6825","nonce":707256400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6827","nonce":707256400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6831","nonce":707256400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6829","nonce":707256400}]},"public_addr":"172.21.15.146:6825/707256400","cluster_addr":"172.21.15.146:6827/707256400","heartbeat_back_addr":"172.21.15.146:6831/707256400","heartbeat_front_addr":"172.21.15.146:6829/707256400","state":["exists","up"]},{"osd":11,"uuid":"7788c2de-6930-4808-be0f-f1fd1ae0cb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6825","nonce":2934336833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6827","nonce":2934336833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6831","nonce":2934336833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6829","nonce":2934336833}]},"public_addr":"172.21.15.152:6825/2934336833","cluster_addr":"172.21.15.152:6827/2934336833","heartbeat_back_addr":"172.21.15.152:6831/2934336833","heartbeat_front_addr":"172.21.15.152:6829/2934336833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:07.497448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:31.227494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:55.027978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:20.713437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:46.214704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:20:11.199983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:20:37.177070+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:02.951240+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:28.963680+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:56.095807+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:22:22.221744+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:22:48.929334+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/1372727546":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6801/1097124842":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6800/1508198215":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3103465990":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3540132960":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3899912190":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6800/1097124842":"2024-04-26T11:16:31.688159+0000","172.21.15.69:0/1199907202":"2024-04-26T11:16:04.860792+0000","172.21.15.69:6801/1508198215":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/2879102529":"2024-04-26T11:16:31.688159+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:24:22.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.825+0000 7f67cd64f700 1 -- 172.21.15.69:0/2762725589 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f67b00afc10 msgr2=0x7f67b00b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:22.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.825+0000 7f67cd64f700 1 --2- 172.21.15.69:0/2762725589 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f67b00afc10 0x7f67b00b2120 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f67bc005fd0 tx=0x7f67bc005f60 comp rx=0 tx=0).stop 2024-04-25T11:24:22.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.825+0000 7f67cd64f700 1 -- 172.21.15.69:0/2762725589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67c8103390 msgr2=0x7f67c819f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:22.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.825+0000 7f67cd64f700 1 --2- 172.21.15.69:0/2762725589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67c8103390 0x7f67c819f6f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f67b8005f50 tx=0x7f67b80057c0 comp rx=0 tx=0).stop 2024-04-25T11:24:22.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.825+0000 7f67cd64f700 1 -- 172.21.15.69:0/2762725589 shutdown_connections 2024-04-25T11:24:22.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.825+0000 7f67cd64f700 1 --2- 172.21.15.69:0/2762725589 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f67b00afc10 0x7f67b00b2120 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:22.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.825+0000 7f67cd64f700 1 --2- 172.21.15.69:0/2762725589 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f67c8108610 0x7f67c8199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:22.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.825+0000 7f67cd64f700 1 --2- 172.21.15.69:0/2762725589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67c8103390 0x7f67c819f6f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:22.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.825+0000 7f67cd64f700 1 --2- 172.21.15.69:0/2762725589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67c8107cc0 0x7f67c819fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:22.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.825+0000 7f67cd64f700 1 -- 172.21.15.69:0/2762725589 >> 172.21.15.69:0/2762725589 conn(0x7f67c80fec90 msgr2=0x7f67c8101c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:22.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.825+0000 7f67cd64f700 1 -- 172.21.15.69:0/2762725589 shutdown_connections 2024-04-25T11:24:22.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:22.825+0000 7f67cd64f700 1 -- 172.21.15.69:0/2762725589 wait complete. 2024-04-25T11:24:22.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:22 smithi069 ceph-mon[29188]: pgmap v305: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 KiB/s wr, 16 op/s 2024-04-25T11:24:22.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:22 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2762725589' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:24:23.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:22 smithi146 ceph-mon[32821]: pgmap v305: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 KiB/s wr, 16 op/s 2024-04-25T11:24:23.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:22 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/2762725589' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:24:23.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:22 smithi152 ceph-mon[32701]: pgmap v305: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 KiB/s wr, 16 op/s 2024-04-25T11:24:23.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:22 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/2762725589' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:24:23.675 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-25T11:24:23.675 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-25T11:24:23.676 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-25T11:24:23.676 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-25T11:24:23.676 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-25T11:24:23.677 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-25T11:24:23.677 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-25T11:24:23.678 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-25T11:24:23.678 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-25T11:24:23.678 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-25T11:24:23.679 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-25T11:24:23.679 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-25T11:24:24.504 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:24:24.509 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:24:24.509 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:24:24.510 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:24:24.511 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:24:24.513 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:24:24.515 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:24:24.515 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:24:24.517 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:24:24.517 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:24:24.523 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:24:24.566 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:24:24.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:24 smithi069 ceph-mon[29188]: pgmap v306: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 3.7 KiB/s wr, 12 op/s 2024-04-25T11:24:25.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:24 smithi152 ceph-mon[32701]: pgmap v306: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 3.7 KiB/s wr, 12 op/s 2024-04-25T11:24:25.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:24 smithi146 ceph-mon[32821]: pgmap v306: 129 pgs: 129 active+clean; 639 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 3.7 KiB/s wr, 12 op/s 2024-04-25T11:24:27.104 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:26 smithi069 ceph-mon[29188]: pgmap v307: 129 pgs: 129 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-04-25T11:24:27.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:26 smithi152 ceph-mon[32701]: pgmap v307: 129 pgs: 129 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-04-25T11:24:27.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:26 smithi146 ceph-mon[32821]: pgmap v307: 129 pgs: 129 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-04-25T11:24:29.129 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:28 smithi069 ceph-mon[29188]: pgmap v308: 129 pgs: 129 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-25T11:24:29.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:28 smithi152 ceph-mon[32701]: pgmap v308: 129 pgs: 129 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-25T11:24:29.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:28 smithi146 ceph-mon[32821]: pgmap v308: 129 pgs: 129 active+clean; 642 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-25T11:24:31.162 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:30 smithi069 ceph-mon[29188]: pgmap v309: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-04-25T11:24:31.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:30 smithi152 ceph-mon[32701]: pgmap v309: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-04-25T11:24:31.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:30 smithi146 ceph-mon[32821]: pgmap v309: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-04-25T11:24:32.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:31 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-04-25T11:24:32.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:31 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-04-25T11:24:32.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:31 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-04-25T11:24:32.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:31 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [5, 11]}]: dispatch 2024-04-25T11:24:32.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:31 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-04-25T11:24:32.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:31 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-04-25T11:24:32.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:31 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]: dispatch 2024-04-25T11:24:32.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:31 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 10]}]: dispatch 2024-04-25T11:24:32.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:31 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-04-25T11:24:32.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:31 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-04-25T11:24:32.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:31 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-04-25T11:24:32.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:31 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-04-25T11:24:32.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:31 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-04-25T11:24:32.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:31 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [5, 11]}]: dispatch 2024-04-25T11:24:32.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:31 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-04-25T11:24:32.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:31 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-04-25T11:24:32.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:31 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]: dispatch 2024-04-25T11:24:32.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:31 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 10]}]: dispatch 2024-04-25T11:24:32.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:31 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-04-25T11:24:32.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:31 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-04-25T11:24:32.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:31 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-04-25T11:24:32.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:31 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-04-25T11:24:32.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:31 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-04-25T11:24:32.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:31 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [5, 11]}]: dispatch 2024-04-25T11:24:32.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:31 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-04-25T11:24:32.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:31 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-04-25T11:24:32.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:31 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]: dispatch 2024-04-25T11:24:32.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:31 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 10]}]: dispatch 2024-04-25T11:24:32.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:31 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-04-25T11:24:32.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:31 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-04-25T11:24:33.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:32 smithi069 ceph-mon[29188]: pgmap v310: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-25T11:24:33.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:32 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-04-25T11:24:33.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:32 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-04-25T11:24:33.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:32 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-04-25T11:24:33.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:32 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [5, 11]}]': finished 2024-04-25T11:24:33.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:32 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-04-25T11:24:33.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:32 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-04-25T11:24:33.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:32 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]': finished 2024-04-25T11:24:33.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:32 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 10]}]': finished 2024-04-25T11:24:33.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:32 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-04-25T11:24:33.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:32 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-04-25T11:24:33.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:32 smithi069 ceph-mon[29188]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T11:24:33.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:32 smithi152 ceph-mon[32701]: pgmap v310: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-25T11:24:33.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:32 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-04-25T11:24:33.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:32 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-04-25T11:24:33.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:32 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-04-25T11:24:33.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:32 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [5, 11]}]': finished 2024-04-25T11:24:33.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:32 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-04-25T11:24:33.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:32 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-04-25T11:24:33.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:32 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]': finished 2024-04-25T11:24:33.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:32 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 10]}]': finished 2024-04-25T11:24:33.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:32 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-04-25T11:24:33.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:32 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-04-25T11:24:33.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:32 smithi152 ceph-mon[32701]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T11:24:33.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:32 smithi146 ceph-mon[32821]: pgmap v310: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-25T11:24:33.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:32 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-04-25T11:24:33.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:32 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-04-25T11:24:33.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:32 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-04-25T11:24:33.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:32 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [5, 11]}]': finished 2024-04-25T11:24:33.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:32 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-04-25T11:24:33.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:32 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-04-25T11:24:33.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:32 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]': finished 2024-04-25T11:24:33.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:32 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 10]}]': finished 2024-04-25T11:24:33.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:32 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-04-25T11:24:33.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:32 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-04-25T11:24:33.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:32 smithi146 ceph-mon[32821]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T11:24:34.154 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:33 smithi069 ceph-mon[29188]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T11:24:34.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:33 smithi152 ceph-mon[32701]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T11:24:34.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:33 smithi146 ceph-mon[32821]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T11:24:34.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:34 smithi069 ceph-mon[29188]: pgmap v313: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T11:24:35.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:34 smithi152 ceph-mon[32701]: pgmap v313: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T11:24:35.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:34 smithi146 ceph-mon[32821]: pgmap v313: 129 pgs: 129 active+clean; 644 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T11:24:36.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.225+0000 7fca9731c700 1 -- 172.21.15.69:0/593814978 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca90102860 msgr2=0x7fca90102c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:36.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.225+0000 7fca9731c700 1 --2- 172.21.15.69:0/593814978 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca90102860 0x7fca90102c70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fca80009990 tx=0x7fca80009ca0 comp rx=0 tx=0).stop 2024-04-25T11:24:36.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.225+0000 7fca9731c700 1 -- 172.21.15.69:0/593814978 shutdown_connections 2024-04-25T11:24:36.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.225+0000 7fca9731c700 1 --2- 172.21.15.69:0/593814978 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca901031b0 0x7fca9010f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:36.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.225+0000 7fca9731c700 1 --2- 172.21.15.69:0/593814978 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca90108890 0x7fca90108ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:36.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.225+0000 7fca9731c700 1 --2- 172.21.15.69:0/593814978 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca90102860 0x7fca90102c70 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:36.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.225+0000 7fca9731c700 1 -- 172.21.15.69:0/593814978 >> 172.21.15.69:0/593814978 conn(0x7fca900fe380 msgr2=0x7fca901007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:36.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.225+0000 7fca9731c700 1 -- 172.21.15.69:0/593814978 shutdown_connections 2024-04-25T11:24:36.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.225+0000 7fca9731c700 1 -- 172.21.15.69:0/593814978 wait complete. 2024-04-25T11:24:36.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.226+0000 7fca9731c700 1 Processor -- start 2024-04-25T11:24:36.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.226+0000 7fca9731c700 1 -- start start 2024-04-25T11:24:36.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.226+0000 7fca9731c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca90102860 0x7fca9019ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:36.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.226+0000 7fca9731c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca901031b0 0x7fca9019bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:36.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.226+0000 7fca9731c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca90108890 0x7fca9019c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:36.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.226+0000 7fca9731c700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fca90113140 con 0x7fca90108890 2024-04-25T11:24:36.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.226+0000 7fca9731c700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fca90112fc0 con 0x7fca901031b0 2024-04-25T11:24:36.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.226+0000 7fca9731c700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fca901132c0 con 0x7fca90102860 2024-04-25T11:24:36.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.226+0000 7fca958b9700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca90108890 0x7fca9019c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:36.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.226+0000 7fca958b9700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca90108890 0x7fca9019c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48432/0 (socket says 172.21.15.69:48432) 2024-04-25T11:24:36.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.226+0000 7fca958b9700 1 -- 172.21.15.69:0/2571508714 learned_addr learned my addr 172.21.15.69:0/2571508714 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:24:36.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.226+0000 7fca948b7700 1 --2- 172.21.15.69:0/2571508714 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca901031b0 0x7fca9019bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:36.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.226+0000 7fca950b8700 1 --2- 172.21.15.69:0/2571508714 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca90102860 0x7fca9019ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:36.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.226+0000 7fca958b9700 1 -- 172.21.15.69:0/2571508714 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca90102860 msgr2=0x7fca9019ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:36.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.226+0000 7fca958b9700 1 --2- 172.21.15.69:0/2571508714 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca90102860 0x7fca9019ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:36.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.226+0000 7fca958b9700 1 -- 172.21.15.69:0/2571508714 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca901031b0 msgr2=0x7fca9019bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:36.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.226+0000 7fca958b9700 1 --2- 172.21.15.69:0/2571508714 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca901031b0 0x7fca9019bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:36.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.226+0000 7fca958b9700 1 -- 172.21.15.69:0/2571508714 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca901a6dd0 con 0x7fca90108890 2024-04-25T11:24:36.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.227+0000 7fca948b7700 1 --2- 172.21.15.69:0/2571508714 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca901031b0 0x7fca9019bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:24:36.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.227+0000 7fca958b9700 1 --2- 172.21.15.69:0/2571508714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca90108890 0x7fca9019c520 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fca8800b970 tx=0x7fca8800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:36.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.227+0000 7fca867fc700 1 -- 172.21.15.69:0/2571508714 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca88019070 con 0x7fca90108890 2024-04-25T11:24:36.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.227+0000 7fca867fc700 1 -- 172.21.15.69:0/2571508714 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fca8800ce40 con 0x7fca90108890 2024-04-25T11:24:36.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.227+0000 7fca9731c700 1 -- 172.21.15.69:0/2571508714 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca901a70c0 con 0x7fca90108890 2024-04-25T11:24:36.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.227+0000 7fca867fc700 1 -- 172.21.15.69:0/2571508714 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca88007370 con 0x7fca90108890 2024-04-25T11:24:36.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.227+0000 7fca9731c700 1 -- 172.21.15.69:0/2571508714 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca901a75b0 con 0x7fca90108890 2024-04-25T11:24:36.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.228+0000 7fca9731c700 1 -- 172.21.15.69:0/2571508714 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fca90063da0 con 0x7fca90108890 2024-04-25T11:24:36.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.229+0000 7fca867fc700 1 -- 172.21.15.69:0/2571508714 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fca8801c070 con 0x7fca90108890 2024-04-25T11:24:36.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.230+0000 7fca867fc700 1 --2- 172.21.15.69:0/2571508714 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fca7c0afc10 0x7fca7c0b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:36.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.230+0000 7fca950b8700 1 --2- 172.21.15.69:0/2571508714 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fca7c0afc10 0x7fca7c0b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:36.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.230+0000 7fca867fc700 1 -- 172.21.15.69:0/2571508714 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7fca880deff0 con 0x7fca90108890 2024-04-25T11:24:36.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.230+0000 7fca867fc700 1 --2- 172.21.15.69:0/2571508714 >> [v2:172.21.15.152:6800/1474319175,v1:172.21.15.152:6801/1474319175] conn(0x7fca7c0badc0 0x7fca7c0bd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:36.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.230+0000 7fca867fc700 1 -- 172.21.15.69:0/2571508714 --> [v2:172.21.15.152:6800/1474319175,v1:172.21.15.152:6801/1474319175] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fca88010ec0 con 0x7fca7c0badc0 2024-04-25T11:24:36.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.230+0000 7fca867fc700 1 -- 172.21.15.69:0/2571508714 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fca880df440 con 0x7fca90108890 2024-04-25T11:24:36.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.231+0000 7fca950b8700 1 --2- 172.21.15.69:0/2571508714 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fca7c0afc10 0x7fca7c0b2120 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fca8c005950 tx=0x7fca8c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:36.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.231+0000 7fca948b7700 1 --2- 172.21.15.69:0/2571508714 >> [v2:172.21.15.152:6800/1474319175,v1:172.21.15.152:6801/1474319175] conn(0x7fca7c0badc0 0x7fca7c0bd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:36.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.231+0000 7fca948b7700 1 --2- 172.21.15.69:0/2571508714 >> [v2:172.21.15.152:6800/1474319175,v1:172.21.15.152:6801/1474319175] conn(0x7fca7c0badc0 0x7fca7c0bd2b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:36.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.232+0000 7fca867fc700 1 -- 172.21.15.69:0/2571508714 <== osd.2 v2:172.21.15.152:6800/1474319175 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fca88010ec0 con 0x7fca7c0badc0 2024-04-25T11:24:36.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.248+0000 7fca9731c700 1 -- 172.21.15.69:0/2571508714 --> [v2:172.21.15.152:6800/1474319175,v1:172.21.15.152:6801/1474319175] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fca90064220 con 0x7fca7c0badc0 2024-04-25T11:24:36.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.249+0000 7fca867fc700 1 -- 172.21.15.69:0/2571508714 <== osd.2 v2:172.21.15.152:6800/1474319175 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fca90064220 con 0x7fca7c0badc0 2024-04-25T11:24:36.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.249+0000 7fca9731c700 1 -- 172.21.15.69:0/2571508714 >> [v2:172.21.15.152:6800/1474319175,v1:172.21.15.152:6801/1474319175] conn(0x7fca7c0badc0 msgr2=0x7fca7c0bd2b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:36.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.249+0000 7fca9731c700 1 --2- 172.21.15.69:0/2571508714 >> [v2:172.21.15.152:6800/1474319175,v1:172.21.15.152:6801/1474319175] conn(0x7fca7c0badc0 0x7fca7c0bd2b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:36.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.249+0000 7fca9731c700 1 -- 172.21.15.69:0/2571508714 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fca7c0afc10 msgr2=0x7fca7c0b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:36.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.249+0000 7fca9731c700 1 --2- 172.21.15.69:0/2571508714 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fca7c0afc10 0x7fca7c0b2120 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fca8c005950 tx=0x7fca8c0058e0 comp rx=0 tx=0).stop 2024-04-25T11:24:36.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.249+0000 7fca9731c700 1 -- 172.21.15.69:0/2571508714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca90108890 msgr2=0x7fca9019c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:36.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.249+0000 7fca9731c700 1 --2- 172.21.15.69:0/2571508714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca90108890 0x7fca9019c520 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fca8800b970 tx=0x7fca8800bd30 comp rx=0 tx=0).stop 2024-04-25T11:24:36.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.249+0000 7fca958b9700 1 -- 172.21.15.69:0/2571508714 reap_dead start 2024-04-25T11:24:36.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.249+0000 7fca9731c700 1 -- 172.21.15.69:0/2571508714 shutdown_connections 2024-04-25T11:24:36.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.249+0000 7fca9731c700 1 -- 172.21.15.69:0/2571508714 >> 172.21.15.69:0/2571508714 conn(0x7fca900fe380 msgr2=0x7fca901007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:36.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.249+0000 7fca9731c700 1 -- 172.21.15.69:0/2571508714 shutdown_connections 2024-04-25T11:24:36.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:35.249+0000 7fca9731c700 1 -- 172.21.15.69:0/2571508714 wait complete. 2024-04-25T11:24:37.185 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:36 smithi069 ceph-mon[29188]: pgmap v314: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T11:24:37.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:36 smithi152 ceph-mon[32701]: pgmap v314: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T11:24:37.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:36 smithi146 ceph-mon[32821]: pgmap v314: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T11:24:37.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.666+0000 7f376a424700 1 -- 172.21.15.69:0/3484104149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f376410e7a0 msgr2=0x7f3764110c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:37.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.666+0000 7f376a424700 1 --2- 172.21.15.69:0/3484104149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f376410e7a0 0x7f3764110c80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f375c00b0a0 tx=0x7f375c00b3b0 comp rx=0 tx=0).stop 2024-04-25T11:24:37.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.666+0000 7f376a424700 1 -- 172.21.15.69:0/3484104149 shutdown_connections 2024-04-25T11:24:37.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.666+0000 7f376a424700 1 --2- 172.21.15.69:0/3484104149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f376410e7a0 0x7f3764110c80 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:37.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.666+0000 7f376a424700 1 --2- 172.21.15.69:0/3484104149 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3764105330 0x7f3764105740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:37.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.666+0000 7f376a424700 1 --2- 172.21.15.69:0/3484104149 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3764105c80 0x7f376410e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:37.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.666+0000 7f376a424700 1 -- 172.21.15.69:0/3484104149 >> 172.21.15.69:0/3484104149 conn(0x7f37640fec90 msgr2=0x7f37641010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:37.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.666+0000 7f376a424700 1 -- 172.21.15.69:0/3484104149 shutdown_connections 2024-04-25T11:24:37.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.666+0000 7f376a424700 1 -- 172.21.15.69:0/3484104149 wait complete. 2024-04-25T11:24:37.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.666+0000 7f376a424700 1 Processor -- start 2024-04-25T11:24:37.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.666+0000 7f376a424700 1 -- start start 2024-04-25T11:24:37.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.667+0000 7f376a424700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3764105330 0x7f37641a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:37.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.667+0000 7f376a424700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3764105c80 0x7f37641a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:37.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.667+0000 7f376a424700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f376410e7a0 0x7f376419a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:37.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.667+0000 7f376a424700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f37641141b0 con 0x7f376410e7a0 2024-04-25T11:24:37.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.667+0000 7f376a424700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3764114030 con 0x7f3764105330 2024-04-25T11:24:37.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.667+0000 7f376a424700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3764114330 con 0x7f3764105c80 2024-04-25T11:24:37.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.667+0000 7f37689c1700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f376410e7a0 0x7f376419a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:37.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.667+0000 7f37689c1700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f376410e7a0 0x7f376419a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48446/0 (socket says 172.21.15.69:48446) 2024-04-25T11:24:37.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.667+0000 7f37689c1700 1 -- 172.21.15.69:0/4135623923 learned_addr learned my addr 172.21.15.69:0/4135623923 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:24:37.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.667+0000 7f37637fe700 1 --2- 172.21.15.69:0/4135623923 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3764105c80 0x7f37641a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:37.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.667+0000 7f3763fff700 1 --2- 172.21.15.69:0/4135623923 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3764105330 0x7f37641a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:37.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.667+0000 7f37689c1700 1 -- 172.21.15.69:0/4135623923 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3764105c80 msgr2=0x7f37641a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:37.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.667+0000 7f37689c1700 1 --2- 172.21.15.69:0/4135623923 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3764105c80 0x7f37641a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:37.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.667+0000 7f37689c1700 1 -- 172.21.15.69:0/4135623923 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3764105330 msgr2=0x7f37641a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:37.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.667+0000 7f37689c1700 1 --2- 172.21.15.69:0/4135623923 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3764105330 0x7f37641a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:37.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.667+0000 7f37689c1700 1 -- 172.21.15.69:0/4135623923 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376419afc0 con 0x7f376410e7a0 2024-04-25T11:24:37.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.668+0000 7f37689c1700 1 --2- 172.21.15.69:0/4135623923 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f376410e7a0 0x7f376419a910 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f375c00b780 tx=0x7f375c004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:37.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.668+0000 7f37617fa700 1 -- 172.21.15.69:0/4135623923 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c021070 con 0x7f376410e7a0 2024-04-25T11:24:37.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.668+0000 7f37617fa700 1 -- 172.21.15.69:0/4135623923 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f375c007b80 con 0x7f376410e7a0 2024-04-25T11:24:37.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.668+0000 7f376a424700 1 -- 172.21.15.69:0/4135623923 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f376419b250 con 0x7f376410e7a0 2024-04-25T11:24:37.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.668+0000 7f37617fa700 1 -- 172.21.15.69:0/4135623923 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c0052a0 con 0x7f376410e7a0 2024-04-25T11:24:37.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.668+0000 7f376a424700 1 -- 172.21.15.69:0/4135623923 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f376419b730 con 0x7f376410e7a0 2024-04-25T11:24:37.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.669+0000 7f376a424700 1 -- 172.21.15.69:0/4135623923 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3764063e20 con 0x7f376410e7a0 2024-04-25T11:24:37.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.671+0000 7f37617fa700 1 -- 172.21.15.69:0/4135623923 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f375c024070 con 0x7f376410e7a0 2024-04-25T11:24:37.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.671+0000 7f37617fa700 1 --2- 172.21.15.69:0/4135623923 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f374c0afc10 0x7f374c0b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:37.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.671+0000 7f3763fff700 1 --2- 172.21.15.69:0/4135623923 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f374c0afc10 0x7f374c0b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:37.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.671+0000 7f37617fa700 1 -- 172.21.15.69:0/4135623923 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f375c00f020 con 0x7f376410e7a0 2024-04-25T11:24:37.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.671+0000 7f37617fa700 1 --2- 172.21.15.69:0/4135623923 >> [v2:172.21.15.152:6808/1996555108,v1:172.21.15.152:6809/1996555108] conn(0x7f374c0bac90 0x7f374c0bd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:37.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.672+0000 7f37617fa700 1 -- 172.21.15.69:0/4135623923 --> [v2:172.21.15.152:6808/1996555108,v1:172.21.15.152:6809/1996555108] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f375c031e50 con 0x7f374c0bac90 2024-04-25T11:24:37.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.672+0000 7f37617fa700 1 -- 172.21.15.69:0/4135623923 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f375c0e0ed0 con 0x7f376410e7a0 2024-04-25T11:24:37.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.672+0000 7f3763fff700 1 --2- 172.21.15.69:0/4135623923 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f374c0afc10 0x7f374c0b2120 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f376419ba50 tx=0x7f3758009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:37.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.672+0000 7f37637fe700 1 --2- 172.21.15.69:0/4135623923 >> [v2:172.21.15.152:6808/1996555108,v1:172.21.15.152:6809/1996555108] conn(0x7f374c0bac90 0x7f374c0bd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:37.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.672+0000 7f37637fe700 1 --2- 172.21.15.69:0/4135623923 >> [v2:172.21.15.152:6808/1996555108,v1:172.21.15.152:6809/1996555108] conn(0x7f374c0bac90 0x7f374c0bd180 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:37.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.673+0000 7f37617fa700 1 -- 172.21.15.69:0/4135623923 <== osd.5 v2:172.21.15.152:6808/1996555108 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f375c031e50 con 0x7f374c0bac90 2024-04-25T11:24:37.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.690+0000 7f376a424700 1 -- 172.21.15.69:0/4135623923 --> [v2:172.21.15.152:6808/1996555108,v1:172.21.15.152:6809/1996555108] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f37640642a0 con 0x7f374c0bac90 2024-04-25T11:24:37.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.690+0000 7f37617fa700 1 -- 172.21.15.69:0/4135623923 <== osd.5 v2:172.21.15.152:6808/1996555108 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f37640642a0 con 0x7f374c0bac90 2024-04-25T11:24:37.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.691+0000 7f376a424700 1 -- 172.21.15.69:0/4135623923 >> [v2:172.21.15.152:6808/1996555108,v1:172.21.15.152:6809/1996555108] conn(0x7f374c0bac90 msgr2=0x7f374c0bd180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:37.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.691+0000 7f376a424700 1 --2- 172.21.15.69:0/4135623923 >> [v2:172.21.15.152:6808/1996555108,v1:172.21.15.152:6809/1996555108] conn(0x7f374c0bac90 0x7f374c0bd180 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:37.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.691+0000 7f376a424700 1 -- 172.21.15.69:0/4135623923 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f374c0afc10 msgr2=0x7f374c0b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:37.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.691+0000 7f376a424700 1 --2- 172.21.15.69:0/4135623923 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f374c0afc10 0x7f374c0b2120 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f376419ba50 tx=0x7f3758009260 comp rx=0 tx=0).stop 2024-04-25T11:24:37.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.691+0000 7f376a424700 1 -- 172.21.15.69:0/4135623923 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f376410e7a0 msgr2=0x7f376419a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:37.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.691+0000 7f376a424700 1 --2- 172.21.15.69:0/4135623923 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f376410e7a0 0x7f376419a910 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f375c00b780 tx=0x7f375c004d00 comp rx=0 tx=0).stop 2024-04-25T11:24:37.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.691+0000 7f37689c1700 1 -- 172.21.15.69:0/4135623923 reap_dead start 2024-04-25T11:24:37.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.691+0000 7f376a424700 1 -- 172.21.15.69:0/4135623923 shutdown_connections 2024-04-25T11:24:37.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.691+0000 7f376a424700 1 -- 172.21.15.69:0/4135623923 >> 172.21.15.69:0/4135623923 conn(0x7f37640fec90 msgr2=0x7f3764100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:37.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.691+0000 7f376a424700 1 -- 172.21.15.69:0/4135623923 shutdown_connections 2024-04-25T11:24:37.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:37.691+0000 7f376a424700 1 -- 172.21.15.69:0/4135623923 wait complete. 2024-04-25T11:24:38.141 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:37 smithi069 ceph-mon[29188]: pgmap v315: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 3 op/s; 213 B/s, 2 objects/s recovering 2024-04-25T11:24:38.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:37 smithi152 ceph-mon[32701]: pgmap v315: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 3 op/s; 213 B/s, 2 objects/s recovering 2024-04-25T11:24:38.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:37 smithi146 ceph-mon[32821]: pgmap v315: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 3 op/s; 213 B/s, 2 objects/s recovering 2024-04-25T11:24:38.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.352+0000 7f37e266b700 1 -- 172.21.15.69:0/1633072785 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37dc105300 msgr2=0x7f37dc105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:38.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.352+0000 7f37e266b700 1 --2- 172.21.15.69:0/1633072785 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37dc105300 0x7f37dc105710 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f37d0009990 tx=0x7f37d0009ca0 comp rx=0 tx=0).stop 2024-04-25T11:24:38.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.352+0000 7f37e266b700 1 -- 172.21.15.69:0/1633072785 shutdown_connections 2024-04-25T11:24:38.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.352+0000 7f37e266b700 1 --2- 172.21.15.69:0/1633072785 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37dc10e770 0x7f37dc110c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:38.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.352+0000 7f37e266b700 1 --2- 172.21.15.69:0/1633072785 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37dc105300 0x7f37dc105710 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:38.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.352+0000 7f37e266b700 1 --2- 172.21.15.69:0/1633072785 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37dc105c50 0x7f37dc10e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:38.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.352+0000 7f37e266b700 1 -- 172.21.15.69:0/1633072785 >> 172.21.15.69:0/1633072785 conn(0x7f37dc0fec90 msgr2=0x7f37dc1010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:38.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.352+0000 7f37e266b700 1 -- 172.21.15.69:0/1633072785 shutdown_connections 2024-04-25T11:24:38.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.352+0000 7f37e266b700 1 -- 172.21.15.69:0/1633072785 wait complete. 2024-04-25T11:24:38.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.353+0000 7f37e266b700 1 Processor -- start 2024-04-25T11:24:38.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.353+0000 7f37e266b700 1 -- start start 2024-04-25T11:24:38.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.353+0000 7f37e266b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37dc105300 0x7f37dc1a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:38.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.353+0000 7f37e266b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37dc105c50 0x7f37dc1a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:38.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.353+0000 7f37e266b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37dc10e770 0x7f37dc19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:38.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.353+0000 7f37e266b700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f37dc1141f0 con 0x7f37dc105300 2024-04-25T11:24:38.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.353+0000 7f37e266b700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f37dc114070 con 0x7f37dc105c50 2024-04-25T11:24:38.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.353+0000 7f37e266b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f37dc114370 con 0x7f37dc10e770 2024-04-25T11:24:38.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.354+0000 7f37dbfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37dc105300 0x7f37dc1a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:38.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.354+0000 7f37dbfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37dc105300 0x7f37dc1a07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48462/0 (socket says 172.21.15.69:48462) 2024-04-25T11:24:38.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.354+0000 7f37dbfff700 1 -- 172.21.15.69:0/3363403240 learned_addr learned my addr 172.21.15.69:0/3363403240 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:24:38.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.354+0000 7f37db7fe700 1 --2- 172.21.15.69:0/3363403240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37dc105c50 0x7f37dc1a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:38.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.354+0000 7f37e0c08700 1 --2- 172.21.15.69:0/3363403240 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37dc10e770 0x7f37dc19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:38.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.354+0000 7f37dbfff700 1 -- 172.21.15.69:0/3363403240 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37dc10e770 msgr2=0x7f37dc19aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:38.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.354+0000 7f37dbfff700 1 --2- 172.21.15.69:0/3363403240 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37dc10e770 0x7f37dc19aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:38.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.354+0000 7f37dbfff700 1 -- 172.21.15.69:0/3363403240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37dc105c50 msgr2=0x7f37dc1a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:38.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.354+0000 7f37dbfff700 1 --2- 172.21.15.69:0/3363403240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37dc105c50 0x7f37dc1a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:38.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.354+0000 7f37dbfff700 1 -- 172.21.15.69:0/3363403240 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37dc19b0c0 con 0x7f37dc105300 2024-04-25T11:24:38.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.354+0000 7f37dbfff700 1 --2- 172.21.15.69:0/3363403240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37dc105300 0x7f37dc1a07f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f37d0000c00 tx=0x7f37d00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:38.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.354+0000 7f37d97fa700 1 -- 172.21.15.69:0/3363403240 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d0027070 con 0x7f37dc105300 2024-04-25T11:24:38.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.354+0000 7f37e266b700 1 -- 172.21.15.69:0/3363403240 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37dc19b350 con 0x7f37dc105300 2024-04-25T11:24:38.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.354+0000 7f37d97fa700 1 -- 172.21.15.69:0/3363403240 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f37d000fdb0 con 0x7f37dc105300 2024-04-25T11:24:38.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.354+0000 7f37d97fa700 1 -- 172.21.15.69:0/3363403240 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d00173f0 con 0x7f37dc105300 2024-04-25T11:24:38.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.354+0000 7f37e266b700 1 -- 172.21.15.69:0/3363403240 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37dc19b830 con 0x7f37dc105300 2024-04-25T11:24:38.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.355+0000 7f37c2ffd700 1 -- 172.21.15.69:0/3363403240 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f37bc001030 con 0x7f37dc105300 2024-04-25T11:24:38.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.357+0000 7f37d97fa700 1 -- 172.21.15.69:0/3363403240 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f37d002a070 con 0x7f37dc105300 2024-04-25T11:24:38.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.358+0000 7f37d97fa700 1 --2- 172.21.15.69:0/3363403240 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f37c40afc10 0x7f37c40b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:38.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.358+0000 7f37d97fa700 1 -- 172.21.15.69:0/3363403240 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f37d00e0e80 con 0x7f37dc105300 2024-04-25T11:24:38.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.358+0000 7f37d97fa700 1 --2- 172.21.15.69:0/3363403240 >> [v2:172.21.15.69:6802/3756025235,v1:172.21.15.69:6803/3756025235] conn(0x7f37c40bac90 0x7f37c40bd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:38.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.358+0000 7f37db7fe700 1 --2- 172.21.15.69:0/3363403240 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f37c40afc10 0x7f37c40b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:38.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.358+0000 7f37d97fa700 1 -- 172.21.15.69:0/3363403240 --> [v2:172.21.15.69:6802/3756025235,v1:172.21.15.69:6803/3756025235] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f37d001de50 con 0x7f37c40bac90 2024-04-25T11:24:38.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.358+0000 7f37d97fa700 1 -- 172.21.15.69:0/3363403240 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f37d00e12d0 con 0x7f37dc105300 2024-04-25T11:24:38.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.359+0000 7f37e0c08700 1 --2- 172.21.15.69:0/3363403240 >> [v2:172.21.15.69:6802/3756025235,v1:172.21.15.69:6803/3756025235] conn(0x7f37c40bac90 0x7f37c40bd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:38.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.359+0000 7f37db7fe700 1 --2- 172.21.15.69:0/3363403240 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f37c40afc10 0x7f37c40b2120 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f37cc005fd0 tx=0x7f37cc00a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:38.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.359+0000 7f37e0c08700 1 --2- 172.21.15.69:0/3363403240 >> [v2:172.21.15.69:6802/3756025235,v1:172.21.15.69:6803/3756025235] conn(0x7f37c40bac90 0x7f37c40bd180 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:38.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.360+0000 7f37d97fa700 1 -- 172.21.15.69:0/3363403240 <== osd.0 v2:172.21.15.69:6802/3756025235 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f37d001de50 con 0x7f37c40bac90 2024-04-25T11:24:38.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.376+0000 7f37c2ffd700 1 -- 172.21.15.69:0/3363403240 --> [v2:172.21.15.69:6802/3756025235,v1:172.21.15.69:6803/3756025235] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f37bc002e60 con 0x7f37c40bac90 2024-04-25T11:24:38.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.376+0000 7f37d97fa700 1 -- 172.21.15.69:0/3363403240 <== osd.0 v2:172.21.15.69:6802/3756025235 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f37bc002e60 con 0x7f37c40bac90 2024-04-25T11:24:38.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.377+0000 7f37c2ffd700 1 -- 172.21.15.69:0/3363403240 >> [v2:172.21.15.69:6802/3756025235,v1:172.21.15.69:6803/3756025235] conn(0x7f37c40bac90 msgr2=0x7f37c40bd180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:38.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.377+0000 7f37c2ffd700 1 --2- 172.21.15.69:0/3363403240 >> [v2:172.21.15.69:6802/3756025235,v1:172.21.15.69:6803/3756025235] conn(0x7f37c40bac90 0x7f37c40bd180 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:38.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.377+0000 7f37c2ffd700 1 -- 172.21.15.69:0/3363403240 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f37c40afc10 msgr2=0x7f37c40b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:38.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.377+0000 7f37c2ffd700 1 --2- 172.21.15.69:0/3363403240 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f37c40afc10 0x7f37c40b2120 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f37cc005fd0 tx=0x7f37cc00a560 comp rx=0 tx=0).stop 2024-04-25T11:24:38.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.377+0000 7f37c2ffd700 1 -- 172.21.15.69:0/3363403240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37dc105300 msgr2=0x7f37dc1a07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:38.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.377+0000 7f37c2ffd700 1 --2- 172.21.15.69:0/3363403240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37dc105300 0x7f37dc1a07f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f37d0000c00 tx=0x7f37d00047b0 comp rx=0 tx=0).stop 2024-04-25T11:24:38.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.377+0000 7f37e0c08700 1 -- 172.21.15.69:0/3363403240 reap_dead start 2024-04-25T11:24:38.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.377+0000 7f37c2ffd700 1 -- 172.21.15.69:0/3363403240 shutdown_connections 2024-04-25T11:24:38.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.377+0000 7f37c2ffd700 1 -- 172.21.15.69:0/3363403240 >> 172.21.15.69:0/3363403240 conn(0x7f37dc0fec90 msgr2=0x7f37dc0ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:38.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.377+0000 7f37c2ffd700 1 -- 172.21.15.69:0/3363403240 shutdown_connections 2024-04-25T11:24:38.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:38.377+0000 7f37c2ffd700 1 -- 172.21.15.69:0/3363403240 wait complete. 2024-04-25T11:24:39.073 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:38 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:24:39.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:38 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:24:39.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:38 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:24:39.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.154+0000 7fcfd0c35700 1 -- 172.21.15.69:0/1763367877 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcfcc1091a0 msgr2=0x7fcfcc1095b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:39.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.154+0000 7fcfd0c35700 1 --2- 172.21.15.69:0/1763367877 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcfcc1091a0 0x7fcfcc1095b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc009990 tx=0x7fcfbc009ca0 comp rx=0 tx=0).stop 2024-04-25T11:24:39.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.155+0000 7fcfd0c35700 1 -- 172.21.15.69:0/1763367877 shutdown_connections 2024-04-25T11:24:39.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.155+0000 7fcfd0c35700 1 --2- 172.21.15.69:0/1763367877 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcfcc102f80 0x7fcfcc111010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:39.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.155+0000 7fcfd0c35700 1 --2- 172.21.15.69:0/1763367877 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcfcc1091a0 0x7fcfcc1095b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:39.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.155+0000 7fcfd0c35700 1 --2- 172.21.15.69:0/1763367877 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcfcc102630 0x7fcfcc102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:39.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.155+0000 7fcfd0c35700 1 -- 172.21.15.69:0/1763367877 >> 172.21.15.69:0/1763367877 conn(0x7fcfcc0fe3f0 msgr2=0x7fcfcc100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:39.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.155+0000 7fcfd0c35700 1 -- 172.21.15.69:0/1763367877 shutdown_connections 2024-04-25T11:24:39.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.155+0000 7fcfd0c35700 1 -- 172.21.15.69:0/1763367877 wait complete. 2024-04-25T11:24:39.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.156+0000 7fcfd0c35700 1 Processor -- start 2024-04-25T11:24:39.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.156+0000 7fcfd0c35700 1 -- start start 2024-04-25T11:24:39.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.156+0000 7fcfd0c35700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcfcc102630 0x7fcfcc19ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:39.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.156+0000 7fcfd0c35700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcfcc102f80 0x7fcfcc19b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:39.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.156+0000 7fcfd0c35700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcfcc1091a0 0x7fcfcc19f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:39.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.156+0000 7fcfd0c35700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcfcc1142e0 con 0x7fcfcc1091a0 2024-04-25T11:24:39.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.156+0000 7fcfd0c35700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fcfcc114160 con 0x7fcfcc102f80 2024-04-25T11:24:39.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.156+0000 7fcfd0c35700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fcfcc114460 con 0x7fcfcc102630 2024-04-25T11:24:39.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.156+0000 7fcfc9d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcfcc102f80 0x7fcfcc19b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:39.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.156+0000 7fcfca59c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcfcc102630 0x7fcfcc19ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:39.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.156+0000 7fcfcad9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcfcc1091a0 0x7fcfcc19f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:39.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.156+0000 7fcfc9d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcfcc102f80 0x7fcfcc19b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.69:56718/0 (socket says 172.21.15.69:56718) 2024-04-25T11:24:39.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.156+0000 7fcfca59c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcfcc102630 0x7fcfcc19ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.69:45594/0 (socket says 172.21.15.69:45594) 2024-04-25T11:24:39.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.156+0000 7fcfcad9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcfcc1091a0 0x7fcfcc19f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48494/0 (socket says 172.21.15.69:48494) 2024-04-25T11:24:39.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.156+0000 7fcfc9d9b700 1 -- 172.21.15.69:0/1250137758 learned_addr learned my addr 172.21.15.69:0/1250137758 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:24:39.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.156+0000 7fcfc9d9b700 1 -- 172.21.15.69:0/1250137758 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcfcc102630 msgr2=0x7fcfcc19ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:39.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.156+0000 7fcfc9d9b700 1 --2- 172.21.15.69:0/1250137758 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcfcc102630 0x7fcfcc19ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:39.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.156+0000 7fcfc9d9b700 1 -- 172.21.15.69:0/1250137758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcfcc1091a0 msgr2=0x7fcfcc19f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:39.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.156+0000 7fcfc9d9b700 1 --2- 172.21.15.69:0/1250137758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcfcc1091a0 0x7fcfcc19f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:39.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.157+0000 7fcfc9d9b700 1 -- 172.21.15.69:0/1250137758 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfcc19fbe0 con 0x7fcfcc102f80 2024-04-25T11:24:39.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.157+0000 7fcfcad9d700 1 --2- 172.21.15.69:0/1250137758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcfcc1091a0 0x7fcfcc19f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:24:39.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.157+0000 7fcfc9d9b700 1 --2- 172.21.15.69:0/1250137758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcfcc102f80 0x7fcfcc19b2b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0009d10 tx=0x7fcfc000b5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:39.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.157+0000 7fcfbb7fe700 1 -- 172.21.15.69:0/1250137758 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc0009e00 con 0x7fcfcc102f80 2024-04-25T11:24:39.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.157+0000 7fcfbb7fe700 1 -- 172.21.15.69:0/1250137758 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcfc000bd10 con 0x7fcfcc102f80 2024-04-25T11:24:39.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.157+0000 7fcfbb7fe700 1 -- 172.21.15.69:0/1250137758 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc0005700 con 0x7fcfcc102f80 2024-04-25T11:24:39.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.157+0000 7fcfd0c35700 1 -- 172.21.15.69:0/1250137758 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfcc19fed0 con 0x7fcfcc102f80 2024-04-25T11:24:39.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.157+0000 7fcfd0c35700 1 -- 172.21.15.69:0/1250137758 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfcc103740 con 0x7fcfcc102f80 2024-04-25T11:24:39.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.158+0000 7fcfd0c35700 1 -- 172.21.15.69:0/1250137758 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcfcc063da0 con 0x7fcfcc102f80 2024-04-25T11:24:39.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.161+0000 7fcfbb7fe700 1 -- 172.21.15.69:0/1250137758 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fcfc0004de0 con 0x7fcfcc102f80 2024-04-25T11:24:39.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.161+0000 7fcfbb7fe700 1 --2- 172.21.15.69:0/1250137758 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fcfb40afc90 0x7fcfb40b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:39.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.161+0000 7fcfca59c700 1 --2- 172.21.15.69:0/1250137758 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fcfb40afc90 0x7fcfb40b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:39.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.161+0000 7fcfbb7fe700 1 -- 172.21.15.69:0/1250137758 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7fcfc00e3330 con 0x7fcfcc102f80 2024-04-25T11:24:39.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.162+0000 7fcfbb7fe700 1 --2- 172.21.15.69:0/1250137758 >> [v2:172.21.15.146:6808/1898520972,v1:172.21.15.146:6809/1898520972] conn(0x7fcfb40bae40 0x7fcfb40bd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:39.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.162+0000 7fcfbb7fe700 1 -- 172.21.15.69:0/1250137758 --> [v2:172.21.15.146:6808/1898520972,v1:172.21.15.146:6809/1898520972] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcfb40bda00 con 0x7fcfb40bae40 2024-04-25T11:24:39.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.162+0000 7fcfbb7fe700 1 -- 172.21.15.69:0/1250137758 <== mon.2 v2:172.21.15.146:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fcfc00e3810 con 0x7fcfcc102f80 2024-04-25T11:24:39.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.162+0000 7fcfcad9d700 1 --2- 172.21.15.69:0/1250137758 >> [v2:172.21.15.146:6808/1898520972,v1:172.21.15.146:6809/1898520972] conn(0x7fcfb40bae40 0x7fcfb40bd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:39.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.162+0000 7fcfca59c700 1 --2- 172.21.15.69:0/1250137758 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fcfb40afc90 0x7fcfb40b21a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc00b650 tx=0x7fcfbc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:39.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.162+0000 7fcfcad9d700 1 --2- 172.21.15.69:0/1250137758 >> [v2:172.21.15.146:6808/1898520972,v1:172.21.15.146:6809/1898520972] conn(0x7fcfb40bae40 0x7fcfb40bd330 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:39.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.163+0000 7fcfbb7fe700 1 -- 172.21.15.69:0/1250137758 <== osd.4 v2:172.21.15.146:6808/1898520972 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcfb40bda00 con 0x7fcfb40bae40 2024-04-25T11:24:39.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.182+0000 7fcfd0c35700 1 -- 172.21.15.69:0/1250137758 --> [v2:172.21.15.146:6808/1898520972,v1:172.21.15.146:6809/1898520972] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcfcc0641f0 con 0x7fcfb40bae40 2024-04-25T11:24:39.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.183+0000 7fcfbb7fe700 1 -- 172.21.15.69:0/1250137758 <== osd.4 v2:172.21.15.146:6808/1898520972 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fcfcc0641f0 con 0x7fcfb40bae40 2024-04-25T11:24:39.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.183+0000 7fcfd0c35700 1 -- 172.21.15.69:0/1250137758 >> [v2:172.21.15.146:6808/1898520972,v1:172.21.15.146:6809/1898520972] conn(0x7fcfb40bae40 msgr2=0x7fcfb40bd330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:39.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.183+0000 7fcfd0c35700 1 --2- 172.21.15.69:0/1250137758 >> [v2:172.21.15.146:6808/1898520972,v1:172.21.15.146:6809/1898520972] conn(0x7fcfb40bae40 0x7fcfb40bd330 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:39.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.183+0000 7fcfd0c35700 1 -- 172.21.15.69:0/1250137758 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fcfb40afc90 msgr2=0x7fcfb40b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:39.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.183+0000 7fcfd0c35700 1 --2- 172.21.15.69:0/1250137758 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fcfb40afc90 0x7fcfb40b21a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc00b650 tx=0x7fcfbc005fb0 comp rx=0 tx=0).stop 2024-04-25T11:24:39.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.183+0000 7fcfd0c35700 1 -- 172.21.15.69:0/1250137758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcfcc102f80 msgr2=0x7fcfcc19b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:39.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.183+0000 7fcfd0c35700 1 --2- 172.21.15.69:0/1250137758 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcfcc102f80 0x7fcfcc19b2b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0009d10 tx=0x7fcfc000b5a0 comp rx=0 tx=0).stop 2024-04-25T11:24:39.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.183+0000 7fcfcad9d700 1 -- 172.21.15.69:0/1250137758 reap_dead start 2024-04-25T11:24:39.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.183+0000 7fcfd0c35700 1 -- 172.21.15.69:0/1250137758 shutdown_connections 2024-04-25T11:24:39.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.183+0000 7fcfd0c35700 1 -- 172.21.15.69:0/1250137758 >> 172.21.15.69:0/1250137758 conn(0x7fcfcc0fe3f0 msgr2=0x7fcfcc100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:39.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.183+0000 7fcfd0c35700 1 -- 172.21.15.69:0/1250137758 shutdown_connections 2024-04-25T11:24:39.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.183+0000 7fcfd0c35700 1 -- 172.21.15.69:0/1250137758 wait complete. 2024-04-25T11:24:39.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.808+0000 7f63cbded700 1 -- 172.21.15.69:0/3522848621 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63c4108570 msgr2=0x7f63c4108a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:39.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.808+0000 7f63cbded700 1 --2- 172.21.15.69:0/3522848621 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63c4108570 0x7f63c4108a70 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f63bc00b0a0 tx=0x7f63bc00b3b0 comp rx=0 tx=0).stop 2024-04-25T11:24:39.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.808+0000 7f63cbded700 1 -- 172.21.15.69:0/3522848621 shutdown_connections 2024-04-25T11:24:39.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.808+0000 7f63cbded700 1 --2- 172.21.15.69:0/3522848621 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63c4108570 0x7f63c4108a70 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:39.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.808+0000 7f63cbded700 1 --2- 172.21.15.69:0/3522848621 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63c4103120 0x7f63c4103530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:39.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.808+0000 7f63cbded700 1 --2- 172.21.15.69:0/3522848621 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63c4103a70 0x7f63c4107fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:39.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.808+0000 7f63cbded700 1 -- 172.21.15.69:0/3522848621 >> 172.21.15.69:0/3522848621 conn(0x7f63c40fecc0 msgr2=0x7f63c41010e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:39.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.808+0000 7f63cbded700 1 -- 172.21.15.69:0/3522848621 shutdown_connections 2024-04-25T11:24:39.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.808+0000 7f63cbded700 1 -- 172.21.15.69:0/3522848621 wait complete. 2024-04-25T11:24:39.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.809+0000 7f63cbded700 1 Processor -- start 2024-04-25T11:24:39.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.809+0000 7f63cbded700 1 -- start start 2024-04-25T11:24:39.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.809+0000 7f63cbded700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63c4103120 0x7f63c41a1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:39.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.809+0000 7f63cbded700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63c4103a70 0x7f63c41a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:39.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.809+0000 7f63cbded700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63c4108570 0x7f63c41a5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:39.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.809+0000 7f63cbded700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f63c411a810 con 0x7f63c4103a70 2024-04-25T11:24:39.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.809+0000 7f63cbded700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f63c411a690 con 0x7f63c4103120 2024-04-25T11:24:39.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.809+0000 7f63cbded700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f63c411a990 con 0x7f63c4108570 2024-04-25T11:24:39.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.809+0000 7f63c9388700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63c4103a70 0x7f63c41a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:39.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.809+0000 7f63c9388700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63c4103a70 0x7f63c41a17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48514/0 (socket says 172.21.15.69:48514) 2024-04-25T11:24:39.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.809+0000 7f63c9388700 1 -- 172.21.15.69:0/1629897740 learned_addr learned my addr 172.21.15.69:0/1629897740 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:24:39.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.810+0000 7f63c9b89700 1 --2- 172.21.15.69:0/1629897740 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63c4103120 0x7f63c41a1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:39.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.810+0000 7f63ca38a700 1 --2- 172.21.15.69:0/1629897740 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63c4108570 0x7f63c41a5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:39.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.810+0000 7f63c9388700 1 -- 172.21.15.69:0/1629897740 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63c4108570 msgr2=0x7f63c41a5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:39.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.810+0000 7f63c9388700 1 --2- 172.21.15.69:0/1629897740 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63c4108570 0x7f63c41a5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:39.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.810+0000 7f63c9388700 1 -- 172.21.15.69:0/1629897740 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63c4103120 msgr2=0x7f63c41a1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:39.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.810+0000 7f63c9388700 1 --2- 172.21.15.69:0/1629897740 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63c4103120 0x7f63c41a1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:39.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.810+0000 7f63c9388700 1 -- 172.21.15.69:0/1629897740 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63c41a60d0 con 0x7f63c4103a70 2024-04-25T11:24:39.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.810+0000 7f63c9388700 1 --2- 172.21.15.69:0/1629897740 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63c4103a70 0x7f63c41a17a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f63b400d950 tx=0x7f63b400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:39.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.810+0000 7f63baffd700 1 -- 172.21.15.69:0/1629897740 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b4017070 con 0x7f63c4103a70 2024-04-25T11:24:39.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.810+0000 7f63baffd700 1 -- 172.21.15.69:0/1629897740 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f63b4005020 con 0x7f63c4103a70 2024-04-25T11:24:39.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.810+0000 7f63cbded700 1 -- 172.21.15.69:0/1629897740 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63c41a6360 con 0x7f63c4103a70 2024-04-25T11:24:39.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.810+0000 7f63baffd700 1 -- 172.21.15.69:0/1629897740 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b400f980 con 0x7f63c4103a70 2024-04-25T11:24:39.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.810+0000 7f63cbded700 1 -- 172.21.15.69:0/1629897740 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63c41adce0 con 0x7f63c4103a70 2024-04-25T11:24:39.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.811+0000 7f63cbded700 1 -- 172.21.15.69:0/1629897740 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f63c4107940 con 0x7f63c4103a70 2024-04-25T11:24:39.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.813+0000 7f63baffd700 1 -- 172.21.15.69:0/1629897740 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f63b4004a30 con 0x7f63c4103a70 2024-04-25T11:24:39.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.813+0000 7f63baffd700 1 --2- 172.21.15.69:0/1629897740 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f63b00afc90 0x7f63b00b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:39.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.813+0000 7f63c9b89700 1 --2- 172.21.15.69:0/1629897740 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f63b00afc90 0x7f63b00b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:39.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.813+0000 7f63baffd700 1 -- 172.21.15.69:0/1629897740 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f63b40debc0 con 0x7f63c4103a70 2024-04-25T11:24:39.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.813+0000 7f63baffd700 1 --2- 172.21.15.69:0/1629897740 >> [v2:172.21.15.152:6824/2934336833,v1:172.21.15.152:6825/2934336833] conn(0x7f63b00badb0 0x7f63b00bd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:39.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.813+0000 7f63baffd700 1 -- 172.21.15.69:0/1629897740 --> [v2:172.21.15.152:6824/2934336833,v1:172.21.15.152:6825/2934336833] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f63b00bd970 con 0x7f63b00badb0 2024-04-25T11:24:39.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.813+0000 7f63baffd700 1 -- 172.21.15.69:0/1629897740 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f63b40df010 con 0x7f63c4103a70 2024-04-25T11:24:39.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.814+0000 7f63ca38a700 1 --2- 172.21.15.69:0/1629897740 >> [v2:172.21.15.152:6824/2934336833,v1:172.21.15.152:6825/2934336833] conn(0x7f63b00badb0 0x7f63b00bd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:39.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.814+0000 7f63c9b89700 1 --2- 172.21.15.69:0/1629897740 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f63b00afc90 0x7f63b00b21a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f63c406d020 tx=0x7f63c000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:39.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.814+0000 7f63ca38a700 1 --2- 172.21.15.69:0/1629897740 >> [v2:172.21.15.152:6824/2934336833,v1:172.21.15.152:6825/2934336833] conn(0x7f63b00badb0 0x7f63b00bd2a0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:39.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.815+0000 7f63baffd700 1 -- 172.21.15.69:0/1629897740 <== osd.11 v2:172.21.15.152:6824/2934336833 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f63b00bd970 con 0x7f63b00badb0 2024-04-25T11:24:39.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.831+0000 7f63cbded700 1 -- 172.21.15.69:0/1629897740 --> [v2:172.21.15.152:6824/2934336833,v1:172.21.15.152:6825/2934336833] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f63c40761a0 con 0x7f63b00badb0 2024-04-25T11:24:39.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.831+0000 7f63baffd700 1 -- 172.21.15.69:0/1629897740 <== osd.11 v2:172.21.15.152:6824/2934336833 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f63c40761a0 con 0x7f63b00badb0 2024-04-25T11:24:39.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.832+0000 7f63cbded700 1 -- 172.21.15.69:0/1629897740 >> [v2:172.21.15.152:6824/2934336833,v1:172.21.15.152:6825/2934336833] conn(0x7f63b00badb0 msgr2=0x7f63b00bd2a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:39.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.832+0000 7f63cbded700 1 --2- 172.21.15.69:0/1629897740 >> [v2:172.21.15.152:6824/2934336833,v1:172.21.15.152:6825/2934336833] conn(0x7f63b00badb0 0x7f63b00bd2a0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:39.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.832+0000 7f63cbded700 1 -- 172.21.15.69:0/1629897740 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f63b00afc90 msgr2=0x7f63b00b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:39.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.832+0000 7f63cbded700 1 --2- 172.21.15.69:0/1629897740 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f63b00afc90 0x7f63b00b21a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f63c406d020 tx=0x7f63c000b410 comp rx=0 tx=0).stop 2024-04-25T11:24:39.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.832+0000 7f63cbded700 1 -- 172.21.15.69:0/1629897740 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63c4103a70 msgr2=0x7f63c41a17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:39.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.832+0000 7f63cbded700 1 --2- 172.21.15.69:0/1629897740 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63c4103a70 0x7f63c41a17a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f63b400d950 tx=0x7f63b400dc60 comp rx=0 tx=0).stop 2024-04-25T11:24:39.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.832+0000 7f63ca38a700 1 -- 172.21.15.69:0/1629897740 reap_dead start 2024-04-25T11:24:39.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.832+0000 7f63cbded700 1 -- 172.21.15.69:0/1629897740 shutdown_connections 2024-04-25T11:24:39.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.832+0000 7f63cbded700 1 -- 172.21.15.69:0/1629897740 >> 172.21.15.69:0/1629897740 conn(0x7f63c40fecc0 msgr2=0x7f63c4105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:39.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.832+0000 7f63cbded700 1 -- 172.21.15.69:0/1629897740 shutdown_connections 2024-04-25T11:24:39.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.832+0000 7f63cbded700 1 -- 172.21.15.69:0/1629897740 wait complete. 2024-04-25T11:24:39.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.054+0000 7f1a30131700 1 -- 172.21.15.69:0/4173060863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a28107cc0 msgr2=0x7f1a281080d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:39.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.054+0000 7f1a30131700 1 --2- 172.21.15.69:0/4173060863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a28107cc0 0x7f1a281080d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1a180099c0 tx=0x7f1a18009cd0 comp rx=0 tx=0).stop 2024-04-25T11:24:39.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.054+0000 7f1a30131700 1 -- 172.21.15.69:0/4173060863 shutdown_connections 2024-04-25T11:24:39.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.054+0000 7f1a30131700 1 --2- 172.21.15.69:0/4173060863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a28108610 0x7f1a2810ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:39.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.054+0000 7f1a30131700 1 --2- 172.21.15.69:0/4173060863 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a28103390 0x7f1a281037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:39.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.054+0000 7f1a30131700 1 --2- 172.21.15.69:0/4173060863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a28107cc0 0x7f1a281080d0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:39.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.054+0000 7f1a30131700 1 -- 172.21.15.69:0/4173060863 >> 172.21.15.69:0/4173060863 conn(0x7f1a280fec90 msgr2=0x7f1a281010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:39.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.054+0000 7f1a30131700 1 -- 172.21.15.69:0/4173060863 shutdown_connections 2024-04-25T11:24:39.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.054+0000 7f1a30131700 1 -- 172.21.15.69:0/4173060863 wait complete. 2024-04-25T11:24:39.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.055+0000 7f1a30131700 1 Processor -- start 2024-04-25T11:24:39.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.055+0000 7f1a30131700 1 -- start start 2024-04-25T11:24:39.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.055+0000 7f1a30131700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a28103390 0x7f1a2819ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:39.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.055+0000 7f1a30131700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a28107cc0 0x7f1a2819bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:39.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.055+0000 7f1a30131700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a28108610 0x7f1a2819c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:39.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.055+0000 7f1a30131700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1a281130d0 con 0x7f1a28107cc0 2024-04-25T11:24:39.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.055+0000 7f1a30131700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1a28112f50 con 0x7f1a28103390 2024-04-25T11:24:39.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.055+0000 7f1a30131700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1a28113250 con 0x7f1a28108610 2024-04-25T11:24:39.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.056+0000 7f1a2d6cc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a28107cc0 0x7f1a2819bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:39.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.056+0000 7f1a2d6cc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a28107cc0 0x7f1a2819bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48480/0 (socket says 172.21.15.69:48480) 2024-04-25T11:24:39.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.056+0000 7f1a2e6ce700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a28108610 0x7f1a2819c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:39.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.056+0000 7f1a2d6cc700 1 -- 172.21.15.69:0/3516206315 learned_addr learned my addr 172.21.15.69:0/3516206315 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:24:39.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.056+0000 7f1a2decd700 1 --2- 172.21.15.69:0/3516206315 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a28103390 0x7f1a2819ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:39.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.056+0000 7f1a2e6ce700 1 -- 172.21.15.69:0/3516206315 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a28103390 msgr2=0x7f1a2819ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:39.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.056+0000 7f1a2e6ce700 1 --2- 172.21.15.69:0/3516206315 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1a28103390 0x7f1a2819ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:39.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.056+0000 7f1a2e6ce700 1 -- 172.21.15.69:0/3516206315 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a28107cc0 msgr2=0x7f1a2819bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:39.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.056+0000 7f1a2e6ce700 1 --2- 172.21.15.69:0/3516206315 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a28107cc0 0x7f1a2819bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:39.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.056+0000 7f1a2e6ce700 1 -- 172.21.15.69:0/3516206315 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a281a6da0 con 0x7f1a28108610 2024-04-25T11:24:39.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.056+0000 7f1a2d6cc700 1 --2- 172.21.15.69:0/3516206315 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1a28107cc0 0x7f1a2819bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:24:39.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.056+0000 7f1a2e6ce700 1 --2- 172.21.15.69:0/3516206315 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a28108610 0x7f1a2819c4f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1a2000efc0 tx=0x7f1a2000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:39.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.056+0000 7f1a1effd700 1 -- 172.21.15.69:0/3516206315 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a200092e0 con 0x7f1a28108610 2024-04-25T11:24:39.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.057+0000 7f1a1effd700 1 -- 172.21.15.69:0/3516206315 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1a20004bb0 con 0x7f1a28108610 2024-04-25T11:24:39.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.057+0000 7f1a30131700 1 -- 172.21.15.69:0/3516206315 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a281a7030 con 0x7f1a28108610 2024-04-25T11:24:39.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.057+0000 7f1a1effd700 1 -- 172.21.15.69:0/3516206315 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a20004020 con 0x7f1a28108610 2024-04-25T11:24:39.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.057+0000 7f1a30131700 1 -- 172.21.15.69:0/3516206315 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a281a7570 con 0x7f1a28108610 2024-04-25T11:24:39.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.058+0000 7f1a30131700 1 -- 172.21.15.69:0/3516206315 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1a28063d60 con 0x7f1a28108610 2024-04-25T11:24:39.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.059+0000 7f1a1effd700 1 -- 172.21.15.69:0/3516206315 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f1a2001b030 con 0x7f1a28108610 2024-04-25T11:24:39.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.060+0000 7f1a1effd700 1 --2- 172.21.15.69:0/3516206315 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f1a140afc90 0x7f1a140b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:39.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.060+0000 7f1a2decd700 1 --2- 172.21.15.69:0/3516206315 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f1a140afc90 0x7f1a140b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:39.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.060+0000 7f1a1effd700 1 -- 172.21.15.69:0/3516206315 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f1a200df250 con 0x7f1a28108610 2024-04-25T11:24:39.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.060+0000 7f1a1effd700 1 --2- 172.21.15.69:0/3516206315 >> [v2:172.21.15.69:6818/491215762,v1:172.21.15.69:6819/491215762] conn(0x7f1a140bae40 0x7f1a140bd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:39.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.060+0000 7f1a1effd700 1 -- 172.21.15.69:0/3516206315 --> [v2:172.21.15.69:6818/491215762,v1:172.21.15.69:6819/491215762] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1a140bda00 con 0x7f1a140bae40 2024-04-25T11:24:39.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.060+0000 7f1a1effd700 1 -- 172.21.15.69:0/3516206315 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f1a200df600 con 0x7f1a28108610 2024-04-25T11:24:39.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.060+0000 7f1a2d6cc700 1 --2- 172.21.15.69:0/3516206315 >> [v2:172.21.15.69:6818/491215762,v1:172.21.15.69:6819/491215762] conn(0x7f1a140bae40 0x7f1a140bd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:39.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.060+0000 7f1a2decd700 1 --2- 172.21.15.69:0/3516206315 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f1a140afc90 0x7f1a140b21a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1a24009c60 tx=0x7f1a24009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:39.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.061+0000 7f1a2d6cc700 1 --2- 172.21.15.69:0/3516206315 >> [v2:172.21.15.69:6818/491215762,v1:172.21.15.69:6819/491215762] conn(0x7f1a140bae40 0x7f1a140bd330 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:39.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.062+0000 7f1a1effd700 1 -- 172.21.15.69:0/3516206315 <== osd.6 v2:172.21.15.69:6818/491215762 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1a140bda00 con 0x7f1a140bae40 2024-04-25T11:24:39.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.079+0000 7f1a30131700 1 -- 172.21.15.69:0/3516206315 --> [v2:172.21.15.69:6818/491215762,v1:172.21.15.69:6819/491215762] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1a280641e0 con 0x7f1a140bae40 2024-04-25T11:24:39.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.079+0000 7f1a1effd700 1 -- 172.21.15.69:0/3516206315 <== osd.6 v2:172.21.15.69:6818/491215762 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1a280641e0 con 0x7f1a140bae40 2024-04-25T11:24:39.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.080+0000 7f1a30131700 1 -- 172.21.15.69:0/3516206315 >> [v2:172.21.15.69:6818/491215762,v1:172.21.15.69:6819/491215762] conn(0x7f1a140bae40 msgr2=0x7f1a140bd330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:39.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.080+0000 7f1a30131700 1 --2- 172.21.15.69:0/3516206315 >> [v2:172.21.15.69:6818/491215762,v1:172.21.15.69:6819/491215762] conn(0x7f1a140bae40 0x7f1a140bd330 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:39.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.080+0000 7f1a30131700 1 -- 172.21.15.69:0/3516206315 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f1a140afc90 msgr2=0x7f1a140b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:39.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.080+0000 7f1a30131700 1 --2- 172.21.15.69:0/3516206315 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f1a140afc90 0x7f1a140b21a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1a24009c60 tx=0x7f1a24009260 comp rx=0 tx=0).stop 2024-04-25T11:24:39.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.080+0000 7f1a30131700 1 -- 172.21.15.69:0/3516206315 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a28108610 msgr2=0x7f1a2819c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:39.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.080+0000 7f1a30131700 1 --2- 172.21.15.69:0/3516206315 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a28108610 0x7f1a2819c4f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1a2000efc0 tx=0x7f1a2000c3a0 comp rx=0 tx=0).stop 2024-04-25T11:24:39.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.080+0000 7f1a2e6ce700 1 -- 172.21.15.69:0/3516206315 reap_dead start 2024-04-25T11:24:39.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.080+0000 7f1a30131700 1 -- 172.21.15.69:0/3516206315 shutdown_connections 2024-04-25T11:24:39.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.080+0000 7f1a30131700 1 -- 172.21.15.69:0/3516206315 >> 172.21.15.69:0/3516206315 conn(0x7f1a280fec90 msgr2=0x7f1a281010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:39.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.080+0000 7f1a30131700 1 -- 172.21.15.69:0/3516206315 shutdown_connections 2024-04-25T11:24:39.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:39.080+0000 7f1a30131700 1 -- 172.21.15.69:0/3516206315 wait complete. 2024-04-25T11:24:40.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:39 smithi152 ceph-mon[32701]: pgmap v316: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 7 op/s; 213 B/s, 2 objects/s recovering 2024-04-25T11:24:40.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:39 smithi146 ceph-mon[32821]: pgmap v316: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 7 op/s; 213 B/s, 2 objects/s recovering 2024-04-25T11:24:40.442 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:39 smithi069 ceph-mon[29188]: pgmap v316: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 7 op/s; 213 B/s, 2 objects/s recovering 2024-04-25T11:24:41.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.166+0000 7f524e1cf700 1 -- 172.21.15.69:0/837756218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5248105d90 msgr2=0x7f524810e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:41.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.166+0000 7f524e1cf700 1 --2- 172.21.15.69:0/837756218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5248105d90 0x7f524810e2e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5230009990 tx=0x7f5230009ca0 comp rx=0 tx=0).stop 2024-04-25T11:24:41.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.166+0000 7f524e1cf700 1 -- 172.21.15.69:0/837756218 shutdown_connections 2024-04-25T11:24:41.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.166+0000 7f524e1cf700 1 --2- 172.21.15.69:0/837756218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f524810e820 0x7f5248110d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:41.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.166+0000 7f524e1cf700 1 --2- 172.21.15.69:0/837756218 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5248105440 0x7f5248105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:41.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.166+0000 7f524e1cf700 1 --2- 172.21.15.69:0/837756218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5248105d90 0x7f524810e2e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:41.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.166+0000 7f524e1cf700 1 -- 172.21.15.69:0/837756218 >> 172.21.15.69:0/837756218 conn(0x7f5248078cd0 msgr2=0x7f5248079110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:41.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.166+0000 7f524e1cf700 1 -- 172.21.15.69:0/837756218 shutdown_connections 2024-04-25T11:24:41.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.166+0000 7f524e1cf700 1 -- 172.21.15.69:0/837756218 wait complete. 2024-04-25T11:24:41.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.167+0000 7f524e1cf700 1 Processor -- start 2024-04-25T11:24:41.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.167+0000 7f524e1cf700 1 -- start start 2024-04-25T11:24:41.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.167+0000 7f524e1cf700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5248105440 0x7f52481a0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:41.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.167+0000 7f524e1cf700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5248105d90 0x7f52481a0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:41.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.167+0000 7f524e1cf700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f524810e820 0x7f524819aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:41.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.167+0000 7f524e1cf700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5248114250 con 0x7f5248105440 2024-04-25T11:24:41.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.167+0000 7f524e1cf700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f52481140d0 con 0x7f5248105d90 2024-04-25T11:24:41.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.167+0000 7f524e1cf700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f52481143d0 con 0x7f524810e820 2024-04-25T11:24:41.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.167+0000 7f52477fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5248105440 0x7f52481a0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:41.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.167+0000 7f52477fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5248105440 0x7f52481a0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48546/0 (socket says 172.21.15.69:48546) 2024-04-25T11:24:41.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.167+0000 7f52477fe700 1 -- 172.21.15.69:0/3197926486 learned_addr learned my addr 172.21.15.69:0/3197926486 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:24:41.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.167+0000 7f5246ffd700 1 --2- 172.21.15.69:0/3197926486 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5248105d90 0x7f52481a0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:41.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.168+0000 7f5247fff700 1 --2- 172.21.15.69:0/3197926486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f524810e820 0x7f524819aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:41.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.168+0000 7f52477fe700 1 -- 172.21.15.69:0/3197926486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f524810e820 msgr2=0x7f524819aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:41.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.168+0000 7f52477fe700 1 --2- 172.21.15.69:0/3197926486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f524810e820 0x7f524819aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:41.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.168+0000 7f52477fe700 1 -- 172.21.15.69:0/3197926486 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5248105d90 msgr2=0x7f52481a0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:41.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.168+0000 7f52477fe700 1 --2- 172.21.15.69:0/3197926486 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5248105d90 0x7f52481a0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:41.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.168+0000 7f52477fe700 1 -- 172.21.15.69:0/3197926486 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f524819b140 con 0x7f5248105440 2024-04-25T11:24:41.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.168+0000 7f52477fe700 1 --2- 172.21.15.69:0/3197926486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5248105440 0x7f52481a0870 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f524000ada0 tx=0x7f524000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:41.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.168+0000 7f5244ff9700 1 -- 172.21.15.69:0/3197926486 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5240017020 con 0x7f5248105440 2024-04-25T11:24:41.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.168+0000 7f5244ff9700 1 -- 172.21.15.69:0/3197926486 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f524000ce10 con 0x7f5248105440 2024-04-25T11:24:41.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.168+0000 7f524e1cf700 1 -- 172.21.15.69:0/3197926486 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f524819b430 con 0x7f5248105440 2024-04-25T11:24:41.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.168+0000 7f5244ff9700 1 -- 172.21.15.69:0/3197926486 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5240005310 con 0x7f5248105440 2024-04-25T11:24:41.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.168+0000 7f524e1cf700 1 -- 172.21.15.69:0/3197926486 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52481a7700 con 0x7f5248105440 2024-04-25T11:24:41.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.169+0000 7f524e1cf700 1 -- 172.21.15.69:0/3197926486 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5248063e20 con 0x7f5248105440 2024-04-25T11:24:41.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.171+0000 7f5244ff9700 1 -- 172.21.15.69:0/3197926486 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f5240004ad0 con 0x7f5248105440 2024-04-25T11:24:41.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.171+0000 7f5244ff9700 1 --2- 172.21.15.69:0/3197926486 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f52340afc10 0x7f52340b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:41.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.172+0000 7f5246ffd700 1 --2- 172.21.15.69:0/3197926486 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f52340afc10 0x7f52340b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:41.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.172+0000 7f5244ff9700 1 -- 172.21.15.69:0/3197926486 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f52400df770 con 0x7f5248105440 2024-04-25T11:24:41.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.172+0000 7f5244ff9700 1 --2- 172.21.15.69:0/3197926486 >> [v2:172.21.15.69:6826/2414186758,v1:172.21.15.69:6827/2414186758] conn(0x7f52340badc0 0x7f52340bd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:41.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.172+0000 7f5244ff9700 1 -- 172.21.15.69:0/3197926486 --> [v2:172.21.15.69:6826/2414186758,v1:172.21.15.69:6827/2414186758] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f524001be50 con 0x7f52340badc0 2024-04-25T11:24:41.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.172+0000 7f5244ff9700 1 -- 172.21.15.69:0/3197926486 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f52400dfbc0 con 0x7f5248105440 2024-04-25T11:24:41.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.172+0000 7f5247fff700 1 --2- 172.21.15.69:0/3197926486 >> [v2:172.21.15.69:6826/2414186758,v1:172.21.15.69:6827/2414186758] conn(0x7f52340badc0 0x7f52340bd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:41.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.172+0000 7f5246ffd700 1 --2- 172.21.15.69:0/3197926486 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f52340afc10 0x7f52340b2120 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5248110310 tx=0x7f523000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:41.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.172+0000 7f5247fff700 1 --2- 172.21.15.69:0/3197926486 >> [v2:172.21.15.69:6826/2414186758,v1:172.21.15.69:6827/2414186758] conn(0x7f52340badc0 0x7f52340bd2b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:41.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.173+0000 7f5244ff9700 1 -- 172.21.15.69:0/3197926486 <== osd.9 v2:172.21.15.69:6826/2414186758 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f524001be50 con 0x7f52340badc0 2024-04-25T11:24:41.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.189+0000 7f524e1cf700 1 -- 172.21.15.69:0/3197926486 --> [v2:172.21.15.69:6826/2414186758,v1:172.21.15.69:6827/2414186758] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f52480642a0 con 0x7f52340badc0 2024-04-25T11:24:41.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.190+0000 7f5244ff9700 1 -- 172.21.15.69:0/3197926486 <== osd.9 v2:172.21.15.69:6826/2414186758 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f52480642a0 con 0x7f52340badc0 2024-04-25T11:24:41.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.190+0000 7f524e1cf700 1 -- 172.21.15.69:0/3197926486 >> [v2:172.21.15.69:6826/2414186758,v1:172.21.15.69:6827/2414186758] conn(0x7f52340badc0 msgr2=0x7f52340bd2b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:41.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.190+0000 7f524e1cf700 1 --2- 172.21.15.69:0/3197926486 >> [v2:172.21.15.69:6826/2414186758,v1:172.21.15.69:6827/2414186758] conn(0x7f52340badc0 0x7f52340bd2b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:41.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.190+0000 7f524e1cf700 1 -- 172.21.15.69:0/3197926486 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f52340afc10 msgr2=0x7f52340b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:41.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.190+0000 7f524e1cf700 1 --2- 172.21.15.69:0/3197926486 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f52340afc10 0x7f52340b2120 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5248110310 tx=0x7f523000b540 comp rx=0 tx=0).stop 2024-04-25T11:24:41.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.190+0000 7f524e1cf700 1 -- 172.21.15.69:0/3197926486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5248105440 msgr2=0x7f52481a0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:41.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.190+0000 7f524e1cf700 1 --2- 172.21.15.69:0/3197926486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5248105440 0x7f52481a0870 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f524000ada0 tx=0x7f524000c330 comp rx=0 tx=0).stop 2024-04-25T11:24:41.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.190+0000 7f5247fff700 1 -- 172.21.15.69:0/3197926486 reap_dead start 2024-04-25T11:24:41.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.190+0000 7f524e1cf700 1 -- 172.21.15.69:0/3197926486 shutdown_connections 2024-04-25T11:24:41.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.190+0000 7f524e1cf700 1 -- 172.21.15.69:0/3197926486 >> 172.21.15.69:0/3197926486 conn(0x7f5248078cd0 msgr2=0x7f52480ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:41.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.190+0000 7f524e1cf700 1 -- 172.21.15.69:0/3197926486 shutdown_connections 2024-04-25T11:24:41.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.190+0000 7f524e1cf700 1 -- 172.21.15.69:0/3197926486 wait complete. 2024-04-25T11:24:41.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.048+0000 7fc6da978700 1 -- 172.21.15.69:0/481387809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6d4103390 msgr2=0x7fc6d41037a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:41.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.048+0000 7fc6da978700 1 --2- 172.21.15.69:0/481387809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6d4103390 0x7fc6d41037a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc6c40099c0 tx=0x7fc6c4009cd0 comp rx=0 tx=0).stop 2024-04-25T11:24:41.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.048+0000 7fc6da978700 1 -- 172.21.15.69:0/481387809 shutdown_connections 2024-04-25T11:24:41.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.048+0000 7fc6da978700 1 --2- 172.21.15.69:0/481387809 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6d4108610 0x7fc6d410ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:41.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.048+0000 7fc6da978700 1 --2- 172.21.15.69:0/481387809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6d4103390 0x7fc6d41037a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:41.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.048+0000 7fc6da978700 1 --2- 172.21.15.69:0/481387809 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6d4107cc0 0x7fc6d41080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:41.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.048+0000 7fc6da978700 1 -- 172.21.15.69:0/481387809 >> 172.21.15.69:0/481387809 conn(0x7fc6d40fec90 msgr2=0x7fc6d41010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:41.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.049+0000 7fc6da978700 1 -- 172.21.15.69:0/481387809 shutdown_connections 2024-04-25T11:24:41.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.049+0000 7fc6da978700 1 -- 172.21.15.69:0/481387809 wait complete. 2024-04-25T11:24:41.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.049+0000 7fc6da978700 1 Processor -- start 2024-04-25T11:24:41.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.049+0000 7fc6da978700 1 -- start start 2024-04-25T11:24:41.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.050+0000 7fc6da978700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6d4103390 0x7fc6d419ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:41.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.050+0000 7fc6da978700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6d4107cc0 0x7fc6d419bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:41.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.050+0000 7fc6da978700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6d4108610 0x7fc6d419c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:41.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.050+0000 7fc6da978700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d41130f0 con 0x7fc6d4108610 2024-04-25T11:24:41.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.050+0000 7fc6da978700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d4112f70 con 0x7fc6d4103390 2024-04-25T11:24:41.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.050+0000 7fc6da978700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d4113270 con 0x7fc6d4107cc0 2024-04-25T11:24:41.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.050+0000 7fc6d8f15700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6d4108610 0x7fc6d419c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:41.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.050+0000 7fc6d8f15700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6d4108610 0x7fc6d419c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48536/0 (socket says 172.21.15.69:48536) 2024-04-25T11:24:41.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.050+0000 7fc6d8f15700 1 -- 172.21.15.69:0/3645003297 learned_addr learned my addr 172.21.15.69:0/3645003297 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:24:41.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.050+0000 7fc6d37fe700 1 --2- 172.21.15.69:0/3645003297 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6d4107cc0 0x7fc6d419bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:41.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.050+0000 7fc6d3fff700 1 --2- 172.21.15.69:0/3645003297 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6d4103390 0x7fc6d419ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:41.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.050+0000 7fc6d8f15700 1 -- 172.21.15.69:0/3645003297 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6d4107cc0 msgr2=0x7fc6d419bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:41.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.050+0000 7fc6d8f15700 1 --2- 172.21.15.69:0/3645003297 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6d4107cc0 0x7fc6d419bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:41.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.050+0000 7fc6d8f15700 1 -- 172.21.15.69:0/3645003297 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6d4103390 msgr2=0x7fc6d419ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:41.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.050+0000 7fc6d8f15700 1 --2- 172.21.15.69:0/3645003297 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6d4103390 0x7fc6d419ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:41.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.050+0000 7fc6d8f15700 1 -- 172.21.15.69:0/3645003297 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6d41a6d80 con 0x7fc6d4108610 2024-04-25T11:24:41.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.050+0000 7fc6d8f15700 1 --2- 172.21.15.69:0/3645003297 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6d4108610 0x7fc6d419c4d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc00ef30 tx=0x7fc6cc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:41.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.051+0000 7fc6d17fa700 1 -- 172.21.15.69:0/3645003297 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc0092e0 con 0x7fc6d4108610 2024-04-25T11:24:41.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.051+0000 7fc6d17fa700 1 -- 172.21.15.69:0/3645003297 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6cc004bb0 con 0x7fc6d4108610 2024-04-25T11:24:41.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.051+0000 7fc6da978700 1 -- 172.21.15.69:0/3645003297 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6d41a7010 con 0x7fc6d4108610 2024-04-25T11:24:41.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.051+0000 7fc6d17fa700 1 -- 172.21.15.69:0/3645003297 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc004020 con 0x7fc6d4108610 2024-04-25T11:24:41.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.051+0000 7fc6da978700 1 -- 172.21.15.69:0/3645003297 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6d41a7500 con 0x7fc6d4108610 2024-04-25T11:24:41.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.052+0000 7fc6da978700 1 -- 172.21.15.69:0/3645003297 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc6d4063da0 con 0x7fc6d4108610 2024-04-25T11:24:41.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.053+0000 7fc6d17fa700 1 -- 172.21.15.69:0/3645003297 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fc6cc01b030 con 0x7fc6d4108610 2024-04-25T11:24:41.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.054+0000 7fc6d17fa700 1 --2- 172.21.15.69:0/3645003297 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc6bc0afc10 0x7fc6bc0b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:41.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.054+0000 7fc6d17fa700 1 -- 172.21.15.69:0/3645003297 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7fc6cc0df500 con 0x7fc6d4108610 2024-04-25T11:24:41.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.054+0000 7fc6d3fff700 1 --2- 172.21.15.69:0/3645003297 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc6bc0afc10 0x7fc6bc0b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:41.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.054+0000 7fc6d17fa700 1 --2- 172.21.15.69:0/3645003297 >> [v2:172.21.15.146:6800/3166305213,v1:172.21.15.146:6801/3166305213] conn(0x7fc6bc0badc0 0x7fc6bc0bd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:41.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.054+0000 7fc6d17fa700 1 -- 172.21.15.69:0/3645003297 --> [v2:172.21.15.146:6800/3166305213,v1:172.21.15.146:6801/3166305213] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc6bc0bd980 con 0x7fc6bc0badc0 2024-04-25T11:24:41.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.054+0000 7fc6d17fa700 1 -- 172.21.15.69:0/3645003297 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fc6cc0df950 con 0x7fc6d4108610 2024-04-25T11:24:41.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.054+0000 7fc6d3fff700 1 --2- 172.21.15.69:0/3645003297 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc6bc0afc10 0x7fc6bc0b2120 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4003760 tx=0x7fc6c401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:41.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.054+0000 7fc6d37fe700 1 --2- 172.21.15.69:0/3645003297 >> [v2:172.21.15.146:6800/3166305213,v1:172.21.15.146:6801/3166305213] conn(0x7fc6bc0badc0 0x7fc6bc0bd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:41.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.055+0000 7fc6d37fe700 1 --2- 172.21.15.69:0/3645003297 >> [v2:172.21.15.146:6800/3166305213,v1:172.21.15.146:6801/3166305213] conn(0x7fc6bc0badc0 0x7fc6bc0bd2b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:41.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.056+0000 7fc6d17fa700 1 -- 172.21.15.69:0/3645003297 <== osd.1 v2:172.21.15.146:6800/3166305213 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc6bc0bd980 con 0x7fc6bc0badc0 2024-04-25T11:24:41.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.089+0000 7fc6da978700 1 -- 172.21.15.69:0/3645003297 --> [v2:172.21.15.146:6800/3166305213,v1:172.21.15.146:6801/3166305213] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc6d4064220 con 0x7fc6bc0badc0 2024-04-25T11:24:41.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.090+0000 7fc6d17fa700 1 -- 172.21.15.69:0/3645003297 <== osd.1 v2:172.21.15.146:6800/3166305213 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc6d4064220 con 0x7fc6bc0badc0 2024-04-25T11:24:41.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.090+0000 7fc6da978700 1 -- 172.21.15.69:0/3645003297 >> [v2:172.21.15.146:6800/3166305213,v1:172.21.15.146:6801/3166305213] conn(0x7fc6bc0badc0 msgr2=0x7fc6bc0bd2b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:41.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.090+0000 7fc6da978700 1 --2- 172.21.15.69:0/3645003297 >> [v2:172.21.15.146:6800/3166305213,v1:172.21.15.146:6801/3166305213] conn(0x7fc6bc0badc0 0x7fc6bc0bd2b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:41.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.090+0000 7fc6da978700 1 -- 172.21.15.69:0/3645003297 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc6bc0afc10 msgr2=0x7fc6bc0b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:41.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.090+0000 7fc6da978700 1 --2- 172.21.15.69:0/3645003297 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc6bc0afc10 0x7fc6bc0b2120 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4003760 tx=0x7fc6c401a040 comp rx=0 tx=0).stop 2024-04-25T11:24:41.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.090+0000 7fc6da978700 1 -- 172.21.15.69:0/3645003297 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6d4108610 msgr2=0x7fc6d419c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:41.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.090+0000 7fc6da978700 1 --2- 172.21.15.69:0/3645003297 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6d4108610 0x7fc6d419c4d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc00ef30 tx=0x7fc6cc00c3a0 comp rx=0 tx=0).stop 2024-04-25T11:24:41.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.090+0000 7fc6d8f15700 1 -- 172.21.15.69:0/3645003297 reap_dead start 2024-04-25T11:24:41.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.090+0000 7fc6da978700 1 -- 172.21.15.69:0/3645003297 shutdown_connections 2024-04-25T11:24:41.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.090+0000 7fc6da978700 1 -- 172.21.15.69:0/3645003297 >> 172.21.15.69:0/3645003297 conn(0x7fc6d40fec90 msgr2=0x7fc6d4101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:41.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.090+0000 7fc6da978700 1 -- 172.21.15.69:0/3645003297 shutdown_connections 2024-04-25T11:24:41.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:41.091+0000 7fc6da978700 1 -- 172.21.15.69:0/3645003297 wait complete. 2024-04-25T11:24:41.693 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:41 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:24:41.693 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:41 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:24:41.693 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:41 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:24:41.693 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:41 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:24:41.730 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:41 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:24:41.730 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:41 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:24:41.730 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:41 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:24:41.730 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:41 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:24:41.849 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:41 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:24:41.849 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:41 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:24:41.849 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:41 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:24:41.849 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:41 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:24:41.949 INFO:teuthology.orchestra.run.smithi069.stdout:180388626483 2024-04-25T11:24:41.949 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-25T11:24:42.535 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:42 smithi069 ceph-mon[29188]: pgmap v317: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 7 op/s; 172 B/s, 1 objects/s recovering 2024-04-25T11:24:42.730 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:42 smithi152 ceph-mon[32701]: pgmap v317: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 7 op/s; 172 B/s, 1 objects/s recovering 2024-04-25T11:24:42.750 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:42 smithi146 ceph-mon[32821]: pgmap v317: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 7 op/s; 172 B/s, 1 objects/s recovering 2024-04-25T11:24:42.962 INFO:teuthology.orchestra.run.smithi069.stdout:77309411399 2024-04-25T11:24:42.962 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-25T11:24:44.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.614+0000 7f8a3f069700 1 -- 172.21.15.69:0/2047239703 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a38103360 msgr2=0x7f8a38103770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:44.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.614+0000 7f8a3f069700 1 --2- 172.21.15.69:0/2047239703 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a38103360 0x7f8a38103770 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c0099c0 tx=0x7f8a2c009cd0 comp rx=0 tx=0).stop 2024-04-25T11:24:44.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.614+0000 7f8a3f069700 1 -- 172.21.15.69:0/2047239703 shutdown_connections 2024-04-25T11:24:44.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.614+0000 7f8a3f069700 1 --2- 172.21.15.69:0/2047239703 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a381085e0 0x7f8a3810ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:44.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.614+0000 7f8a3f069700 1 --2- 172.21.15.69:0/2047239703 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a38103360 0x7f8a38103770 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:44.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.614+0000 7f8a3f069700 1 --2- 172.21.15.69:0/2047239703 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a38107c90 0x7f8a381080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:44.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.614+0000 7f8a3f069700 1 -- 172.21.15.69:0/2047239703 >> 172.21.15.69:0/2047239703 conn(0x7f8a380fec90 msgr2=0x7f8a381010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:44.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.615+0000 7f8a3f069700 1 -- 172.21.15.69:0/2047239703 shutdown_connections 2024-04-25T11:24:44.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.615+0000 7f8a3f069700 1 -- 172.21.15.69:0/2047239703 wait complete. 2024-04-25T11:24:44.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.615+0000 7f8a3f069700 1 Processor -- start 2024-04-25T11:24:44.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.615+0000 7f8a3f069700 1 -- start start 2024-04-25T11:24:44.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.615+0000 7f8a3f069700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a38103360 0x7f8a3819ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:44.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.615+0000 7f8a3f069700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a38107c90 0x7f8a3819bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:44.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.616+0000 7f8a3f069700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a381085e0 0x7f8a3819c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:44.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.616+0000 7f8a3f069700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8a38113130 con 0x7f8a38107c90 2024-04-25T11:24:44.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.616+0000 7f8a3f069700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8a38112fb0 con 0x7f8a38103360 2024-04-25T11:24:44.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.616+0000 7f8a3f069700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8a381132b0 con 0x7f8a381085e0 2024-04-25T11:24:44.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.616+0000 7f8a37fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a38107c90 0x7f8a3819bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:44.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.616+0000 7f8a37fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a38107c90 0x7f8a3819bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47798/0 (socket says 172.21.15.69:47798) 2024-04-25T11:24:44.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.616+0000 7f8a37fff700 1 -- 172.21.15.69:0/1755148316 learned_addr learned my addr 172.21.15.69:0/1755148316 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:24:44.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.616+0000 7f8a3ce05700 1 --2- 172.21.15.69:0/1755148316 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a38103360 0x7f8a3819ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:44.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.616+0000 7f8a37fff700 1 -- 172.21.15.69:0/1755148316 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a381085e0 msgr2=0x7f8a3819c510 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T11:24:44.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.616+0000 7f8a37fff700 1 --2- 172.21.15.69:0/1755148316 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a381085e0 0x7f8a3819c510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:44.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.616+0000 7f8a37fff700 1 -- 172.21.15.69:0/1755148316 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a38103360 msgr2=0x7f8a3819ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:44.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.616+0000 7f8a37fff700 1 --2- 172.21.15.69:0/1755148316 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a38103360 0x7f8a3819ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:44.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.616+0000 7f8a37fff700 1 -- 172.21.15.69:0/1755148316 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a381a6dc0 con 0x7f8a38107c90 2024-04-25T11:24:44.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.616+0000 7f8a37fff700 1 --2- 172.21.15.69:0/1755148316 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a38107c90 0x7f8a3819bf90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8a2400b500 tx=0x7f8a2400b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:44.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.616+0000 7f8a35ffb700 1 -- 172.21.15.69:0/1755148316 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a24015020 con 0x7f8a38107c90 2024-04-25T11:24:44.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.616+0000 7f8a35ffb700 1 -- 172.21.15.69:0/1755148316 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a24011cc0 con 0x7f8a38107c90 2024-04-25T11:24:44.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.616+0000 7f8a3f069700 1 -- 172.21.15.69:0/1755148316 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a381a70b0 con 0x7f8a38107c90 2024-04-25T11:24:44.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.616+0000 7f8a35ffb700 1 -- 172.21.15.69:0/1755148316 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a24010c80 con 0x7f8a38107c90 2024-04-25T11:24:44.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.617+0000 7f8a3f069700 1 -- 172.21.15.69:0/1755148316 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a381a7640 con 0x7f8a38107c90 2024-04-25T11:24:44.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.617+0000 7f8a3f069700 1 -- 172.21.15.69:0/1755148316 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8a38063da0 con 0x7f8a38107c90 2024-04-25T11:24:44.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.619+0000 7f8a35ffb700 1 -- 172.21.15.69:0/1755148316 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f8a24018020 con 0x7f8a38107c90 2024-04-25T11:24:44.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.619+0000 7f8a35ffb700 1 --2- 172.21.15.69:0/1755148316 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f8a280afc90 0x7f8a280b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:44.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.620+0000 7f8a3ce05700 1 --2- 172.21.15.69:0/1755148316 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f8a280afc90 0x7f8a280b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:44.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.620+0000 7f8a35ffb700 1 -- 172.21.15.69:0/1755148316 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f8a240ded70 con 0x7f8a38107c90 2024-04-25T11:24:44.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.620+0000 7f8a35ffb700 1 --2- 172.21.15.69:0/1755148316 >> [v2:172.21.15.152:6816/950669381,v1:172.21.15.152:6817/950669381] conn(0x7f8a280bae40 0x7f8a280bd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:44.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.620+0000 7f8a35ffb700 1 -- 172.21.15.69:0/1755148316 --> [v2:172.21.15.152:6816/950669381,v1:172.21.15.152:6817/950669381] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8a280bda00 con 0x7f8a280bae40 2024-04-25T11:24:44.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.620+0000 7f8a35ffb700 1 -- 172.21.15.69:0/1755148316 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f8a240df1c0 con 0x7f8a38107c90 2024-04-25T11:24:44.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.620+0000 7f8a3ce05700 1 --2- 172.21.15.69:0/1755148316 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f8a280afc90 0x7f8a280b21a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c000c00 tx=0x7f8a2c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:44.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.620+0000 7f8a3d606700 1 --2- 172.21.15.69:0/1755148316 >> [v2:172.21.15.152:6816/950669381,v1:172.21.15.152:6817/950669381] conn(0x7f8a280bae40 0x7f8a280bd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:44.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.621+0000 7f8a3d606700 1 --2- 172.21.15.69:0/1755148316 >> [v2:172.21.15.152:6816/950669381,v1:172.21.15.152:6817/950669381] conn(0x7f8a280bae40 0x7f8a280bd330 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:44.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.622+0000 7f8a35ffb700 1 -- 172.21.15.69:0/1755148316 <== osd.8 v2:172.21.15.152:6816/950669381 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8a280bda00 con 0x7f8a280bae40 2024-04-25T11:24:44.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.638+0000 7f8a3f069700 1 -- 172.21.15.69:0/1755148316 --> [v2:172.21.15.152:6816/950669381,v1:172.21.15.152:6817/950669381] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8a380641a0 con 0x7f8a280bae40 2024-04-25T11:24:44.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.638+0000 7f8a35ffb700 1 -- 172.21.15.69:0/1755148316 <== osd.8 v2:172.21.15.152:6816/950669381 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8a380641a0 con 0x7f8a280bae40 2024-04-25T11:24:44.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.639+0000 7f8a3f069700 1 -- 172.21.15.69:0/1755148316 >> [v2:172.21.15.152:6816/950669381,v1:172.21.15.152:6817/950669381] conn(0x7f8a280bae40 msgr2=0x7f8a280bd330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:44.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.639+0000 7f8a3f069700 1 --2- 172.21.15.69:0/1755148316 >> [v2:172.21.15.152:6816/950669381,v1:172.21.15.152:6817/950669381] conn(0x7f8a280bae40 0x7f8a280bd330 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:44.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.639+0000 7f8a3f069700 1 -- 172.21.15.69:0/1755148316 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f8a280afc90 msgr2=0x7f8a280b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:44.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.639+0000 7f8a3f069700 1 --2- 172.21.15.69:0/1755148316 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f8a280afc90 0x7f8a280b21a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c000c00 tx=0x7f8a2c005fb0 comp rx=0 tx=0).stop 2024-04-25T11:24:44.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.639+0000 7f8a3f069700 1 -- 172.21.15.69:0/1755148316 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a38107c90 msgr2=0x7f8a3819bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:44.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.639+0000 7f8a3f069700 1 --2- 172.21.15.69:0/1755148316 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a38107c90 0x7f8a3819bf90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8a2400b500 tx=0x7f8a2400b810 comp rx=0 tx=0).stop 2024-04-25T11:24:44.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.639+0000 7f8a3d606700 1 -- 172.21.15.69:0/1755148316 reap_dead start 2024-04-25T11:24:44.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.639+0000 7f8a3f069700 1 -- 172.21.15.69:0/1755148316 shutdown_connections 2024-04-25T11:24:44.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.639+0000 7f8a3f069700 1 -- 172.21.15.69:0/1755148316 >> 172.21.15.69:0/1755148316 conn(0x7f8a380fec90 msgr2=0x7f8a38101c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:44.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.639+0000 7f8a3f069700 1 -- 172.21.15.69:0/1755148316 shutdown_connections 2024-04-25T11:24:44.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:44.639+0000 7f8a3f069700 1 -- 172.21.15.69:0/1755148316 wait complete. 2024-04-25T11:24:44.942 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:44 smithi069 ceph-mon[29188]: pgmap v318: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 7 op/s; 156 B/s, 1 objects/s recovering 2024-04-25T11:24:45.037 INFO:teuthology.orchestra.run.smithi069.stdout:34359738449 2024-04-25T11:24:45.037 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-25T11:24:45.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:44 smithi146 ceph-mon[32821]: pgmap v318: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 7 op/s; 156 B/s, 1 objects/s recovering 2024-04-25T11:24:45.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:44 smithi152 ceph-mon[32701]: pgmap v318: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 7 op/s; 156 B/s, 1 objects/s recovering 2024-04-25T11:24:45.373 INFO:teuthology.orchestra.run.smithi069.stdout:133143986238 2024-04-25T11:24:45.374 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-25T11:24:45.880 INFO:teuthology.orchestra.run.smithi069.stdout:253403070500 2024-04-25T11:24:45.881 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-25T11:24:46.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.580+0000 7fa0d3116700 1 -- 172.21.15.69:0/3808086444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0cc104160 msgr2=0x7fa0cc10ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:46.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.580+0000 7fa0d3116700 1 --2- 172.21.15.69:0/3808086444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0cc104160 0x7fa0cc10ff20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc009990 tx=0x7fa0bc009ca0 comp rx=0 tx=0).stop 2024-04-25T11:24:46.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.580+0000 7fa0d3116700 1 -- 172.21.15.69:0/3808086444 shutdown_connections 2024-04-25T11:24:46.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.580+0000 7fa0d3116700 1 --2- 172.21.15.69:0/3808086444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0cc104160 0x7fa0cc10ff20 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:46.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.580+0000 7fa0d3116700 1 --2- 172.21.15.69:0/3808086444 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0cc1080b0 0x7fa0cc1084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:46.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.580+0000 7fa0d3116700 1 --2- 172.21.15.69:0/3808086444 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0cc103810 0x7fa0cc103c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:46.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.580+0000 7fa0d3116700 1 -- 172.21.15.69:0/3808086444 >> 172.21.15.69:0/3808086444 conn(0x7fa0cc0fe370 msgr2=0x7fa0cc1007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:46.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.580+0000 7fa0d3116700 1 -- 172.21.15.69:0/3808086444 shutdown_connections 2024-04-25T11:24:46.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.580+0000 7fa0d3116700 1 -- 172.21.15.69:0/3808086444 wait complete. 2024-04-25T11:24:46.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.581+0000 7fa0d3116700 1 Processor -- start 2024-04-25T11:24:46.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.581+0000 7fa0d3116700 1 -- start start 2024-04-25T11:24:46.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.581+0000 7fa0d3116700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0cc103810 0x7fa0cc19b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:46.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.581+0000 7fa0d3116700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0cc104160 0x7fa0cc19be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:46.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.581+0000 7fa0d3116700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0cc1080b0 0x7fa0cc19c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:46.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.581+0000 7fa0d3116700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa0cc1130f0 con 0x7fa0cc104160 2024-04-25T11:24:46.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.581+0000 7fa0d3116700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa0cc112f70 con 0x7fa0cc1080b0 2024-04-25T11:24:46.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.581+0000 7fa0d3116700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa0cc113270 con 0x7fa0cc103810 2024-04-25T11:24:46.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.581+0000 7fa0cbfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0cc104160 0x7fa0cc19be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:46.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.581+0000 7fa0cbfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0cc104160 0x7fa0cc19be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47816/0 (socket says 172.21.15.69:47816) 2024-04-25T11:24:46.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.581+0000 7fa0cbfff700 1 -- 172.21.15.69:0/2525271337 learned_addr learned my addr 172.21.15.69:0/2525271337 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:24:46.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.581+0000 7fa0d16b3700 1 --2- 172.21.15.69:0/2525271337 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0cc1080b0 0x7fa0cc19c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:46.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.581+0000 7fa0d0eb2700 1 --2- 172.21.15.69:0/2525271337 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0cc103810 0x7fa0cc19b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:46.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.581+0000 7fa0cbfff700 1 -- 172.21.15.69:0/2525271337 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0cc103810 msgr2=0x7fa0cc19b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:46.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.581+0000 7fa0cbfff700 1 --2- 172.21.15.69:0/2525271337 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0cc103810 0x7fa0cc19b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:46.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.581+0000 7fa0cbfff700 1 -- 172.21.15.69:0/2525271337 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0cc1080b0 msgr2=0x7fa0cc19c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:46.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.581+0000 7fa0cbfff700 1 --2- 172.21.15.69:0/2525271337 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0cc1080b0 0x7fa0cc19c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:46.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.581+0000 7fa0cbfff700 1 -- 172.21.15.69:0/2525271337 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0cc1a6c10 con 0x7fa0cc104160 2024-04-25T11:24:46.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.581+0000 7fa0d0eb2700 1 --2- 172.21.15.69:0/2525271337 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0cc103810 0x7fa0cc19b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:24:46.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.581+0000 7fa0d16b3700 1 --2- 172.21.15.69:0/2525271337 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0cc1080b0 0x7fa0cc19c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:24:46.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.581+0000 7fa0cbfff700 1 --2- 172.21.15.69:0/2525271337 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0cc104160 0x7fa0cc19be70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa0c000c9b0 tx=0x7fa0c000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:46.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.582+0000 7fa0c9ffb700 1 -- 172.21.15.69:0/2525271337 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0007b00 con 0x7fa0cc104160 2024-04-25T11:24:46.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.582+0000 7fa0c9ffb700 1 -- 172.21.15.69:0/2525271337 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0c0007ca0 con 0x7fa0cc104160 2024-04-25T11:24:46.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.582+0000 7fa0d3116700 1 -- 172.21.15.69:0/2525271337 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0cc1a6f00 con 0x7fa0cc104160 2024-04-25T11:24:46.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.582+0000 7fa0c9ffb700 1 -- 172.21.15.69:0/2525271337 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0018660 con 0x7fa0cc104160 2024-04-25T11:24:46.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.582+0000 7fa0d3116700 1 -- 172.21.15.69:0/2525271337 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0cc1a73b0 con 0x7fa0cc104160 2024-04-25T11:24:46.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.583+0000 7fa0d3116700 1 -- 172.21.15.69:0/2525271337 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa0cc063e20 con 0x7fa0cc104160 2024-04-25T11:24:46.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.584+0000 7fa0c9ffb700 1 -- 172.21.15.69:0/2525271337 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fa0c000f450 con 0x7fa0cc104160 2024-04-25T11:24:46.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.585+0000 7fa0c9ffb700 1 --2- 172.21.15.69:0/2525271337 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fa0b40afc10 0x7fa0b40b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:46.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.585+0000 7fa0d0eb2700 1 --2- 172.21.15.69:0/2525271337 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fa0b40afc10 0x7fa0b40b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:46.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.585+0000 7fa0c9ffb700 1 -- 172.21.15.69:0/2525271337 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7fa0c00df600 con 0x7fa0cc104160 2024-04-25T11:24:46.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.585+0000 7fa0c9ffb700 1 --2- 172.21.15.69:0/2525271337 >> [v2:172.21.15.146:6824/707256400,v1:172.21.15.146:6825/707256400] conn(0x7fa0b40badc0 0x7fa0b40bd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:46.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.585+0000 7fa0c9ffb700 1 -- 172.21.15.69:0/2525271337 --> [v2:172.21.15.146:6824/707256400,v1:172.21.15.146:6825/707256400] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa0c0007e50 con 0x7fa0b40badc0 2024-04-25T11:24:46.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.585+0000 7fa0c9ffb700 1 -- 172.21.15.69:0/2525271337 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fa0c00dfa50 con 0x7fa0cc104160 2024-04-25T11:24:46.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.585+0000 7fa0d0eb2700 1 --2- 172.21.15.69:0/2525271337 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fa0b40afc10 0x7fa0b40b2120 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc06c1c0 tx=0x7fa0c400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:46.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.585+0000 7fa0d16b3700 1 --2- 172.21.15.69:0/2525271337 >> [v2:172.21.15.146:6824/707256400,v1:172.21.15.146:6825/707256400] conn(0x7fa0b40badc0 0x7fa0b40bd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:46.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.586+0000 7fa0d16b3700 1 --2- 172.21.15.69:0/2525271337 >> [v2:172.21.15.146:6824/707256400,v1:172.21.15.146:6825/707256400] conn(0x7fa0b40badc0 0x7fa0b40bd2b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:46.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.587+0000 7fa0c9ffb700 1 -- 172.21.15.69:0/2525271337 <== osd.10 v2:172.21.15.146:6824/707256400 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa0c0007e50 con 0x7fa0b40badc0 2024-04-25T11:24:46.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.603+0000 7fa0d3116700 1 -- 172.21.15.69:0/2525271337 --> [v2:172.21.15.146:6824/707256400,v1:172.21.15.146:6825/707256400] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa0cc0642a0 con 0x7fa0b40badc0 2024-04-25T11:24:46.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.603+0000 7fa0c9ffb700 1 -- 172.21.15.69:0/2525271337 <== osd.10 v2:172.21.15.146:6824/707256400 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa0cc0642a0 con 0x7fa0b40badc0 2024-04-25T11:24:46.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.604+0000 7fa0d3116700 1 -- 172.21.15.69:0/2525271337 >> [v2:172.21.15.146:6824/707256400,v1:172.21.15.146:6825/707256400] conn(0x7fa0b40badc0 msgr2=0x7fa0b40bd2b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:46.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.604+0000 7fa0d3116700 1 --2- 172.21.15.69:0/2525271337 >> [v2:172.21.15.146:6824/707256400,v1:172.21.15.146:6825/707256400] conn(0x7fa0b40badc0 0x7fa0b40bd2b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:46.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.604+0000 7fa0d3116700 1 -- 172.21.15.69:0/2525271337 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fa0b40afc10 msgr2=0x7fa0b40b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:46.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.604+0000 7fa0d3116700 1 --2- 172.21.15.69:0/2525271337 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fa0b40afc10 0x7fa0b40b2120 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc06c1c0 tx=0x7fa0c400b040 comp rx=0 tx=0).stop 2024-04-25T11:24:46.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.604+0000 7fa0d3116700 1 -- 172.21.15.69:0/2525271337 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0cc104160 msgr2=0x7fa0cc19be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:46.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.604+0000 7fa0d3116700 1 --2- 172.21.15.69:0/2525271337 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0cc104160 0x7fa0cc19be70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa0c000c9b0 tx=0x7fa0c000cd70 comp rx=0 tx=0).stop 2024-04-25T11:24:46.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.604+0000 7fa0d16b3700 1 -- 172.21.15.69:0/2525271337 reap_dead start 2024-04-25T11:24:46.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.604+0000 7fa0d3116700 1 -- 172.21.15.69:0/2525271337 shutdown_connections 2024-04-25T11:24:46.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.604+0000 7fa0d3116700 1 -- 172.21.15.69:0/2525271337 >> 172.21.15.69:0/2525271337 conn(0x7fa0cc0fe370 msgr2=0x7fa0cc0fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:46.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.604+0000 7fa0d3116700 1 -- 172.21.15.69:0/2525271337 shutdown_connections 2024-04-25T11:24:46.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:45.604+0000 7fa0d3116700 1 -- 172.21.15.69:0/2525271337 wait complete. 2024-04-25T11:24:46.864 INFO:teuthology.orchestra.run.smithi069.stdout:274877906975 2024-04-25T11:24:46.865 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-25T11:24:47.098 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:46 smithi146 ceph-mon[32821]: pgmap v319: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 9 op/s; 142 B/s, 1 objects/s recovering 2024-04-25T11:24:47.132 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:46 smithi069 ceph-mon[29188]: pgmap v319: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 9 op/s; 142 B/s, 1 objects/s recovering 2024-04-25T11:24:47.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:46 smithi152 ceph-mon[32701]: pgmap v319: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 9 op/s; 142 B/s, 1 objects/s recovering 2024-04-25T11:24:47.611 INFO:teuthology.orchestra.run.smithi069.stdout:227633266729 2024-04-25T11:24:47.612 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-25T11:24:47.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.930+0000 7f77dc5be700 1 -- 172.21.15.69:0/2197426249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77d4108630 msgr2=0x7f77d410ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:47.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.930+0000 7f77dc5be700 1 --2- 172.21.15.69:0/2197426249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77d4108630 0x7f77d410ff90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f77c400b0d0 tx=0x7f77c400b3e0 comp rx=0 tx=0).stop 2024-04-25T11:24:47.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.930+0000 7f77dc5be700 1 -- 172.21.15.69:0/2197426249 shutdown_connections 2024-04-25T11:24:47.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.930+0000 7f77dc5be700 1 --2- 172.21.15.69:0/2197426249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77d4108630 0x7f77d410ff90 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:47.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.930+0000 7f77dc5be700 1 --2- 172.21.15.69:0/2197426249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77d41033b0 0x7f77d41037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:47.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.930+0000 7f77dc5be700 1 --2- 172.21.15.69:0/2197426249 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77d4107ce0 0x7f77d41080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:47.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.930+0000 7f77dc5be700 1 -- 172.21.15.69:0/2197426249 >> 172.21.15.69:0/2197426249 conn(0x7f77d40fee20 msgr2=0x7f77d4101240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:47.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.930+0000 7f77dc5be700 1 -- 172.21.15.69:0/2197426249 shutdown_connections 2024-04-25T11:24:47.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.930+0000 7f77dc5be700 1 -- 172.21.15.69:0/2197426249 wait complete. 2024-04-25T11:24:47.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.930+0000 7f77dc5be700 1 Processor -- start 2024-04-25T11:24:47.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.931+0000 7f77dc5be700 1 -- start start 2024-04-25T11:24:47.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.931+0000 7f77dc5be700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77d41033b0 0x7f77d41a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:47.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.931+0000 7f77dc5be700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77d4107ce0 0x7f77d41a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:47.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.931+0000 7f77dc5be700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77d4108630 0x7f77d419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:47.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.931+0000 7f77dc5be700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f77d4114260 con 0x7f77d41033b0 2024-04-25T11:24:47.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.931+0000 7f77dc5be700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f77d41140e0 con 0x7f77d4107ce0 2024-04-25T11:24:47.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.931+0000 7f77dc5be700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f77d41143e0 con 0x7f77d4108630 2024-04-25T11:24:47.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.931+0000 7f77da35a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77d41033b0 0x7f77d41a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:47.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.931+0000 7f77da35a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77d41033b0 0x7f77d41a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47840/0 (socket says 172.21.15.69:47840) 2024-04-25T11:24:47.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.931+0000 7f77da35a700 1 -- 172.21.15.69:0/3658256925 learned_addr learned my addr 172.21.15.69:0/3658256925 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:24:47.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.931+0000 7f77dab5b700 1 --2- 172.21.15.69:0/3658256925 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77d4108630 0x7f77d419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:47.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.931+0000 7f77d9b59700 1 --2- 172.21.15.69:0/3658256925 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77d4107ce0 0x7f77d41a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:47.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.931+0000 7f77da35a700 1 -- 172.21.15.69:0/3658256925 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77d4108630 msgr2=0x7f77d419aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:47.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.931+0000 7f77da35a700 1 --2- 172.21.15.69:0/3658256925 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77d4108630 0x7f77d419aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:47.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.931+0000 7f77da35a700 1 -- 172.21.15.69:0/3658256925 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77d4107ce0 msgr2=0x7f77d41a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:47.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.931+0000 7f77da35a700 1 --2- 172.21.15.69:0/3658256925 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77d4107ce0 0x7f77d41a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:47.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.931+0000 7f77da35a700 1 -- 172.21.15.69:0/3658256925 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d419b0b0 con 0x7f77d41033b0 2024-04-25T11:24:47.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.931+0000 7f77d9b59700 1 --2- 172.21.15.69:0/3658256925 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f77d4107ce0 0x7f77d41a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:24:47.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.931+0000 7f77da35a700 1 --2- 172.21.15.69:0/3658256925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77d41033b0 0x7f77d41a07e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f77cc00d950 tx=0x7f77cc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:47.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.932+0000 7f77d37fe700 1 -- 172.21.15.69:0/3658256925 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77cc016070 con 0x7f77d41033b0 2024-04-25T11:24:47.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.932+0000 7f77dc5be700 1 -- 172.21.15.69:0/3658256925 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77d419b3a0 con 0x7f77d41033b0 2024-04-25T11:24:47.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.932+0000 7f77d37fe700 1 -- 172.21.15.69:0/3658256925 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f77cc009dd0 con 0x7f77d41033b0 2024-04-25T11:24:47.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.932+0000 7f77d37fe700 1 -- 172.21.15.69:0/3658256925 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77cc00f3c0 con 0x7f77d41033b0 2024-04-25T11:24:47.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.932+0000 7f77dc5be700 1 -- 172.21.15.69:0/3658256925 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77d41a7670 con 0x7f77d41033b0 2024-04-25T11:24:47.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.932+0000 7f77dc5be700 1 -- 172.21.15.69:0/3658256925 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f77d4063e20 con 0x7f77d41033b0 2024-04-25T11:24:47.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.934+0000 7f77d37fe700 1 -- 172.21.15.69:0/3658256925 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f77cc019070 con 0x7f77d41033b0 2024-04-25T11:24:47.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.935+0000 7f77d37fe700 1 --2- 172.21.15.69:0/3658256925 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f77bc0afc10 0x7f77bc0b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:47.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.935+0000 7f77d9b59700 1 --2- 172.21.15.69:0/3658256925 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f77bc0afc10 0x7f77bc0b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:47.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.935+0000 7f77d37fe700 1 -- 172.21.15.69:0/3658256925 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f77cc0df2a0 con 0x7f77d41033b0 2024-04-25T11:24:47.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.935+0000 7f77d37fe700 1 --2- 172.21.15.69:0/3658256925 >> [v2:172.21.15.146:6816/2330294690,v1:172.21.15.146:6817/2330294690] conn(0x7f77bc0badc0 0x7f77bc0bd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:47.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.935+0000 7f77d37fe700 1 -- 172.21.15.69:0/3658256925 --> [v2:172.21.15.146:6816/2330294690,v1:172.21.15.146:6817/2330294690] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f77cc013ec0 con 0x7f77bc0badc0 2024-04-25T11:24:47.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.935+0000 7f77d37fe700 1 -- 172.21.15.69:0/3658256925 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f77cc0df6f0 con 0x7f77d41033b0 2024-04-25T11:24:47.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.935+0000 7f77d9b59700 1 --2- 172.21.15.69:0/3658256925 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f77bc0afc10 0x7f77bc0b2120 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f77c8005e10 tx=0x7f77c8005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:47.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.935+0000 7f77dab5b700 1 --2- 172.21.15.69:0/3658256925 >> [v2:172.21.15.146:6816/2330294690,v1:172.21.15.146:6817/2330294690] conn(0x7f77bc0badc0 0x7f77bc0bd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:47.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.936+0000 7f77dab5b700 1 --2- 172.21.15.69:0/3658256925 >> [v2:172.21.15.146:6816/2330294690,v1:172.21.15.146:6817/2330294690] conn(0x7f77bc0badc0 0x7f77bc0bd2b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:47.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.937+0000 7f77d37fe700 1 -- 172.21.15.69:0/3658256925 <== osd.7 v2:172.21.15.146:6816/2330294690 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f77cc013ec0 con 0x7f77bc0badc0 2024-04-25T11:24:47.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.953+0000 7f77dc5be700 1 -- 172.21.15.69:0/3658256925 --> [v2:172.21.15.146:6816/2330294690,v1:172.21.15.146:6817/2330294690] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f77d4064220 con 0x7f77bc0badc0 2024-04-25T11:24:47.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.954+0000 7f77d37fe700 1 -- 172.21.15.69:0/3658256925 <== osd.7 v2:172.21.15.146:6816/2330294690 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f77d4064220 con 0x7f77bc0badc0 2024-04-25T11:24:47.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.954+0000 7f77dc5be700 1 -- 172.21.15.69:0/3658256925 >> [v2:172.21.15.146:6816/2330294690,v1:172.21.15.146:6817/2330294690] conn(0x7f77bc0badc0 msgr2=0x7f77bc0bd2b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:47.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.954+0000 7f77dc5be700 1 --2- 172.21.15.69:0/3658256925 >> [v2:172.21.15.146:6816/2330294690,v1:172.21.15.146:6817/2330294690] conn(0x7f77bc0badc0 0x7f77bc0bd2b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:47.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.954+0000 7f77dc5be700 1 -- 172.21.15.69:0/3658256925 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f77bc0afc10 msgr2=0x7f77bc0b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:47.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.954+0000 7f77dc5be700 1 --2- 172.21.15.69:0/3658256925 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f77bc0afc10 0x7f77bc0b2120 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f77c8005e10 tx=0x7f77c8005d80 comp rx=0 tx=0).stop 2024-04-25T11:24:47.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.954+0000 7f77dc5be700 1 -- 172.21.15.69:0/3658256925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77d41033b0 msgr2=0x7f77d41a07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:47.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.954+0000 7f77dc5be700 1 --2- 172.21.15.69:0/3658256925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f77d41033b0 0x7f77d41a07e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f77cc00d950 tx=0x7f77cc00dc60 comp rx=0 tx=0).stop 2024-04-25T11:24:47.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.954+0000 7f77dab5b700 1 -- 172.21.15.69:0/3658256925 reap_dead start 2024-04-25T11:24:47.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.954+0000 7f77dc5be700 1 -- 172.21.15.69:0/3658256925 shutdown_connections 2024-04-25T11:24:47.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.954+0000 7f77dc5be700 1 -- 172.21.15.69:0/3658256925 >> 172.21.15.69:0/3658256925 conn(0x7f77d40fee20 msgr2=0x7f77d4107450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:47.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.954+0000 7f77dc5be700 1 -- 172.21.15.69:0/3658256925 shutdown_connections 2024-04-25T11:24:47.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:47.954+0000 7f77dc5be700 1 -- 172.21.15.69:0/3658256925 wait complete. 2024-04-25T11:24:48.560 INFO:teuthology.orchestra.run.smithi069.stdout:55834574925 2024-04-25T11:24:48.560 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-25T11:24:49.059 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:48 smithi069 ceph-mon[29188]: pgmap v320: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 7 op/s; 142 B/s, 1 objects/s recovering 2024-04-25T11:24:49.094 INFO:teuthology.orchestra.run.smithi069.stdout:154618822711 2024-04-25T11:24:49.094 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-25T11:24:49.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:48 smithi152 ceph-mon[32701]: pgmap v320: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 7 op/s; 142 B/s, 1 objects/s recovering 2024-04-25T11:24:49.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:48 smithi146 ceph-mon[32821]: pgmap v320: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 7 op/s; 142 B/s, 1 objects/s recovering 2024-04-25T11:24:49.779 INFO:teuthology.orchestra.run.smithi069.stdout:201863462958 2024-04-25T11:24:49.779 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-25T11:24:50.376 INFO:teuthology.orchestra.run.smithi069.stdout:300647710745 2024-04-25T11:24:50.376 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-25T11:24:50.832 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:24:51.141 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:50 smithi069 ceph-mon[29188]: pgmap v321: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:24:51.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:50 smithi152 ceph-mon[32701]: pgmap v321: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:24:51.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:50 smithi146 ceph-mon[32821]: pgmap v321: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:24:52.200 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:24:52.201 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:24:52.202 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:24:52.202 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:24:53.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:52 smithi069 ceph-mon[29188]: pgmap v322: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:24:53.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:52 smithi152 ceph-mon[32701]: pgmap v322: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:24:53.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.318+0000 7f20c5496700 1 -- 172.21.15.69:0/2618925815 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20c0103350 msgr2=0x7f20c0103760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:53.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.318+0000 7f20c5496700 1 --2- 172.21.15.69:0/2618925815 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20c0103350 0x7f20c0103760 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f20b00099c0 tx=0x7f20b0009cd0 comp rx=0 tx=0).stop 2024-04-25T11:24:53.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.318+0000 7f20c5496700 1 -- 172.21.15.69:0/2618925815 shutdown_connections 2024-04-25T11:24:53.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.318+0000 7f20c5496700 1 --2- 172.21.15.69:0/2618925815 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20c01085d0 0x7f20c010ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:53.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.318+0000 7f20c5496700 1 --2- 172.21.15.69:0/2618925815 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20c0103350 0x7f20c0103760 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:53.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.318+0000 7f20c5496700 1 --2- 172.21.15.69:0/2618925815 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20c0107c80 0x7f20c0108090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:53.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.318+0000 7f20c5496700 1 -- 172.21.15.69:0/2618925815 >> 172.21.15.69:0/2618925815 conn(0x7f20c00fec30 msgr2=0x7f20c0101050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:53.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.318+0000 7f20c5496700 1 -- 172.21.15.69:0/2618925815 shutdown_connections 2024-04-25T11:24:53.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.319+0000 7f20c5496700 1 -- 172.21.15.69:0/2618925815 wait complete. 2024-04-25T11:24:53.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.319+0000 7f20c5496700 1 Processor -- start 2024-04-25T11:24:53.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.319+0000 7f20c5496700 1 -- start start 2024-04-25T11:24:53.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.320+0000 7f20c5496700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20c0103350 0x7f20c00ff110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:53.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.320+0000 7f20c5496700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20c0107c80 0x7f20c00ff650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:53.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.320+0000 7f20c5496700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20c01085d0 0x7f20c00ffb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:53.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.320+0000 7f20c5496700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f20c01130b0 con 0x7f20c01085d0 2024-04-25T11:24:53.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.320+0000 7f20c5496700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f20c0112f30 con 0x7f20c0107c80 2024-04-25T11:24:53.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.320+0000 7f20c5496700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f20c0113230 con 0x7f20c0103350 2024-04-25T11:24:53.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.320+0000 7f20beffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20c0103350 0x7f20c00ff110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:53.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.320+0000 7f20beffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20c0103350 0x7f20c00ff110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.69:36044/0 (socket says 172.21.15.69:36044) 2024-04-25T11:24:53.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.320+0000 7f20beffd700 1 -- 172.21.15.69:0/4158186426 learned_addr learned my addr 172.21.15.69:0/4158186426 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:24:53.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.321+0000 7f20beffd700 1 -- 172.21.15.69:0/4158186426 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20c0107c80 msgr2=0x7f20c00ff650 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T11:24:53.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.321+0000 7f20beffd700 1 --2- 172.21.15.69:0/4158186426 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20c0107c80 0x7f20c00ff650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:53.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.321+0000 7f20beffd700 1 -- 172.21.15.69:0/4158186426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20c01085d0 msgr2=0x7f20c00ffb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:53.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.321+0000 7f20beffd700 1 --2- 172.21.15.69:0/4158186426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20c01085d0 0x7f20c00ffb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:53.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.321+0000 7f20beffd700 1 -- 172.21.15.69:0/4158186426 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20c01a8f70 con 0x7f20c0103350 2024-04-25T11:24:53.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.321+0000 7f20beffd700 1 --2- 172.21.15.69:0/4158186426 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20c0103350 0x7f20c00ff110 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f20b0005f50 tx=0x7f20b000f460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:53.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.321+0000 7f20a7fff700 1 -- 172.21.15.69:0/4158186426 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b0027070 con 0x7f20c0103350 2024-04-25T11:24:53.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.321+0000 7f20a7fff700 1 -- 172.21.15.69:0/4158186426 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20b0022470 con 0x7f20c0103350 2024-04-25T11:24:53.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.321+0000 7f20a7fff700 1 -- 172.21.15.69:0/4158186426 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b0017650 con 0x7f20c0103350 2024-04-25T11:24:53.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.323+0000 7f20c5496700 1 -- 172.21.15.69:0/4158186426 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20c01a9200 con 0x7f20c0103350 2024-04-25T11:24:53.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.323+0000 7f20c5496700 1 -- 172.21.15.69:0/4158186426 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20c01a9600 con 0x7f20c0103350 2024-04-25T11:24:53.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.324+0000 7f20c5496700 1 -- 172.21.15.69:0/4158186426 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f20c0063da0 con 0x7f20c0103350 2024-04-25T11:24:53.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.327+0000 7f20a7fff700 1 -- 172.21.15.69:0/4158186426 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f20b0022620 con 0x7f20c0103350 2024-04-25T11:24:53.321 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:24:53.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.328+0000 7f20a7fff700 1 --2- 172.21.15.69:0/4158186426 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f20a80afd10 0x7f20a80b2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:53.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.328+0000 7f20a7fff700 1 -- 172.21.15.69:0/4158186426 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f20b00e04f0 con 0x7f20c0103350 2024-04-25T11:24:53.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.328+0000 7f20a7fff700 1 --2- 172.21.15.69:0/4158186426 >> [v2:172.21.15.69:6810/1809071296,v1:172.21.15.69:6811/1809071296] conn(0x7f20a80baec0 0x7f20a80bd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:53.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.328+0000 7f20a7fff700 1 -- 172.21.15.69:0/4158186426 --> [v2:172.21.15.69:6810/1809071296,v1:172.21.15.69:6811/1809071296] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f20b002de50 con 0x7f20a80baec0 2024-04-25T11:24:53.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.328+0000 7f20a7fff700 1 -- 172.21.15.69:0/4158186426 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f20b00e08e0 con 0x7f20c0103350 2024-04-25T11:24:53.323 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:24:53.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.330+0000 7f20bf7fe700 1 --2- 172.21.15.69:0/4158186426 >> [v2:172.21.15.69:6810/1809071296,v1:172.21.15.69:6811/1809071296] conn(0x7f20a80baec0 0x7f20a80bd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:53.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.332+0000 7f20be7fc700 1 --2- 172.21.15.69:0/4158186426 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f20a80afd10 0x7f20a80b2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:53.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.333+0000 7f20be7fc700 1 --2- 172.21.15.69:0/4158186426 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f20a80afd10 0x7f20a80b2220 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f20c0109a50 tx=0x7f20b4005c10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:53.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.334+0000 7f20bf7fe700 1 --2- 172.21.15.69:0/4158186426 >> [v2:172.21.15.69:6810/1809071296,v1:172.21.15.69:6811/1809071296] conn(0x7f20a80baec0 0x7f20a80bd3b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:53.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.335+0000 7f20a7fff700 1 -- 172.21.15.69:0/4158186426 <== osd.3 v2:172.21.15.69:6810/1809071296 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f20b002de50 con 0x7f20a80baec0 2024-04-25T11:24:53.330 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:24:53.331 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:24:53.331 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:24:53.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:52 smithi146 ceph-mon[32821]: pgmap v322: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:24:53.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.365+0000 7f20c5496700 1 -- 172.21.15.69:0/4158186426 --> [v2:172.21.15.69:6810/1809071296,v1:172.21.15.69:6811/1809071296] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f20c0064220 con 0x7f20a80baec0 2024-04-25T11:24:53.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.366+0000 7f20a7fff700 1 -- 172.21.15.69:0/4158186426 <== osd.3 v2:172.21.15.69:6810/1809071296 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f20c0064220 con 0x7f20a80baec0 2024-04-25T11:24:53.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.368+0000 7f20a5ffb700 1 -- 172.21.15.69:0/4158186426 >> [v2:172.21.15.69:6810/1809071296,v1:172.21.15.69:6811/1809071296] conn(0x7f20a80baec0 msgr2=0x7f20a80bd3b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:53.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.368+0000 7f20a5ffb700 1 --2- 172.21.15.69:0/4158186426 >> [v2:172.21.15.69:6810/1809071296,v1:172.21.15.69:6811/1809071296] conn(0x7f20a80baec0 0x7f20a80bd3b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:53.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.368+0000 7f20a5ffb700 1 -- 172.21.15.69:0/4158186426 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f20a80afd10 msgr2=0x7f20a80b2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:53.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.368+0000 7f20a5ffb700 1 --2- 172.21.15.69:0/4158186426 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f20a80afd10 0x7f20a80b2220 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f20c0109a50 tx=0x7f20b4005c10 comp rx=0 tx=0).stop 2024-04-25T11:24:53.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.368+0000 7f20a5ffb700 1 -- 172.21.15.69:0/4158186426 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20c0103350 msgr2=0x7f20c00ff110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:53.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.368+0000 7f20a5ffb700 1 --2- 172.21.15.69:0/4158186426 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20c0103350 0x7f20c00ff110 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f20b0005f50 tx=0x7f20b000f460 comp rx=0 tx=0).stop 2024-04-25T11:24:53.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.368+0000 7f20bf7fe700 1 -- 172.21.15.69:0/4158186426 reap_dead start 2024-04-25T11:24:53.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.369+0000 7f20a5ffb700 1 -- 172.21.15.69:0/4158186426 shutdown_connections 2024-04-25T11:24:53.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.369+0000 7f20a5ffb700 1 -- 172.21.15.69:0/4158186426 >> 172.21.15.69:0/4158186426 conn(0x7f20c00fec30 msgr2=0x7f20c006c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:53.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.369+0000 7f20a5ffb700 1 -- 172.21.15.69:0/4158186426 shutdown_connections 2024-04-25T11:24:53.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:53.369+0000 7f20a5ffb700 1 -- 172.21.15.69:0/4158186426 wait complete. 2024-04-25T11:24:54.716 INFO:teuthology.orchestra.run.smithi069.stdout:107374182469 2024-04-25T11:24:54.716 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-25T11:24:55.030 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:54 smithi069 ceph-mon[29188]: pgmap v323: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-25T11:24:55.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:54 smithi152 ceph-mon[32701]: pgmap v323: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-25T11:24:55.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:54 smithi146 ceph-mon[32821]: pgmap v323: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-04-25T11:24:56.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.168+0000 7fecd040d700 1 -- 172.21.15.69:0/882331356 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fecc8105330 msgr2=0x7fecc8105740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:56.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.168+0000 7fecd040d700 1 --2- 172.21.15.69:0/882331356 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fecc8105330 0x7fecc8105740 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fecc4009990 tx=0x7fecc4009ca0 comp rx=0 tx=0).stop 2024-04-25T11:24:56.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.168+0000 7fecd040d700 1 -- 172.21.15.69:0/882331356 shutdown_connections 2024-04-25T11:24:56.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.168+0000 7fecd040d700 1 --2- 172.21.15.69:0/882331356 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fecc810e7a0 0x7fecc8110c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:56.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.168+0000 7fecd040d700 1 --2- 172.21.15.69:0/882331356 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fecc8105330 0x7fecc8105740 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:56.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.168+0000 7fecd040d700 1 --2- 172.21.15.69:0/882331356 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fecc8105c80 0x7fecc810e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:56.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.168+0000 7fecd040d700 1 -- 172.21.15.69:0/882331356 >> 172.21.15.69:0/882331356 conn(0x7fecc80fec90 msgr2=0x7fecc81010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:56.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.168+0000 7fecd040d700 1 -- 172.21.15.69:0/882331356 shutdown_connections 2024-04-25T11:24:56.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.168+0000 7fecd040d700 1 -- 172.21.15.69:0/882331356 wait complete. 2024-04-25T11:24:56.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.169+0000 7fecd040d700 1 Processor -- start 2024-04-25T11:24:56.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.169+0000 7fecd040d700 1 -- start start 2024-04-25T11:24:56.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.169+0000 7fecd040d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fecc8105330 0x7fecc8103020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:56.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.169+0000 7fecd040d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fecc8105c80 0x7fecc8103560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:56.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.169+0000 7fecd040d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fecc810e7a0 0x7fecc8103aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:56.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.169+0000 7fecd040d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fecc81142f0 con 0x7fecc810e7a0 2024-04-25T11:24:56.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.169+0000 7fecd040d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fecc8114170 con 0x7fecc8105c80 2024-04-25T11:24:56.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.169+0000 7fecd040d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fecc8114470 con 0x7fecc8105330 2024-04-25T11:24:56.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.169+0000 7fecce1a9700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fecc8105330 0x7fecc8103020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:56.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.169+0000 7fecce9aa700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fecc810e7a0 0x7fecc8103aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:56.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.169+0000 7fecce1a9700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fecc8105330 0x7fecc8103020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.69:36054/0 (socket says 172.21.15.69:36054) 2024-04-25T11:24:56.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.169+0000 7fecce1a9700 1 -- 172.21.15.69:0/2146471404 learned_addr learned my addr 172.21.15.69:0/2146471404 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:24:56.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.169+0000 7feccd9a8700 1 --2- 172.21.15.69:0/2146471404 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fecc8105c80 0x7fecc8103560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:56.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.170+0000 7fecce1a9700 1 -- 172.21.15.69:0/2146471404 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fecc8105c80 msgr2=0x7fecc8103560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:56.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.170+0000 7fecce1a9700 1 --2- 172.21.15.69:0/2146471404 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fecc8105c80 0x7fecc8103560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:56.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.170+0000 7fecce1a9700 1 -- 172.21.15.69:0/2146471404 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fecc810e7a0 msgr2=0x7fecc8103aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:56.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.170+0000 7fecce1a9700 1 --2- 172.21.15.69:0/2146471404 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fecc810e7a0 0x7fecc8103aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:56.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.170+0000 7fecce1a9700 1 -- 172.21.15.69:0/2146471404 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc81a90e0 con 0x7fecc8105330 2024-04-25T11:24:56.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.170+0000 7fecce1a9700 1 --2- 172.21.15.69:0/2146471404 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fecc8105330 0x7fecc8103020 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fecc4000c00 tx=0x7fecc40057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:56.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.170+0000 7fecbf7fe700 1 -- 172.21.15.69:0/2146471404 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc4026070 con 0x7fecc8105330 2024-04-25T11:24:56.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.170+0000 7fecd040d700 1 -- 172.21.15.69:0/2146471404 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecc81a9370 con 0x7fecc8105330 2024-04-25T11:24:56.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.170+0000 7fecd040d700 1 -- 172.21.15.69:0/2146471404 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecc81a9820 con 0x7fecc8105330 2024-04-25T11:24:56.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.170+0000 7fecbf7fe700 1 -- 172.21.15.69:0/2146471404 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fecc400fb30 con 0x7fecc8105330 2024-04-25T11:24:56.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.170+0000 7fecbf7fe700 1 -- 172.21.15.69:0/2146471404 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc4021bb0 con 0x7fecc8105330 2024-04-25T11:24:56.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.171+0000 7fecd040d700 1 -- 172.21.15.69:0/2146471404 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecc8051190 con 0x7fecc8105330 2024-04-25T11:24:56.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.173+0000 7fecbf7fe700 1 -- 172.21.15.69:0/2146471404 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fecc4029070 con 0x7fecc8105330 2024-04-25T11:24:56.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.173+0000 7fecbf7fe700 1 --2- 172.21.15.69:0/2146471404 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fecb40afc90 0x7fecb40b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:56.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.174+0000 7feccd9a8700 1 --2- 172.21.15.69:0/2146471404 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fecb40afc90 0x7fecb40b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:56.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.174+0000 7fecbf7fe700 1 -- 172.21.15.69:0/2146471404 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7fecc40dfef0 con 0x7fecc8105330 2024-04-25T11:24:56.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.174+0000 7feccd9a8700 1 --2- 172.21.15.69:0/2146471404 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fecb40afc90 0x7fecb40b21a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fecc8102370 tx=0x7fecb8009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:56.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.178+0000 7fecbf7fe700 1 -- 172.21.15.69:0/2146471404 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fecc410eff0 con 0x7fecc8105330 2024-04-25T11:24:56.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.380+0000 7fecd040d700 1 -- 172.21.15.69:0/2146471404 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fecc802e510 con 0x7fecc8105330 2024-04-25T11:24:56.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.381+0000 7fecbf7fe700 1 -- 172.21.15.69:0/2146471404 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fecc4013070 con 0x7fecc8105330 2024-04-25T11:24:56.374 INFO:teuthology.orchestra.run.smithi069.stdout:77309411403 2024-04-25T11:24:56.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.383+0000 7fecd040d700 1 -- 172.21.15.69:0/2146471404 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fecb40afc90 msgr2=0x7fecb40b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:56.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.383+0000 7fecd040d700 1 --2- 172.21.15.69:0/2146471404 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fecb40afc90 0x7fecb40b21a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fecc8102370 tx=0x7fecb8009190 comp rx=0 tx=0).stop 2024-04-25T11:24:56.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.383+0000 7fecd040d700 1 -- 172.21.15.69:0/2146471404 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fecc8105330 msgr2=0x7fecc8103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:56.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.383+0000 7fecd040d700 1 --2- 172.21.15.69:0/2146471404 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fecc8105330 0x7fecc8103020 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fecc4000c00 tx=0x7fecc40057c0 comp rx=0 tx=0).stop 2024-04-25T11:24:56.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.383+0000 7fecd040d700 1 -- 172.21.15.69:0/2146471404 shutdown_connections 2024-04-25T11:24:56.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.383+0000 7fecd040d700 1 --2- 172.21.15.69:0/2146471404 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fecb40afc90 0x7fecb40b21a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:56.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.383+0000 7fecd040d700 1 --2- 172.21.15.69:0/2146471404 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fecc810e7a0 0x7fecc8103aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:56.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.383+0000 7fecd040d700 1 --2- 172.21.15.69:0/2146471404 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fecc8105330 0x7fecc8103020 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:56.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.383+0000 7fecd040d700 1 --2- 172.21.15.69:0/2146471404 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fecc8105c80 0x7fecc8103560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:56.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.383+0000 7fecd040d700 1 -- 172.21.15.69:0/2146471404 >> 172.21.15.69:0/2146471404 conn(0x7fecc80fec90 msgr2=0x7fecc80ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:56.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.383+0000 7fecd040d700 1 -- 172.21.15.69:0/2146471404 shutdown_connections 2024-04-25T11:24:56.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:56.383+0000 7fecd040d700 1 -- 172.21.15.69:0/2146471404 wait complete. 2024-04-25T11:24:56.942 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:56 smithi069 ceph-mon[29188]: pgmap v324: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-04-25T11:24:56.943 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:56 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2146471404' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T11:24:57.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:56 smithi152 ceph-mon[32701]: pgmap v324: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-04-25T11:24:57.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:56 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/2146471404' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T11:24:57.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:56 smithi146 ceph-mon[32821]: pgmap v324: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-04-25T11:24:57.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:56 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/2146471404' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T11:24:58.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:58.874+0000 7f8b0e025700 1 -- 172.21.15.69:0/4036271106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b08107ce0 msgr2=0x7f8b081080f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:58.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:58.874+0000 7f8b0e025700 1 --2- 172.21.15.69:0/4036271106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b08107ce0 0x7f8b081080f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8af00099c0 tx=0x7f8af0009cd0 comp rx=0 tx=0).stop 2024-04-25T11:24:58.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:58.874+0000 7f8b0e025700 1 -- 172.21.15.69:0/4036271106 shutdown_connections 2024-04-25T11:24:58.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:58.874+0000 7f8b0e025700 1 --2- 172.21.15.69:0/4036271106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b08108630 0x7f8b0810ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:58.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:58.874+0000 7f8b0e025700 1 --2- 172.21.15.69:0/4036271106 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b081033b0 0x7f8b081037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:58.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:58.874+0000 7f8b0e025700 1 --2- 172.21.15.69:0/4036271106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b08107ce0 0x7f8b081080f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:58.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:58.874+0000 7f8b0e025700 1 -- 172.21.15.69:0/4036271106 >> 172.21.15.69:0/4036271106 conn(0x7f8b080fee20 msgr2=0x7f8b08101240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:58.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:58.874+0000 7f8b0e025700 1 -- 172.21.15.69:0/4036271106 shutdown_connections 2024-04-25T11:24:58.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:58.874+0000 7f8b0e025700 1 -- 172.21.15.69:0/4036271106 wait complete. 2024-04-25T11:24:58.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:58.875+0000 7f8b0e025700 1 Processor -- start 2024-04-25T11:24:58.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:58.875+0000 7f8b0e025700 1 -- start start 2024-04-25T11:24:58.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:58.875+0000 7f8b0e025700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b081033b0 0x7f8b0819baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:58.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:58.875+0000 7f8b0e025700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b08107ce0 0x7f8b0819c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:58.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:58.875+0000 7f8b0e025700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b08108630 0x7f8b0819c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:58.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:58.875+0000 7f8b0e025700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8b081130f0 con 0x7f8b081033b0 2024-04-25T11:24:58.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:58.875+0000 7f8b0e025700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8b08112f70 con 0x7f8b08108630 2024-04-25T11:24:58.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:58.875+0000 7f8b0e025700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8b08113270 con 0x7f8b08107ce0 2024-04-25T11:24:58.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:58.875+0000 7f8b077fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b081033b0 0x7f8b0819baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:58.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:58.875+0000 7f8b077fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b081033b0 0x7f8b0819baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56110/0 (socket says 172.21.15.69:56110) 2024-04-25T11:24:58.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:58.875+0000 7f8b077fe700 1 -- 172.21.15.69:0/2012532776 learned_addr learned my addr 172.21.15.69:0/2012532776 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:24:58.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:58.875+0000 7f8afeffd700 1 --2- 172.21.15.69:0/2012532776 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b08107ce0 0x7f8b0819c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:58.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:58.875+0000 7f8b07fff700 1 --2- 172.21.15.69:0/2012532776 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b08108630 0x7f8b0819c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:58.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:58.876+0000 7f8b077fe700 1 -- 172.21.15.69:0/2012532776 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b08107ce0 msgr2=0x7f8b0819c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:58.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:58.876+0000 7f8b077fe700 1 --2- 172.21.15.69:0/2012532776 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b08107ce0 0x7f8b0819c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:58.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:58.876+0000 7f8b077fe700 1 -- 172.21.15.69:0/2012532776 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b08108630 msgr2=0x7f8b0819c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:58.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:58.876+0000 7f8b077fe700 1 --2- 172.21.15.69:0/2012532776 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b08108630 0x7f8b0819c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:58.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:58.876+0000 7f8b077fe700 1 -- 172.21.15.69:0/2012532776 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b081998e0 con 0x7f8b081033b0 2024-04-25T11:24:58.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:58.876+0000 7f8b077fe700 1 --2- 172.21.15.69:0/2012532776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b081033b0 0x7f8b0819baf0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8af800d950 tx=0x7f8af800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:58.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:58.876+0000 7f8b057fa700 1 -- 172.21.15.69:0/2012532776 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af8016070 con 0x7f8b081033b0 2024-04-25T11:24:58.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:58.876+0000 7f8b057fa700 1 -- 172.21.15.69:0/2012532776 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8af800fd20 con 0x7f8b081033b0 2024-04-25T11:24:58.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:58.876+0000 7f8b057fa700 1 -- 172.21.15.69:0/2012532776 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af800e3b0 con 0x7f8b081033b0 2024-04-25T11:24:58.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:58.876+0000 7f8b0e025700 1 -- 172.21.15.69:0/2012532776 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b08199bd0 con 0x7f8b081033b0 2024-04-25T11:24:58.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:58.876+0000 7f8b0e025700 1 -- 172.21.15.69:0/2012532776 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b0819a130 con 0x7f8b081033b0 2024-04-25T11:24:58.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:58.877+0000 7f8b0e025700 1 -- 172.21.15.69:0/2012532776 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b08051190 con 0x7f8b081033b0 2024-04-25T11:24:58.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:58.879+0000 7f8b057fa700 1 -- 172.21.15.69:0/2012532776 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f8af8019070 con 0x7f8b081033b0 2024-04-25T11:24:58.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:58.879+0000 7f8b057fa700 1 --2- 172.21.15.69:0/2012532776 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f8af40afc90 0x7f8af40b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:24:58.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:58.879+0000 7f8afeffd700 1 --2- 172.21.15.69:0/2012532776 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f8af40afc90 0x7f8af40b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:24:58.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:58.879+0000 7f8b057fa700 1 -- 172.21.15.69:0/2012532776 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f8af80deb40 con 0x7f8b081033b0 2024-04-25T11:24:58.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:58.880+0000 7f8afeffd700 1 --2- 172.21.15.69:0/2012532776 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f8af40afc90 0x7f8af40b21a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8af0009990 tx=0x7f8af0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:24:58.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:58.883+0000 7f8b057fa700 1 -- 172.21.15.69:0/2012532776 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8af80a7260 con 0x7f8b081033b0 2024-04-25T11:24:58.942 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:58 smithi069 ceph-mon[29188]: pgmap v325: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-25T11:24:59.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.089+0000 7f8b0e025700 1 -- 172.21.15.69:0/2012532776 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f8b0819b510 con 0x7f8b081033b0 2024-04-25T11:24:59.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.089+0000 7f8b057fa700 1 -- 172.21.15.69:0/2012532776 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8af80a6b60 con 0x7f8b081033b0 2024-04-25T11:24:59.083 INFO:teuthology.orchestra.run.smithi069.stdout:154618822715 2024-04-25T11:24:59.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.091+0000 7f8b0e025700 1 -- 172.21.15.69:0/2012532776 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f8af40afc90 msgr2=0x7f8af40b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:59.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.091+0000 7f8b0e025700 1 --2- 172.21.15.69:0/2012532776 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f8af40afc90 0x7f8af40b21a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8af0009990 tx=0x7f8af0005fb0 comp rx=0 tx=0).stop 2024-04-25T11:24:59.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.091+0000 7f8b0e025700 1 -- 172.21.15.69:0/2012532776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b081033b0 msgr2=0x7f8b0819baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:24:59.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.091+0000 7f8b0e025700 1 --2- 172.21.15.69:0/2012532776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b081033b0 0x7f8b0819baf0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8af800d950 tx=0x7f8af800dc60 comp rx=0 tx=0).stop 2024-04-25T11:24:59.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.092+0000 7f8b0e025700 1 -- 172.21.15.69:0/2012532776 shutdown_connections 2024-04-25T11:24:59.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.092+0000 7f8b0e025700 1 --2- 172.21.15.69:0/2012532776 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f8af40afc90 0x7f8af40b21a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:59.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.092+0000 7f8b0e025700 1 --2- 172.21.15.69:0/2012532776 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b08108630 0x7f8b0819c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:59.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.092+0000 7f8b0e025700 1 --2- 172.21.15.69:0/2012532776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b081033b0 0x7f8b0819baf0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:59.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.092+0000 7f8b0e025700 1 --2- 172.21.15.69:0/2012532776 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b08107ce0 0x7f8b0819c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:24:59.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.092+0000 7f8b0e025700 1 -- 172.21.15.69:0/2012532776 >> 172.21.15.69:0/2012532776 conn(0x7f8b080fee20 msgr2=0x7f8b08109590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:24:59.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.092+0000 7f8b0e025700 1 -- 172.21.15.69:0/2012532776 shutdown_connections 2024-04-25T11:24:59.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.092+0000 7f8b0e025700 1 -- 172.21.15.69:0/2012532776 wait complete. 2024-04-25T11:24:59.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:58 smithi152 ceph-mon[32701]: pgmap v325: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-25T11:24:59.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:58 smithi146 ceph-mon[32821]: pgmap v325: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-25T11:25:00.187 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:59 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2012532776' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T11:25:00.188 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:24:59 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/353895102' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T11:25:00.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:59 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/2012532776' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T11:25:00.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:24:59 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/353895102' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T11:25:00.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:59 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/2012532776' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T11:25:00.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:24:59 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/353895102' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T11:25:01.173 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:00 smithi069 ceph-mon[29188]: pgmap v326: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-04-25T11:25:01.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:00 smithi152 ceph-mon[32701]: pgmap v326: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-04-25T11:25:01.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:00 smithi146 ceph-mon[32821]: pgmap v326: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-04-25T11:25:03.192 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:02 smithi069 ceph-mon[29188]: pgmap v327: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:25:03.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:02 smithi152 ceph-mon[32701]: pgmap v327: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:25:03.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:02 smithi146 ceph-mon[32821]: pgmap v327: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:25:03.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.722+0000 7f7dc9290700 1 -- 172.21.15.69:0/2567011938 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7dc4109780 msgr2=0x7f7dc41110c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:03.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.722+0000 7f7dc9290700 1 --2- 172.21.15.69:0/2567011938 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7dc4109780 0x7f7dc41110c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc00b0a0 tx=0x7f7dbc00b3b0 comp rx=0 tx=0).stop 2024-04-25T11:25:03.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.722+0000 7f7dc9290700 1 -- 172.21.15.69:0/2567011938 shutdown_connections 2024-04-25T11:25:03.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.722+0000 7f7dc9290700 1 --2- 172.21.15.69:0/2567011938 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7dc4109780 0x7f7dc41110c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:03.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.722+0000 7f7dc9290700 1 --2- 172.21.15.69:0/2567011938 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7dc4102a10 0x7f7dc4102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:03.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.722+0000 7f7dc9290700 1 --2- 172.21.15.69:0/2567011938 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7dc4108e30 0x7f7dc4109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:03.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.722+0000 7f7dc9290700 1 -- 172.21.15.69:0/2567011938 >> 172.21.15.69:0/2567011938 conn(0x7f7dc40fe3f0 msgr2=0x7f7dc4100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:03.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.722+0000 7f7dc9290700 1 -- 172.21.15.69:0/2567011938 shutdown_connections 2024-04-25T11:25:03.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.722+0000 7f7dc9290700 1 -- 172.21.15.69:0/2567011938 wait complete. 2024-04-25T11:25:03.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.723+0000 7f7dc9290700 1 Processor -- start 2024-04-25T11:25:03.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.723+0000 7f7dc9290700 1 -- start start 2024-04-25T11:25:03.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.724+0000 7f7dc9290700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7dc4102a10 0x7f7dc41a1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:03.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.724+0000 7f7dc9290700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7dc4108e30 0x7f7dc41a2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:03.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.724+0000 7f7dc9290700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7dc4109780 0x7f7dc41ad070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:03.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.724+0000 7f7dc9290700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc4114310 con 0x7f7dc4102a10 2024-04-25T11:25:03.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.724+0000 7f7dc9290700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc4114190 con 0x7f7dc4108e30 2024-04-25T11:25:03.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.724+0000 7f7dc9290700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc4114490 con 0x7f7dc4109780 2024-04-25T11:25:03.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.724+0000 7f7dc2ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7dc4102a10 0x7f7dc41a1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:03.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.724+0000 7f7dc2ffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7dc4102a10 0x7f7dc41a1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49284/0 (socket says 172.21.15.69:49284) 2024-04-25T11:25:03.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.724+0000 7f7dc2ffd700 1 -- 172.21.15.69:0/2532587842 learned_addr learned my addr 172.21.15.69:0/2532587842 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:03.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.724+0000 7f7dc27fc700 1 --2- 172.21.15.69:0/2532587842 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7dc4108e30 0x7f7dc41a2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:03.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.724+0000 7f7dc37fe700 1 --2- 172.21.15.69:0/2532587842 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7dc4109780 0x7f7dc41ad070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:03.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.724+0000 7f7dc2ffd700 1 -- 172.21.15.69:0/2532587842 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7dc4109780 msgr2=0x7f7dc41ad070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:03.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.724+0000 7f7dc2ffd700 1 --2- 172.21.15.69:0/2532587842 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7dc4109780 0x7f7dc41ad070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:03.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.724+0000 7f7dc2ffd700 1 -- 172.21.15.69:0/2532587842 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7dc4108e30 msgr2=0x7f7dc41a2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:03.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.724+0000 7f7dc2ffd700 1 --2- 172.21.15.69:0/2532587842 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7dc4108e30 0x7f7dc41a2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:03.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.724+0000 7f7dc2ffd700 1 -- 172.21.15.69:0/2532587842 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dc41ad5b0 con 0x7f7dc4102a10 2024-04-25T11:25:03.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.725+0000 7f7dc2ffd700 1 --2- 172.21.15.69:0/2532587842 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7dc4102a10 0x7f7dc41a1ff0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7db400b500 tx=0x7f7db400b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:03.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.725+0000 7f7dabfff700 1 -- 172.21.15.69:0/2532587842 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db4015020 con 0x7f7dc4102a10 2024-04-25T11:25:03.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.725+0000 7f7dc9290700 1 -- 172.21.15.69:0/2532587842 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dc41ad780 con 0x7f7dc4102a10 2024-04-25T11:25:03.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.725+0000 7f7dabfff700 1 -- 172.21.15.69:0/2532587842 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7db4011cc0 con 0x7f7dc4102a10 2024-04-25T11:25:03.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.725+0000 7f7dabfff700 1 -- 172.21.15.69:0/2532587842 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db4010ba0 con 0x7f7dc4102a10 2024-04-25T11:25:03.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.725+0000 7f7dc9290700 1 -- 172.21.15.69:0/2532587842 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dc41adce0 con 0x7f7dc4102a10 2024-04-25T11:25:03.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.726+0000 7f7dc9290700 1 -- 172.21.15.69:0/2532587842 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dc4051190 con 0x7f7dc4102a10 2024-04-25T11:25:03.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.728+0000 7f7dabfff700 1 -- 172.21.15.69:0/2532587842 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f7db4018020 con 0x7f7dc4102a10 2024-04-25T11:25:03.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.729+0000 7f7dabfff700 1 --2- 172.21.15.69:0/2532587842 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f7dac0afc90 0x7f7dac0b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:03.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.729+0000 7f7dc27fc700 1 --2- 172.21.15.69:0/2532587842 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f7dac0afc90 0x7f7dac0b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:03.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.729+0000 7f7dabfff700 1 -- 172.21.15.69:0/2532587842 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f7db40decb0 con 0x7f7dc4102a10 2024-04-25T11:25:03.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.729+0000 7f7dc27fc700 1 --2- 172.21.15.69:0/2532587842 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f7dac0afc90 0x7f7dac0b21a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4105980 tx=0x7f7db8009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:03.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.733+0000 7f7dabfff700 1 -- 172.21.15.69:0/2532587842 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7db401f020 con 0x7f7dc4102a10 2024-04-25T11:25:03.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.938+0000 7f7dc9290700 1 -- 172.21.15.69:0/2532587842 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f7dc402e510 con 0x7f7dc4102a10 2024-04-25T11:25:03.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.938+0000 7f7dabfff700 1 -- 172.21.15.69:0/2532587842 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7db40a74e0 con 0x7f7dc4102a10 2024-04-25T11:25:03.932 INFO:teuthology.orchestra.run.smithi069.stdout:34359738454 2024-04-25T11:25:03.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.940+0000 7f7dc9290700 1 -- 172.21.15.69:0/2532587842 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f7dac0afc90 msgr2=0x7f7dac0b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:03.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.940+0000 7f7dc9290700 1 --2- 172.21.15.69:0/2532587842 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f7dac0afc90 0x7f7dac0b21a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4105980 tx=0x7f7db8009190 comp rx=0 tx=0).stop 2024-04-25T11:25:03.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.941+0000 7f7dc9290700 1 -- 172.21.15.69:0/2532587842 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7dc4102a10 msgr2=0x7f7dc41a1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:03.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.941+0000 7f7dc9290700 1 --2- 172.21.15.69:0/2532587842 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7dc4102a10 0x7f7dc41a1ff0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7db400b500 tx=0x7f7db400b810 comp rx=0 tx=0).stop 2024-04-25T11:25:03.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.941+0000 7f7dc9290700 1 -- 172.21.15.69:0/2532587842 shutdown_connections 2024-04-25T11:25:03.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.941+0000 7f7dc9290700 1 --2- 172.21.15.69:0/2532587842 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f7dac0afc90 0x7f7dac0b21a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:03.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.941+0000 7f7dc9290700 1 --2- 172.21.15.69:0/2532587842 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7dc4109780 0x7f7dc41ad070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:03.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.941+0000 7f7dc9290700 1 --2- 172.21.15.69:0/2532587842 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7dc4102a10 0x7f7dc41a1ff0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:03.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.941+0000 7f7dc9290700 1 --2- 172.21.15.69:0/2532587842 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7dc4108e30 0x7f7dc41a2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:03.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.941+0000 7f7dc9290700 1 -- 172.21.15.69:0/2532587842 >> 172.21.15.69:0/2532587842 conn(0x7f7dc40fe3f0 msgr2=0x7f7dc40fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:03.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.941+0000 7f7dc9290700 1 -- 172.21.15.69:0/2532587842 shutdown_connections 2024-04-25T11:25:03.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:03.941+0000 7f7dc9290700 1 -- 172.21.15.69:0/2532587842 wait complete. 2024-04-25T11:25:04.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.324+0000 7f2819297700 1 -- 172.21.15.69:0/3594595142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2814103390 msgr2=0x7f28141037a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:04.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.324+0000 7f2819297700 1 --2- 172.21.15.69:0/3594595142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2814103390 0x7f28141037a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f280c00b0d0 tx=0x7f280c00b3e0 comp rx=0 tx=0).stop 2024-04-25T11:25:04.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.324+0000 7f2819297700 1 -- 172.21.15.69:0/3594595142 shutdown_connections 2024-04-25T11:25:04.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.324+0000 7f2819297700 1 --2- 172.21.15.69:0/3594595142 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2814108610 0x7f281410ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:04.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.324+0000 7f2819297700 1 --2- 172.21.15.69:0/3594595142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2814103390 0x7f28141037a0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:04.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.324+0000 7f2819297700 1 --2- 172.21.15.69:0/3594595142 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2814107cc0 0x7f28141080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:04.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.324+0000 7f2819297700 1 -- 172.21.15.69:0/3594595142 >> 172.21.15.69:0/3594595142 conn(0x7f28140fec90 msgr2=0x7f28141010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:04.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.324+0000 7f2819297700 1 -- 172.21.15.69:0/3594595142 shutdown_connections 2024-04-25T11:25:04.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.324+0000 7f2819297700 1 -- 172.21.15.69:0/3594595142 wait complete. 2024-04-25T11:25:04.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.325+0000 7f2819297700 1 Processor -- start 2024-04-25T11:25:04.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.325+0000 7f2819297700 1 -- start start 2024-04-25T11:25:04.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.325+0000 7f2819297700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2814103390 0x7f28141a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:04.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.325+0000 7f2819297700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2814107cc0 0x7f28141a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:04.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.325+0000 7f2819297700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2814108610 0x7f281419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:04.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.325+0000 7f2819297700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2814114240 con 0x7f2814108610 2024-04-25T11:25:04.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.325+0000 7f2819297700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f28141140c0 con 0x7f2814107cc0 2024-04-25T11:25:04.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.325+0000 7f2819297700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f28141143c0 con 0x7f2814103390 2024-04-25T11:25:04.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.325+0000 7f28137fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2814108610 0x7f281419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:04.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.326+0000 7f28137fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2814108610 0x7f281419a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56122/0 (socket says 172.21.15.69:56122) 2024-04-25T11:25:04.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.326+0000 7f28137fe700 1 -- 172.21.15.69:0/353895102 learned_addr learned my addr 172.21.15.69:0/353895102 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:04.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.326+0000 7f28127fc700 1 --2- 172.21.15.69:0/353895102 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2814107cc0 0x7f28141a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:04.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.326+0000 7f2812ffd700 1 --2- 172.21.15.69:0/353895102 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2814103390 0x7f28141a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:04.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.326+0000 7f28137fe700 1 -- 172.21.15.69:0/353895102 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2814103390 msgr2=0x7f28141a07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:04.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.326+0000 7f28137fe700 1 --2- 172.21.15.69:0/353895102 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2814103390 0x7f28141a07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:04.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.326+0000 7f28137fe700 1 -- 172.21.15.69:0/353895102 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2814107cc0 msgr2=0x7f28141a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:04.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.326+0000 7f28137fe700 1 --2- 172.21.15.69:0/353895102 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2814107cc0 0x7f28141a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:04.637 INFO:teuthology.orchestra.run.smithi069.stdout:55834574928 2024-04-25T11:25:04.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.326+0000 7f28137fe700 1 -- 172.21.15.69:0/353895102 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f281419b000 con 0x7f2814108610 2024-04-25T11:25:04.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.326+0000 7f28127fc700 1 --2- 172.21.15.69:0/353895102 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2814107cc0 0x7f28141a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:25:04.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.326+0000 7f28137fe700 1 --2- 172.21.15.69:0/353895102 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2814108610 0x7f281419a9c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f280400c900 tx=0x7f280400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:04.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.326+0000 7f280bfff700 1 -- 172.21.15.69:0/353895102 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2804007b00 con 0x7f2814108610 2024-04-25T11:25:04.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.326+0000 7f2819297700 1 -- 172.21.15.69:0/353895102 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f281419b2f0 con 0x7f2814108610 2024-04-25T11:25:04.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.326+0000 7f280bfff700 1 -- 172.21.15.69:0/353895102 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2804007ca0 con 0x7f2814108610 2024-04-25T11:25:04.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.326+0000 7f280bfff700 1 -- 172.21.15.69:0/353895102 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2804018660 con 0x7f2814108610 2024-04-25T11:25:04.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.326+0000 7f2819297700 1 -- 172.21.15.69:0/353895102 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28141a7630 con 0x7f2814108610 2024-04-25T11:25:04.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.327+0000 7f2819297700 1 -- 172.21.15.69:0/353895102 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2814051190 con 0x7f2814108610 2024-04-25T11:25:04.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.329+0000 7f280bfff700 1 -- 172.21.15.69:0/353895102 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f280400f450 con 0x7f2814108610 2024-04-25T11:25:04.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.329+0000 7f280bfff700 1 --2- 172.21.15.69:0/353895102 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f28000afc10 0x7f28000b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:04.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.329+0000 7f2812ffd700 1 --2- 172.21.15.69:0/353895102 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f28000afc10 0x7f28000b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:04.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.329+0000 7f280bfff700 1 -- 172.21.15.69:0/353895102 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f28040df040 con 0x7f2814108610 2024-04-25T11:25:04.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.330+0000 7f2812ffd700 1 --2- 172.21.15.69:0/353895102 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f28000afc10 0x7f28000b2120 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f280c009900 tx=0x7f280c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:04.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.334+0000 7f280bfff700 1 -- 172.21.15.69:0/353895102 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2804014030 con 0x7f2814108610 2024-04-25T11:25:04.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.551+0000 7f2819297700 1 -- 172.21.15.69:0/353895102 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f281419bef0 con 0x7f2814108610 2024-04-25T11:25:04.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.551+0000 7f280bfff700 1 -- 172.21.15.69:0/353895102 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f28040a7760 con 0x7f2814108610 2024-04-25T11:25:04.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.554+0000 7f2819297700 1 -- 172.21.15.69:0/353895102 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f28000afc10 msgr2=0x7f28000b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:04.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.554+0000 7f2819297700 1 --2- 172.21.15.69:0/353895102 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f28000afc10 0x7f28000b2120 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f280c009900 tx=0x7f280c00b040 comp rx=0 tx=0).stop 2024-04-25T11:25:04.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.554+0000 7f2819297700 1 -- 172.21.15.69:0/353895102 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2814108610 msgr2=0x7f281419a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:04.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.554+0000 7f2819297700 1 --2- 172.21.15.69:0/353895102 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2814108610 0x7f281419a9c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f280400c900 tx=0x7f280400ccc0 comp rx=0 tx=0).stop 2024-04-25T11:25:04.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.554+0000 7f2819297700 1 -- 172.21.15.69:0/353895102 shutdown_connections 2024-04-25T11:25:04.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.554+0000 7f2819297700 1 --2- 172.21.15.69:0/353895102 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f28000afc10 0x7f28000b2120 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:04.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.554+0000 7f2819297700 1 --2- 172.21.15.69:0/353895102 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2814108610 0x7f281419a9c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:04.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.554+0000 7f2819297700 1 --2- 172.21.15.69:0/353895102 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2814103390 0x7f28141a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:04.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.554+0000 7f2819297700 1 --2- 172.21.15.69:0/353895102 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2814107cc0 0x7f28141a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:04.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.554+0000 7f2819297700 1 -- 172.21.15.69:0/353895102 >> 172.21.15.69:0/353895102 conn(0x7f28140fec90 msgr2=0x7f2814101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:04.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.554+0000 7f2819297700 1 -- 172.21.15.69:0/353895102 shutdown_connections 2024-04-25T11:25:04.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:24:59.554+0000 7f2819297700 1 -- 172.21.15.69:0/353895102 wait complete. 2024-04-25T11:25:05.192 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:04 smithi069 ceph-mon[29188]: pgmap v328: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:25:05.192 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:04 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2532587842' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T11:25:05.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:04 smithi152 ceph-mon[32701]: pgmap v328: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:25:05.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:04 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/2532587842' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T11:25:05.296 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574925 got 55834574928 for osd.1 2024-04-25T11:25:05.297 DEBUG:teuthology.parallel:result is None 2024-04-25T11:25:05.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:04 smithi146 ceph-mon[32821]: pgmap v328: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:25:05.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:04 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/2532587842' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T11:25:06.922 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822711 got 154618822715 for osd.5 2024-04-25T11:25:06.923 DEBUG:teuthology.parallel:result is None 2024-04-25T11:25:07.181 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:06 smithi069 ceph-mon[29188]: pgmap v329: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:25:07.182 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:06 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/1555998576' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T11:25:07.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:06 smithi152 ceph-mon[32701]: pgmap v329: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:25:07.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:06 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/1555998576' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T11:25:07.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:06 smithi146 ceph-mon[32821]: pgmap v329: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:25:07.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:06 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/1555998576' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T11:25:07.816 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:25:07.841 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738449 got 34359738454 for osd.0 2024-04-25T11:25:07.842 DEBUG:teuthology.parallel:result is None 2024-04-25T11:25:08.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:08.819+0000 7f94b7b37700 1 -- 172.21.15.69:0/3731351470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94b0103890 msgr2=0x7f94b0103ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:08.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:08.819+0000 7f94b7b37700 1 --2- 172.21.15.69:0/3731351470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94b0103890 0x7f94b0103ca0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f94a0009990 tx=0x7f94a0009ca0 comp rx=0 tx=0).stop 2024-04-25T11:25:08.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:08.819+0000 7f94b7b37700 1 -- 172.21.15.69:0/3731351470 shutdown_connections 2024-04-25T11:25:08.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:08.819+0000 7f94b7b37700 1 --2- 172.21.15.69:0/3731351470 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94b01041e0 0x7f94b010fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:08.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:08.819+0000 7f94b7b37700 1 --2- 172.21.15.69:0/3731351470 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94b0109180 0x7f94b0109590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:08.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:08.819+0000 7f94b7b37700 1 --2- 172.21.15.69:0/3731351470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94b0103890 0x7f94b0103ca0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:08.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:08.819+0000 7f94b7b37700 1 -- 172.21.15.69:0/3731351470 >> 172.21.15.69:0/3731351470 conn(0x7f94b00fe370 msgr2=0x7f94b01007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:08.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:08.819+0000 7f94b7b37700 1 -- 172.21.15.69:0/3731351470 shutdown_connections 2024-04-25T11:25:08.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:08.819+0000 7f94b7b37700 1 -- 172.21.15.69:0/3731351470 wait complete. 2024-04-25T11:25:08.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:08.820+0000 7f94b7b37700 1 Processor -- start 2024-04-25T11:25:08.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:08.820+0000 7f94b7b37700 1 -- start start 2024-04-25T11:25:08.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:08.820+0000 7f94b7b37700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94b0103890 0x7f94b019ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:08.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:08.820+0000 7f94b7b37700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94b01041e0 0x7f94b019bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:08.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:08.820+0000 7f94b7b37700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94b0109180 0x7f94b01a6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:08.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:08.820+0000 7f94b7b37700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f94b01130d0 con 0x7f94b0109180 2024-04-25T11:25:08.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:08.820+0000 7f94b7b37700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f94b0112f50 con 0x7f94b01041e0 2024-04-25T11:25:08.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:08.820+0000 7f94b7b37700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f94b0113250 con 0x7f94b0103890 2024-04-25T11:25:08.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:08.820+0000 7f94b60d4700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94b0109180 0x7f94b01a6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:08.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:08.820+0000 7f94b60d4700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94b0109180 0x7f94b01a6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49312/0 (socket says 172.21.15.69:49312) 2024-04-25T11:25:08.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:08.820+0000 7f94b60d4700 1 -- 172.21.15.69:0/2959621218 learned_addr learned my addr 172.21.15.69:0/2959621218 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:08.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:08.820+0000 7f94b58d3700 1 --2- 172.21.15.69:0/2959621218 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94b0103890 0x7f94b019ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:08.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:08.821+0000 7f94b50d2700 1 --2- 172.21.15.69:0/2959621218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94b01041e0 0x7f94b019bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:08.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:08.821+0000 7f94b60d4700 1 -- 172.21.15.69:0/2959621218 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94b0103890 msgr2=0x7f94b019ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:08.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:08.821+0000 7f94b60d4700 1 --2- 172.21.15.69:0/2959621218 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94b0103890 0x7f94b019ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:08.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:08.821+0000 7f94b60d4700 1 -- 172.21.15.69:0/2959621218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94b01041e0 msgr2=0x7f94b019bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:08.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:08.821+0000 7f94b60d4700 1 --2- 172.21.15.69:0/2959621218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94b01041e0 0x7f94b019bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:08.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:08.821+0000 7f94b60d4700 1 -- 172.21.15.69:0/2959621218 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94b019c860 con 0x7f94b0109180 2024-04-25T11:25:08.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:08.821+0000 7f94b58d3700 1 --2- 172.21.15.69:0/2959621218 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94b0103890 0x7f94b019ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:25:08.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:08.821+0000 7f94b60d4700 1 --2- 172.21.15.69:0/2959621218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94b0109180 0x7f94b01a6af0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f94a800ef30 tx=0x7f94a800c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:08.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:08.821+0000 7f94a6ffd700 1 -- 172.21.15.69:0/2959621218 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a80092e0 con 0x7f94b0109180 2024-04-25T11:25:08.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:08.821+0000 7f94b7b37700 1 -- 172.21.15.69:0/2959621218 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94b01a70f0 con 0x7f94b0109180 2024-04-25T11:25:08.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:08.821+0000 7f94a6ffd700 1 -- 172.21.15.69:0/2959621218 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f94a8004bb0 con 0x7f94b0109180 2024-04-25T11:25:08.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:08.821+0000 7f94a6ffd700 1 -- 172.21.15.69:0/2959621218 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a8004080 con 0x7f94b0109180 2024-04-25T11:25:08.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:08.821+0000 7f94b7b37700 1 -- 172.21.15.69:0/2959621218 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94b01a7600 con 0x7f94b0109180 2024-04-25T11:25:08.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:08.822+0000 7f94b7b37700 1 -- 172.21.15.69:0/2959621218 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94b01048a0 con 0x7f94b0109180 2024-04-25T11:25:08.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:08.823+0000 7f94a6ffd700 1 -- 172.21.15.69:0/2959621218 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f94a801b030 con 0x7f94b0109180 2024-04-25T11:25:08.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:08.824+0000 7f94a6ffd700 1 --2- 172.21.15.69:0/2959621218 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f949c0afc10 0x7f949c0b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:08.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:08.824+0000 7f94b58d3700 1 --2- 172.21.15.69:0/2959621218 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f949c0afc10 0x7f949c0b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:08.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:08.824+0000 7f94a6ffd700 1 -- 172.21.15.69:0/2959621218 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f94a80dece0 con 0x7f94b0109180 2024-04-25T11:25:08.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:08.825+0000 7f94b58d3700 1 --2- 172.21.15.69:0/2959621218 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f949c0afc10 0x7f949c0b2120 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f94ac009c90 tx=0x7f94ac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:08.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:08.829+0000 7f94a6ffd700 1 -- 172.21.15.69:0/2959621218 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f94a8022020 con 0x7f94b0109180 2024-04-25T11:25:09.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.014+0000 7fcdb0b80700 1 -- 172.21.15.69:0/2800893777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcdac103360 msgr2=0x7fcdac103770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:09.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.014+0000 7fcdb0b80700 1 --2- 172.21.15.69:0/2800893777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcdac103360 0x7fcdac103770 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fcda00099c0 tx=0x7fcda0009cd0 comp rx=0 tx=0).stop 2024-04-25T11:25:09.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.014+0000 7fcdb0b80700 1 -- 172.21.15.69:0/2800893777 shutdown_connections 2024-04-25T11:25:09.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.014+0000 7fcdb0b80700 1 --2- 172.21.15.69:0/2800893777 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcdac1085e0 0x7fcdac10ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:09.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.014+0000 7fcdb0b80700 1 --2- 172.21.15.69:0/2800893777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcdac103360 0x7fcdac103770 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:09.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.014+0000 7fcdb0b80700 1 --2- 172.21.15.69:0/2800893777 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcdac107c90 0x7fcdac1080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:09.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.014+0000 7fcdb0b80700 1 -- 172.21.15.69:0/2800893777 >> 172.21.15.69:0/2800893777 conn(0x7fcdac0fec90 msgr2=0x7fcdac1010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:09.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.014+0000 7fcdb0b80700 1 -- 172.21.15.69:0/2800893777 shutdown_connections 2024-04-25T11:25:09.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.014+0000 7fcdb0b80700 1 -- 172.21.15.69:0/2800893777 wait complete. 2024-04-25T11:25:09.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.015+0000 7fcdb0b80700 1 Processor -- start 2024-04-25T11:25:09.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.015+0000 7fcdb0b80700 1 -- start start 2024-04-25T11:25:09.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.016+0000 7fcdb0b80700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcdac103360 0x7fcdac19ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:09.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.016+0000 7fcdb0b80700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcdac107c90 0x7fcdac19d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:09.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.016+0000 7fcdb0b80700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcdac1085e0 0x7fcdac1a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:09.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.016+0000 7fcdb0b80700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcdac1130e0 con 0x7fcdac103360 2024-04-25T11:25:09.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.016+0000 7fcdb0b80700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fcdac112f60 con 0x7fcdac107c90 2024-04-25T11:25:09.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.016+0000 7fcdb0b80700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fcdac113260 con 0x7fcdac1085e0 2024-04-25T11:25:09.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.016+0000 7fcda9d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcdac107c90 0x7fcdac19d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:09.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.016+0000 7fcdaa59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcdac103360 0x7fcdac19ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:09.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.016+0000 7fcdaad9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcdac1085e0 0x7fcdac1a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:09.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.016+0000 7fcdaad9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcdac1085e0 0x7fcdac1a1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.69:38228/0 (socket says 172.21.15.69:38228) 2024-04-25T11:25:09.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.016+0000 7fcda9d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcdac107c90 0x7fcdac19d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.69:50312/0 (socket says 172.21.15.69:50312) 2024-04-25T11:25:09.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.016+0000 7fcdaad9d700 1 -- 172.21.15.69:0/1153912934 learned_addr learned my addr 172.21.15.69:0/1153912934 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:09.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.016+0000 7fcdaa59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcdac103360 0x7fcdac19ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49336/0 (socket says 172.21.15.69:49336) 2024-04-25T11:25:09.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.016+0000 7fcdaad9d700 1 -- 172.21.15.69:0/1153912934 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcdac107c90 msgr2=0x7fcdac19d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:09.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.016+0000 7fcdaad9d700 1 --2- 172.21.15.69:0/1153912934 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcdac107c90 0x7fcdac19d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:09.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.016+0000 7fcdaad9d700 1 -- 172.21.15.69:0/1153912934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcdac103360 msgr2=0x7fcdac19ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:09.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.016+0000 7fcdaad9d700 1 --2- 172.21.15.69:0/1153912934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcdac103360 0x7fcdac19ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:09.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.016+0000 7fcdaad9d700 1 -- 172.21.15.69:0/1153912934 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdac1a1c70 con 0x7fcdac1085e0 2024-04-25T11:25:09.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.016+0000 7fcdaa59c700 1 --2- 172.21.15.69:0/1153912934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcdac103360 0x7fcdac19ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:25:09.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.016+0000 7fcdaad9d700 1 --2- 172.21.15.69:0/1153912934 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcdac1085e0 0x7fcdac1a1680 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcda400ada0 tx=0x7fcda400c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:09.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.017+0000 7fcd9b7fe700 1 -- 172.21.15.69:0/1153912934 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda4017020 con 0x7fcdac1085e0 2024-04-25T11:25:09.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.017+0000 7fcd9b7fe700 1 -- 172.21.15.69:0/1153912934 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcda400cd60 con 0x7fcdac1085e0 2024-04-25T11:25:09.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.017+0000 7fcd9b7fe700 1 -- 172.21.15.69:0/1153912934 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda4005280 con 0x7fcdac1085e0 2024-04-25T11:25:09.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.017+0000 7fcdb0b80700 1 -- 172.21.15.69:0/1153912934 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdac1a1f60 con 0x7fcdac1085e0 2024-04-25T11:25:09.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.018+0000 7fcdb0b80700 1 -- 172.21.15.69:0/1153912934 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdac1a2390 con 0x7fcdac1085e0 2024-04-25T11:25:09.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.019+0000 7fcdb0b80700 1 -- 172.21.15.69:0/1153912934 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdac051190 con 0x7fcdac1085e0 2024-04-25T11:25:09.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.021+0000 7fcd9b7fe700 1 -- 172.21.15.69:0/1153912934 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fcda400c8d0 con 0x7fcdac1085e0 2024-04-25T11:25:09.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.022+0000 7fcd9b7fe700 1 --2- 172.21.15.69:0/1153912934 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fcd940afc10 0x7fcd940b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:09.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.022+0000 7fcdaa59c700 1 --2- 172.21.15.69:0/1153912934 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fcd940afc10 0x7fcd940b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:09.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.022+0000 7fcd9b7fe700 1 -- 172.21.15.69:0/1153912934 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7fcda40dee20 con 0x7fcdac1085e0 2024-04-25T11:25:09.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.022+0000 7fcdaa59c700 1 --2- 172.21.15.69:0/1153912934 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fcd940afc10 0x7fcd940b2120 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcda0000c00 tx=0x7fcda0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:09.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.025+0000 7fcd9b7fe700 1 -- 172.21.15.69:0/1153912934 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcda4021020 con 0x7fcdac1085e0 2024-04-25T11:25:09.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.042+0000 7f94b7b37700 1 -- 172.21.15.69:0/2959621218 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f94b002e510 con 0x7f94b0109180 2024-04-25T11:25:09.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:08 smithi069 ceph-mon[29188]: pgmap v330: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:25:09.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.042+0000 7f94a6ffd700 1 -- 172.21.15.69:0/2959621218 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f94a8010040 con 0x7f94b0109180 2024-04-25T11:25:09.036 INFO:teuthology.orchestra.run.smithi069.stdout:274877906979 2024-04-25T11:25:09.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.044+0000 7f94b7b37700 1 -- 172.21.15.69:0/2959621218 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f949c0afc10 msgr2=0x7f949c0b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:09.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.044+0000 7f94b7b37700 1 --2- 172.21.15.69:0/2959621218 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f949c0afc10 0x7f949c0b2120 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f94ac009c90 tx=0x7f94ac009260 comp rx=0 tx=0).stop 2024-04-25T11:25:09.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.045+0000 7f94b7b37700 1 -- 172.21.15.69:0/2959621218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94b0109180 msgr2=0x7f94b01a6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:09.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.045+0000 7f94b7b37700 1 --2- 172.21.15.69:0/2959621218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94b0109180 0x7f94b01a6af0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f94a800ef30 tx=0x7f94a800c450 comp rx=0 tx=0).stop 2024-04-25T11:25:09.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.045+0000 7f94b7b37700 1 -- 172.21.15.69:0/2959621218 shutdown_connections 2024-04-25T11:25:09.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.045+0000 7f94b7b37700 1 --2- 172.21.15.69:0/2959621218 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f949c0afc10 0x7f949c0b2120 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:09.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.045+0000 7f94b7b37700 1 --2- 172.21.15.69:0/2959621218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94b0109180 0x7f94b01a6af0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:09.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.045+0000 7f94b7b37700 1 --2- 172.21.15.69:0/2959621218 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94b0103890 0x7f94b019ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:09.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.045+0000 7f94b7b37700 1 --2- 172.21.15.69:0/2959621218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94b01041e0 0x7f94b019bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:09.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.045+0000 7f94b7b37700 1 -- 172.21.15.69:0/2959621218 >> 172.21.15.69:0/2959621218 conn(0x7f94b00fe370 msgr2=0x7f94b00ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:09.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.045+0000 7f94b7b37700 1 -- 172.21.15.69:0/2959621218 shutdown_connections 2024-04-25T11:25:09.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.045+0000 7f94b7b37700 1 -- 172.21.15.69:0/2959621218 wait complete. 2024-04-25T11:25:09.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.231+0000 7fcdb0b80700 1 -- 172.21.15.69:0/1153912934 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fcdac02e510 con 0x7fcdac1085e0 2024-04-25T11:25:09.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.231+0000 7fcd9b7fe700 1 -- 172.21.15.69:0/1153912934 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fcda40a75d0 con 0x7fcdac1085e0 2024-04-25T11:25:09.226 INFO:teuthology.orchestra.run.smithi069.stdout:180388626489 2024-04-25T11:25:09.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.233+0000 7fcdb0b80700 1 -- 172.21.15.69:0/1153912934 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fcd940afc10 msgr2=0x7fcd940b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:09.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.233+0000 7fcdb0b80700 1 --2- 172.21.15.69:0/1153912934 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fcd940afc10 0x7fcd940b2120 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcda0000c00 tx=0x7fcda0005fb0 comp rx=0 tx=0).stop 2024-04-25T11:25:09.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.233+0000 7fcdb0b80700 1 -- 172.21.15.69:0/1153912934 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcdac1085e0 msgr2=0x7fcdac1a1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:09.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.233+0000 7fcdb0b80700 1 --2- 172.21.15.69:0/1153912934 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcdac1085e0 0x7fcdac1a1680 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcda400ada0 tx=0x7fcda400c330 comp rx=0 tx=0).stop 2024-04-25T11:25:09.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.233+0000 7fcdb0b80700 1 -- 172.21.15.69:0/1153912934 shutdown_connections 2024-04-25T11:25:09.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.233+0000 7fcdb0b80700 1 --2- 172.21.15.69:0/1153912934 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fcd940afc10 0x7fcd940b2120 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:09.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.233+0000 7fcdb0b80700 1 --2- 172.21.15.69:0/1153912934 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcdac1085e0 0x7fcdac1a1680 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:09.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.233+0000 7fcdb0b80700 1 --2- 172.21.15.69:0/1153912934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcdac103360 0x7fcdac19ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:09.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.233+0000 7fcdb0b80700 1 --2- 172.21.15.69:0/1153912934 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcdac107c90 0x7fcdac19d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:09.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.234+0000 7fcdb0b80700 1 -- 172.21.15.69:0/1153912934 >> 172.21.15.69:0/1153912934 conn(0x7fcdac0fec90 msgr2=0x7fcdac109680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:09.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.234+0000 7fcdb0b80700 1 -- 172.21.15.69:0/1153912934 shutdown_connections 2024-04-25T11:25:09.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:09.234+0000 7fcdb0b80700 1 -- 172.21.15.69:0/1153912934 wait complete. 2024-04-25T11:25:09.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:08 smithi152 ceph-mon[32701]: pgmap v330: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:25:09.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:08 smithi146 ceph-mon[32821]: pgmap v330: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:25:10.079 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411399 got 77309411403 for osd.2 2024-04-25T11:25:10.079 DEBUG:teuthology.parallel:result is None 2024-04-25T11:25:10.154 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:09 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2959621218' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T11:25:10.154 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:09 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/1153912934' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T11:25:10.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:09 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/2959621218' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T11:25:10.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:09 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/1153912934' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T11:25:10.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.273+0000 7fc801900700 1 -- 172.21.15.69:0/4143175894 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7fc107c90 msgr2=0x7fc7fc1080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:10.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.273+0000 7fc801900700 1 --2- 172.21.15.69:0/4143175894 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7fc107c90 0x7fc7fc1080a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc7f00099c0 tx=0x7fc7f0009cd0 comp rx=0 tx=0).stop 2024-04-25T11:25:10.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.273+0000 7fc801900700 1 -- 172.21.15.69:0/4143175894 shutdown_connections 2024-04-25T11:25:10.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.273+0000 7fc801900700 1 --2- 172.21.15.69:0/4143175894 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7fc1085e0 0x7fc7fc10ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:10.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.273+0000 7fc801900700 1 --2- 172.21.15.69:0/4143175894 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc7fc103360 0x7fc7fc103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:10.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.273+0000 7fc801900700 1 --2- 172.21.15.69:0/4143175894 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7fc107c90 0x7fc7fc1080a0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:10.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.273+0000 7fc801900700 1 -- 172.21.15.69:0/4143175894 >> 172.21.15.69:0/4143175894 conn(0x7fc7fc0fec90 msgr2=0x7fc7fc1010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:10.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.273+0000 7fc801900700 1 -- 172.21.15.69:0/4143175894 shutdown_connections 2024-04-25T11:25:10.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.273+0000 7fc801900700 1 -- 172.21.15.69:0/4143175894 wait complete. 2024-04-25T11:25:10.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.274+0000 7fc801900700 1 Processor -- start 2024-04-25T11:25:10.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.274+0000 7fc801900700 1 -- start start 2024-04-25T11:25:10.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.274+0000 7fc801900700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7fc103360 0x7fc7fc19ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:10.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.274+0000 7fc801900700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc7fc107c90 0x7fc7fc19bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:10.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.274+0000 7fc801900700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7fc1085e0 0x7fc7fc19c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:10.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.274+0000 7fc801900700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc7fc1130a0 con 0x7fc7fc103360 2024-04-25T11:25:10.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.274+0000 7fc801900700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc7fc112f20 con 0x7fc7fc1085e0 2024-04-25T11:25:10.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.274+0000 7fc801900700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc7fc113220 con 0x7fc7fc107c90 2024-04-25T11:25:10.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.274+0000 7fc7faffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7fc103360 0x7fc7fc19ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:10.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.274+0000 7fc7faffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7fc103360 0x7fc7fc19ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49364/0 (socket says 172.21.15.69:49364) 2024-04-25T11:25:10.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.274+0000 7fc7faffd700 1 -- 172.21.15.69:0/2639903727 learned_addr learned my addr 172.21.15.69:0/2639903727 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:10.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.275+0000 7fc7fb7fe700 1 --2- 172.21.15.69:0/2639903727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7fc1085e0 0x7fc7fc19c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:10.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.275+0000 7fc7fa7fc700 1 --2- 172.21.15.69:0/2639903727 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc7fc107c90 0x7fc7fc19bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:10.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.275+0000 7fc7faffd700 1 -- 172.21.15.69:0/2639903727 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc7fc107c90 msgr2=0x7fc7fc19bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:10.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.275+0000 7fc7faffd700 1 --2- 172.21.15.69:0/2639903727 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc7fc107c90 0x7fc7fc19bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:10.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.275+0000 7fc7faffd700 1 -- 172.21.15.69:0/2639903727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7fc1085e0 msgr2=0x7fc7fc19c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:10.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.275+0000 7fc7faffd700 1 --2- 172.21.15.69:0/2639903727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7fc1085e0 0x7fc7fc19c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:10.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.275+0000 7fc7faffd700 1 -- 172.21.15.69:0/2639903727 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7fc1a6d90 con 0x7fc7fc103360 2024-04-25T11:25:10.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.275+0000 7fc7fa7fc700 1 --2- 172.21.15.69:0/2639903727 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc7fc107c90 0x7fc7fc19bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:25:10.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.275+0000 7fc7faffd700 1 --2- 172.21.15.69:0/2639903727 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7fc103360 0x7fc7fc19ba20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec00c920 tx=0x7fc7ec00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:10.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.275+0000 7fc8008fe700 1 -- 172.21.15.69:0/2639903727 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec007b00 con 0x7fc7fc103360 2024-04-25T11:25:10.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.275+0000 7fc8008fe700 1 -- 172.21.15.69:0/2639903727 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc7ec004cc0 con 0x7fc7fc103360 2024-04-25T11:25:10.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.275+0000 7fc8008fe700 1 -- 172.21.15.69:0/2639903727 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec005440 con 0x7fc7fc103360 2024-04-25T11:25:10.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.275+0000 7fc801900700 1 -- 172.21.15.69:0/2639903727 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7fc1a7080 con 0x7fc7fc103360 2024-04-25T11:25:10.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.275+0000 7fc801900700 1 -- 172.21.15.69:0/2639903727 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7fc1a75e0 con 0x7fc7fc103360 2024-04-25T11:25:10.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.276+0000 7fc801900700 1 -- 172.21.15.69:0/2639903727 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7fc051190 con 0x7fc7fc103360 2024-04-25T11:25:10.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.278+0000 7fc8008fe700 1 -- 172.21.15.69:0/2639903727 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fc7ec004750 con 0x7fc7fc103360 2024-04-25T11:25:10.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.278+0000 7fc8008fe700 1 --2- 172.21.15.69:0/2639903727 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc7e40afc90 0x7fc7e40b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:10.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.278+0000 7fc7fa7fc700 1 --2- 172.21.15.69:0/2639903727 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc7e40afc90 0x7fc7e40b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:10.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.278+0000 7fc8008fe700 1 -- 172.21.15.69:0/2639903727 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7fc7ec0dec10 con 0x7fc7fc103360 2024-04-25T11:25:10.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.279+0000 7fc7fa7fc700 1 --2- 172.21.15.69:0/2639903727 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc7e40afc90 0x7fc7e40b21a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0009990 tx=0x7fc7f0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:10.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.283+0000 7fc8008fe700 1 -- 172.21.15.69:0/2639903727 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc7ec014030 con 0x7fc7fc103360 2024-04-25T11:25:10.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:09 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/2959621218' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T11:25:10.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:09 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/1153912934' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T11:25:10.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:05.960+0000 7f0d42a0c700 1 -- 172.21.15.69:0/187857485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d3c109710 msgr2=0x7f0d3c111070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:10.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:05.960+0000 7f0d42a0c700 1 --2- 172.21.15.69:0/187857485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d3c109710 0x7f0d3c111070 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400b0d0 tx=0x7f0d3400b3e0 comp rx=0 tx=0).stop 2024-04-25T11:25:10.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:05.960+0000 7f0d42a0c700 1 -- 172.21.15.69:0/187857485 shutdown_connections 2024-04-25T11:25:10.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:05.960+0000 7f0d42a0c700 1 --2- 172.21.15.69:0/187857485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d3c109710 0x7f0d3c111070 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:10.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:05.960+0000 7f0d42a0c700 1 --2- 172.21.15.69:0/187857485 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0d3c104490 0x7f0d3c1048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:10.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:05.960+0000 7f0d42a0c700 1 --2- 172.21.15.69:0/187857485 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d3c108dc0 0x7f0d3c1091d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:10.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:05.960+0000 7f0d42a0c700 1 -- 172.21.15.69:0/187857485 >> 172.21.15.69:0/187857485 conn(0x7f0d3c0ffda0 msgr2=0x7f0d3c1021f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:10.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:05.960+0000 7f0d42a0c700 1 -- 172.21.15.69:0/187857485 shutdown_connections 2024-04-25T11:25:10.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:05.960+0000 7f0d42a0c700 1 -- 172.21.15.69:0/187857485 wait complete. 2024-04-25T11:25:10.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:05.961+0000 7f0d42a0c700 1 Processor -- start 2024-04-25T11:25:10.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:05.961+0000 7f0d42a0c700 1 -- start start 2024-04-25T11:25:10.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:05.961+0000 7f0d42a0c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0d3c104490 0x7f0d3c1a2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:10.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:05.961+0000 7f0d42a0c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d3c108dc0 0x7f0d3c1a2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:10.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:05.961+0000 7f0d42a0c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d3c109710 0x7f0d3c19cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:10.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:05.961+0000 7f0d42a0c700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c077f10 con 0x7f0d3c108dc0 2024-04-25T11:25:10.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:05.961+0000 7f0d42a0c700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c077d90 con 0x7f0d3c104490 2024-04-25T11:25:10.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:05.961+0000 7f0d42a0c700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c078090 con 0x7f0d3c109710 2024-04-25T11:25:10.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:05.961+0000 7f0d3b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d3c108dc0 0x7f0d3c1a2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:10.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:05.961+0000 7f0d3b7fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d3c108dc0 0x7f0d3c1a2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49298/0 (socket says 172.21.15.69:49298) 2024-04-25T11:25:10.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:05.961+0000 7f0d3b7fe700 1 -- 172.21.15.69:0/1555998576 learned_addr learned my addr 172.21.15.69:0/1555998576 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:10.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:05.961+0000 7f0d40fa9700 1 --2- 172.21.15.69:0/1555998576 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d3c109710 0x7f0d3c19cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:10.478 INFO:teuthology.orchestra.run.smithi069.stdout:227633266733 2024-04-25T11:25:10.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:05.962+0000 7f0d3bfff700 1 --2- 172.21.15.69:0/1555998576 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0d3c104490 0x7f0d3c1a2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:10.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:05.962+0000 7f0d3b7fe700 1 -- 172.21.15.69:0/1555998576 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d3c109710 msgr2=0x7f0d3c19cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:10.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:05.962+0000 7f0d3b7fe700 1 --2- 172.21.15.69:0/1555998576 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d3c109710 0x7f0d3c19cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:10.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:05.962+0000 7f0d3b7fe700 1 -- 172.21.15.69:0/1555998576 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0d3c104490 msgr2=0x7f0d3c1a2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:10.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:05.962+0000 7f0d3b7fe700 1 --2- 172.21.15.69:0/1555998576 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0d3c104490 0x7f0d3c1a2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:10.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:05.962+0000 7f0d3b7fe700 1 -- 172.21.15.69:0/1555998576 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d3c19d190 con 0x7f0d3c108dc0 2024-04-25T11:25:10.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:05.962+0000 7f0d3b7fe700 1 --2- 172.21.15.69:0/1555998576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d3c108dc0 0x7f0d3c1a2df0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0d3000c9e0 tx=0x7f0d3000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:10.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:05.962+0000 7f0d397fa700 1 -- 172.21.15.69:0/1555998576 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d30007aa0 con 0x7f0d3c108dc0 2024-04-25T11:25:10.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:05.962+0000 7f0d42a0c700 1 -- 172.21.15.69:0/1555998576 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d3c19d480 con 0x7f0d3c108dc0 2024-04-25T11:25:10.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:05.962+0000 7f0d397fa700 1 -- 172.21.15.69:0/1555998576 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0d30007c40 con 0x7f0d3c108dc0 2024-04-25T11:25:10.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:05.962+0000 7f0d397fa700 1 -- 172.21.15.69:0/1555998576 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d30018400 con 0x7f0d3c108dc0 2024-04-25T11:25:10.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:05.962+0000 7f0d42a0c700 1 -- 172.21.15.69:0/1555998576 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d3c1a9740 con 0x7f0d3c108dc0 2024-04-25T11:25:10.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:05.963+0000 7f0d42a0c700 1 -- 172.21.15.69:0/1555998576 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d3c051190 con 0x7f0d3c108dc0 2024-04-25T11:25:10.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:05.966+0000 7f0d397fa700 1 -- 172.21.15.69:0/1555998576 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f0d3000f450 con 0x7f0d3c108dc0 2024-04-25T11:25:10.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:05.966+0000 7f0d397fa700 1 --2- 172.21.15.69:0/1555998576 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f0d240afc10 0x7f0d240b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:10.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:05.966+0000 7f0d3bfff700 1 --2- 172.21.15.69:0/1555998576 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f0d240afc10 0x7f0d240b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:10.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:05.966+0000 7f0d397fa700 1 -- 172.21.15.69:0/1555998576 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f0d300df240 con 0x7f0d3c108dc0 2024-04-25T11:25:10.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:05.967+0000 7f0d3bfff700 1 --2- 172.21.15.69:0/1555998576 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f0d240afc10 0x7f0d240b2120 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c107ce0 tx=0x7f0d2c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:10.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:05.970+0000 7f0d397fa700 1 -- 172.21.15.69:0/1555998576 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0d30014030 con 0x7f0d3c108dc0 2024-04-25T11:25:10.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:06.176+0000 7f0d42a0c700 1 -- 172.21.15.69:0/1555998576 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f0d3c19dfc0 con 0x7f0d3c108dc0 2024-04-25T11:25:10.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:06.177+0000 7f0d397fa700 1 -- 172.21.15.69:0/1555998576 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0d300a79f0 con 0x7f0d3c108dc0 2024-04-25T11:25:10.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:06.179+0000 7f0d42a0c700 1 -- 172.21.15.69:0/1555998576 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f0d240afc10 msgr2=0x7f0d240b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:10.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:06.179+0000 7f0d42a0c700 1 --2- 172.21.15.69:0/1555998576 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f0d240afc10 0x7f0d240b2120 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c107ce0 tx=0x7f0d2c00a380 comp rx=0 tx=0).stop 2024-04-25T11:25:10.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:06.179+0000 7f0d42a0c700 1 -- 172.21.15.69:0/1555998576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d3c108dc0 msgr2=0x7f0d3c1a2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:10.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:06.179+0000 7f0d42a0c700 1 --2- 172.21.15.69:0/1555998576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d3c108dc0 0x7f0d3c1a2df0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0d3000c9e0 tx=0x7f0d3000ccf0 comp rx=0 tx=0).stop 2024-04-25T11:25:10.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:06.179+0000 7f0d42a0c700 1 -- 172.21.15.69:0/1555998576 shutdown_connections 2024-04-25T11:25:10.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:06.179+0000 7f0d42a0c700 1 --2- 172.21.15.69:0/1555998576 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f0d240afc10 0x7f0d240b2120 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:10.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:06.179+0000 7f0d42a0c700 1 --2- 172.21.15.69:0/1555998576 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d3c109710 0x7f0d3c19cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:10.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:06.179+0000 7f0d42a0c700 1 --2- 172.21.15.69:0/1555998576 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0d3c104490 0x7f0d3c1a2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:10.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:06.179+0000 7f0d42a0c700 1 --2- 172.21.15.69:0/1555998576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d3c108dc0 0x7f0d3c1a2df0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:10.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:06.179+0000 7f0d42a0c700 1 -- 172.21.15.69:0/1555998576 >> 172.21.15.69:0/1555998576 conn(0x7f0d3c0ffda0 msgr2=0x7f0d3c107100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:10.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:06.179+0000 7f0d42a0c700 1 -- 172.21.15.69:0/1555998576 shutdown_connections 2024-04-25T11:25:10.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:06.179+0000 7f0d42a0c700 1 -- 172.21.15.69:0/1555998576 wait complete. 2024-04-25T11:25:10.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.485+0000 7fc801900700 1 -- 172.21.15.69:0/2639903727 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fc7fc06baf0 con 0x7fc7fc103360 2024-04-25T11:25:10.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.486+0000 7fc8008fe700 1 -- 172.21.15.69:0/2639903727 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fc7ec0a7330 con 0x7fc7fc103360 2024-04-25T11:25:10.482 INFO:teuthology.orchestra.run.smithi069.stdout:300647710751 2024-04-25T11:25:10.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.488+0000 7fc801900700 1 -- 172.21.15.69:0/2639903727 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc7e40afc90 msgr2=0x7fc7e40b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:10.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.488+0000 7fc801900700 1 --2- 172.21.15.69:0/2639903727 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc7e40afc90 0x7fc7e40b21a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0009990 tx=0x7fc7f0005fd0 comp rx=0 tx=0).stop 2024-04-25T11:25:10.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.488+0000 7fc801900700 1 -- 172.21.15.69:0/2639903727 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7fc103360 msgr2=0x7fc7fc19ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:10.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.488+0000 7fc801900700 1 --2- 172.21.15.69:0/2639903727 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7fc103360 0x7fc7fc19ba20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec00c920 tx=0x7fc7ec00cc30 comp rx=0 tx=0).stop 2024-04-25T11:25:10.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.488+0000 7fc801900700 1 -- 172.21.15.69:0/2639903727 shutdown_connections 2024-04-25T11:25:10.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.488+0000 7fc801900700 1 --2- 172.21.15.69:0/2639903727 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc7e40afc90 0x7fc7e40b21a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:10.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.488+0000 7fc801900700 1 --2- 172.21.15.69:0/2639903727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7fc1085e0 0x7fc7fc19c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:10.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.488+0000 7fc801900700 1 --2- 172.21.15.69:0/2639903727 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7fc103360 0x7fc7fc19ba20 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:10.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.488+0000 7fc801900700 1 --2- 172.21.15.69:0/2639903727 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc7fc107c90 0x7fc7fc19bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:10.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.488+0000 7fc801900700 1 -- 172.21.15.69:0/2639903727 >> 172.21.15.69:0/2639903727 conn(0x7fc7fc0fec90 msgr2=0x7fc7fc1010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:10.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.488+0000 7fc801900700 1 -- 172.21.15.69:0/2639903727 shutdown_connections 2024-04-25T11:25:10.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:10.488+0000 7fc801900700 1 -- 172.21.15.69:0/2639903727 wait complete. 2024-04-25T11:25:11.156 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:10 smithi069 ceph-mon[29188]: pgmap v331: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:25:11.156 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:10 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2639903727' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T11:25:11.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:10 smithi152 ceph-mon[32701]: pgmap v331: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:25:11.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:10 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/2639903727' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T11:25:11.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:10 smithi146 ceph-mon[32821]: pgmap v331: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:25:11.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:10 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/2639903727' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T11:25:11.576 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266729 got 227633266733 for osd.8 2024-04-25T11:25:11.577 DEBUG:teuthology.parallel:result is None 2024-04-25T11:25:11.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:11.796+0000 7ffbd581e700 1 -- 172.21.15.69:0/1842005875 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbd01031c0 msgr2=0x7ffbd010f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:11.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:11.796+0000 7ffbd581e700 1 --2- 172.21.15.69:0/1842005875 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbd01031c0 0x7ffbd010f740 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ffbc800b0a0 tx=0x7ffbc800b3b0 comp rx=0 tx=0).stop 2024-04-25T11:25:11.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:11.796+0000 7ffbd581e700 1 -- 172.21.15.69:0/1842005875 shutdown_connections 2024-04-25T11:25:11.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:11.796+0000 7ffbd581e700 1 --2- 172.21.15.69:0/1842005875 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbd01031c0 0x7ffbd010f740 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:11.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:11.796+0000 7ffbd581e700 1 --2- 172.21.15.69:0/1842005875 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffbd01088a0 0x7ffbd0108cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:11.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:11.796+0000 7ffbd581e700 1 --2- 172.21.15.69:0/1842005875 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffbd0102870 0x7ffbd0102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:11.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:11.796+0000 7ffbd581e700 1 -- 172.21.15.69:0/1842005875 >> 172.21.15.69:0/1842005875 conn(0x7ffbd00fe370 msgr2=0x7ffbd01007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:11.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:11.796+0000 7ffbd581e700 1 -- 172.21.15.69:0/1842005875 shutdown_connections 2024-04-25T11:25:11.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:11.796+0000 7ffbd581e700 1 -- 172.21.15.69:0/1842005875 wait complete. 2024-04-25T11:25:11.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:11.797+0000 7ffbd581e700 1 Processor -- start 2024-04-25T11:25:11.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:11.797+0000 7ffbd581e700 1 -- start start 2024-04-25T11:25:11.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:11.797+0000 7ffbd581e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbd0102870 0x7ffbd01a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:11.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:11.797+0000 7ffbd581e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffbd01031c0 0x7ffbd01a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:11.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:11.797+0000 7ffbd581e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffbd01088a0 0x7ffbd019aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:11.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:11.797+0000 7ffbd581e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd0112890 con 0x7ffbd0102870 2024-04-25T11:25:11.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:11.797+0000 7ffbd581e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd0112710 con 0x7ffbd01088a0 2024-04-25T11:25:11.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:11.797+0000 7ffbd581e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd0112a10 con 0x7ffbd01031c0 2024-04-25T11:25:11.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:11.797+0000 7ffbceffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbd0102870 0x7ffbd01a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:11.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:11.797+0000 7ffbceffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbd0102870 0x7ffbd01a0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49386/0 (socket says 172.21.15.69:49386) 2024-04-25T11:25:11.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:11.797+0000 7ffbceffd700 1 -- 172.21.15.69:0/329227409 learned_addr learned my addr 172.21.15.69:0/329227409 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:11.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:11.797+0000 7ffbcf7fe700 1 --2- 172.21.15.69:0/329227409 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffbd01088a0 0x7ffbd019aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:11.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:11.798+0000 7ffbce7fc700 1 --2- 172.21.15.69:0/329227409 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffbd01031c0 0x7ffbd01a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:11.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:11.798+0000 7ffbceffd700 1 -- 172.21.15.69:0/329227409 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffbd01031c0 msgr2=0x7ffbd01a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:11.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:11.798+0000 7ffbceffd700 1 --2- 172.21.15.69:0/329227409 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffbd01031c0 0x7ffbd01a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:11.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:11.798+0000 7ffbceffd700 1 -- 172.21.15.69:0/329227409 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffbd01088a0 msgr2=0x7ffbd019aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:11.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:11.798+0000 7ffbceffd700 1 --2- 172.21.15.69:0/329227409 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffbd01088a0 0x7ffbd019aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:11.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:11.798+0000 7ffbceffd700 1 -- 172.21.15.69:0/329227409 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbd019b0e0 con 0x7ffbd0102870 2024-04-25T11:25:11.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:11.798+0000 7ffbce7fc700 1 --2- 172.21.15.69:0/329227409 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffbd01031c0 0x7ffbd01a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:25:11.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:11.798+0000 7ffbceffd700 1 --2- 172.21.15.69:0/329227409 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbd0102870 0x7ffbd01a0810 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ffbc400d950 tx=0x7ffbc400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:11.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:11.798+0000 7ffbd481c700 1 -- 172.21.15.69:0/329227409 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc4016070 con 0x7ffbd0102870 2024-04-25T11:25:11.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:11.798+0000 7ffbd481c700 1 -- 172.21.15.69:0/329227409 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffbc4004830 con 0x7ffbd0102870 2024-04-25T11:25:11.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:11.798+0000 7ffbd581e700 1 -- 172.21.15.69:0/329227409 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbd019b370 con 0x7ffbd0102870 2024-04-25T11:25:11.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:11.798+0000 7ffbd481c700 1 -- 172.21.15.69:0/329227409 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc40053e0 con 0x7ffbd0102870 2024-04-25T11:25:11.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:11.798+0000 7ffbd581e700 1 -- 172.21.15.69:0/329227409 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbd01a76a0 con 0x7ffbd0102870 2024-04-25T11:25:11.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:11.800+0000 7ffbd581e700 1 -- 172.21.15.69:0/329227409 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbd0051190 con 0x7ffbd0102870 2024-04-25T11:25:11.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:11.801+0000 7ffbd481c700 1 -- 172.21.15.69:0/329227409 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7ffbc4005580 con 0x7ffbd0102870 2024-04-25T11:25:11.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:11.801+0000 7ffbd481c700 1 --2- 172.21.15.69:0/329227409 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ffbb80afc90 0x7ffbb80b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:11.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:11.802+0000 7ffbce7fc700 1 --2- 172.21.15.69:0/329227409 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ffbb80afc90 0x7ffbb80b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:11.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:11.802+0000 7ffbd481c700 1 -- 172.21.15.69:0/329227409 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7ffbc40dee30 con 0x7ffbd0102870 2024-04-25T11:25:11.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:11.802+0000 7ffbce7fc700 1 --2- 172.21.15.69:0/329227409 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ffbb80afc90 0x7ffbb80b21a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ffbd01046b0 tx=0x7ffbc000b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:11.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:11.806+0000 7ffbd481c700 1 -- 172.21.15.69:0/329227409 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffbc401f020 con 0x7ffbd0102870 2024-04-25T11:25:12.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:12.012+0000 7ffbd581e700 1 -- 172.21.15.69:0/329227409 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7ffbd01a7870 con 0x7ffbd0102870 2024-04-25T11:25:12.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:12.013+0000 7ffbd481c700 1 -- 172.21.15.69:0/329227409 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ffbc40a74f0 con 0x7ffbd0102870 2024-04-25T11:25:12.006 INFO:teuthology.orchestra.run.smithi069.stdout:133143986244 2024-04-25T11:25:12.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:12.015+0000 7ffbd581e700 1 -- 172.21.15.69:0/329227409 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ffbb80afc90 msgr2=0x7ffbb80b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:12.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:12.015+0000 7ffbd581e700 1 --2- 172.21.15.69:0/329227409 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ffbb80afc90 0x7ffbb80b21a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ffbd01046b0 tx=0x7ffbc000b2c0 comp rx=0 tx=0).stop 2024-04-25T11:25:12.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:12.015+0000 7ffbd581e700 1 -- 172.21.15.69:0/329227409 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbd0102870 msgr2=0x7ffbd01a0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:12.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:12.015+0000 7ffbd581e700 1 --2- 172.21.15.69:0/329227409 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbd0102870 0x7ffbd01a0810 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ffbc400d950 tx=0x7ffbc400dc60 comp rx=0 tx=0).stop 2024-04-25T11:25:12.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:12.015+0000 7ffbd581e700 1 -- 172.21.15.69:0/329227409 shutdown_connections 2024-04-25T11:25:12.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:12.015+0000 7ffbd581e700 1 --2- 172.21.15.69:0/329227409 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ffbb80afc90 0x7ffbb80b21a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:12.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:12.015+0000 7ffbd581e700 1 --2- 172.21.15.69:0/329227409 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffbd01088a0 0x7ffbd019aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:12.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:12.015+0000 7ffbd581e700 1 --2- 172.21.15.69:0/329227409 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbd0102870 0x7ffbd01a0810 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:12.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:12.015+0000 7ffbd581e700 1 --2- 172.21.15.69:0/329227409 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffbd01031c0 0x7ffbd01a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:12.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:12.015+0000 7ffbd581e700 1 -- 172.21.15.69:0/329227409 >> 172.21.15.69:0/329227409 conn(0x7ffbd00fe370 msgr2=0x7ffbd0113050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:12.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:12.015+0000 7ffbd581e700 1 -- 172.21.15.69:0/329227409 shutdown_connections 2024-04-25T11:25:12.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:12.015+0000 7ffbd581e700 1 -- 172.21.15.69:0/329227409 wait complete. 2024-04-25T11:25:12.905 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710745 got 300647710751 for osd.11 2024-04-25T11:25:12.906 DEBUG:teuthology.parallel:result is None 2024-04-25T11:25:13.151 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:12 smithi146 ceph-mon[32821]: pgmap v332: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:25:13.151 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:12 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/329227409' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T11:25:13.192 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:12 smithi069 ceph-mon[29188]: pgmap v332: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:25:13.192 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:12 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/329227409' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T11:25:13.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:12 smithi152 ceph-mon[32701]: pgmap v332: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:25:13.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:12 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/329227409' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T11:25:14.133 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906975 got 274877906979 for osd.10 2024-04-25T11:25:14.133 DEBUG:teuthology.parallel:result is None 2024-04-25T11:25:14.977 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626483 got 180388626489 for osd.6 2024-04-25T11:25:14.977 DEBUG:teuthology.parallel:result is None 2024-04-25T11:25:15.192 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:14 smithi069 ceph-mon[29188]: pgmap v333: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:25:15.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:14 smithi152 ceph-mon[32701]: pgmap v333: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:25:15.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:14 smithi146 ceph-mon[32821]: pgmap v333: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:25:15.438 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:25:15.465 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986238 got 133143986244 for osd.4 2024-04-25T11:25:15.465 DEBUG:teuthology.parallel:result is None 2024-04-25T11:25:16.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.115+0000 7fbd94206700 1 -- 172.21.15.69:0/1981024477 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd8c1080a0 msgr2=0x7fbd8c1084b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:16.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.115+0000 7fbd94206700 1 --2- 172.21.15.69:0/1981024477 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd8c1080a0 0x7fbd8c1084b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbd88009990 tx=0x7fbd88009ca0 comp rx=0 tx=0).stop 2024-04-25T11:25:16.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.115+0000 7fbd94206700 1 -- 172.21.15.69:0/1981024477 shutdown_connections 2024-04-25T11:25:16.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.115+0000 7fbd94206700 1 --2- 172.21.15.69:0/1981024477 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd8c104150 0x7fbd8c10ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:16.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.115+0000 7fbd94206700 1 --2- 172.21.15.69:0/1981024477 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd8c1080a0 0x7fbd8c1084b0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:16.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.115+0000 7fbd94206700 1 --2- 172.21.15.69:0/1981024477 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd8c103800 0x7fbd8c103c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:16.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.115+0000 7fbd94206700 1 -- 172.21.15.69:0/1981024477 >> 172.21.15.69:0/1981024477 conn(0x7fbd8c0fe3f0 msgr2=0x7fbd8c100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:16.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.116+0000 7fbd94206700 1 -- 172.21.15.69:0/1981024477 shutdown_connections 2024-04-25T11:25:16.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.116+0000 7fbd94206700 1 -- 172.21.15.69:0/1981024477 wait complete. 2024-04-25T11:25:16.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.117+0000 7fbd94206700 1 Processor -- start 2024-04-25T11:25:16.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.117+0000 7fbd94206700 1 -- start start 2024-04-25T11:25:16.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.117+0000 7fbd94206700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd8c103800 0x7fbd8c1a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:16.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.117+0000 7fbd94206700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd8c104150 0x7fbd8c1a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:16.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.117+0000 7fbd94206700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd8c1080a0 0x7fbd8c19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:16.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.117+0000 7fbd94206700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8c1130c0 con 0x7fbd8c103800 2024-04-25T11:25:16.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.117+0000 7fbd94206700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8c112f40 con 0x7fbd8c104150 2024-04-25T11:25:16.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.117+0000 7fbd94206700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8c113240 con 0x7fbd8c1080a0 2024-04-25T11:25:16.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.117+0000 7fbd91fa2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd8c103800 0x7fbd8c1a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:16.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.117+0000 7fbd91fa2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd8c103800 0x7fbd8c1a07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45914/0 (socket says 172.21.15.69:45914) 2024-04-25T11:25:16.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.117+0000 7fbd91fa2700 1 -- 172.21.15.69:0/3524788780 learned_addr learned my addr 172.21.15.69:0/3524788780 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:16.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.117+0000 7fbd927a3700 1 --2- 172.21.15.69:0/3524788780 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd8c1080a0 0x7fbd8c19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:16.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.117+0000 7fbd917a1700 1 --2- 172.21.15.69:0/3524788780 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd8c104150 0x7fbd8c1a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:16.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.117+0000 7fbd91fa2700 1 -- 172.21.15.69:0/3524788780 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd8c1080a0 msgr2=0x7fbd8c19aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:16.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.117+0000 7fbd91fa2700 1 --2- 172.21.15.69:0/3524788780 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd8c1080a0 0x7fbd8c19aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:16.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.117+0000 7fbd91fa2700 1 -- 172.21.15.69:0/3524788780 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd8c104150 msgr2=0x7fbd8c1a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:16.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.117+0000 7fbd91fa2700 1 --2- 172.21.15.69:0/3524788780 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd8c104150 0x7fbd8c1a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:16.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.117+0000 7fbd91fa2700 1 -- 172.21.15.69:0/3524788780 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd8c19b050 con 0x7fbd8c103800 2024-04-25T11:25:16.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.117+0000 7fbd917a1700 1 --2- 172.21.15.69:0/3524788780 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd8c104150 0x7fbd8c1a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:25:16.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.118+0000 7fbd91fa2700 1 --2- 172.21.15.69:0/3524788780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd8c103800 0x7fbd8c1a07f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fbd88004790 tx=0x7fbd880047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:16.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.118+0000 7fbd82ffd700 1 -- 172.21.15.69:0/3524788780 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd88004830 con 0x7fbd8c103800 2024-04-25T11:25:16.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.118+0000 7fbd82ffd700 1 -- 172.21.15.69:0/3524788780 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbd88005020 con 0x7fbd8c103800 2024-04-25T11:25:16.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.118+0000 7fbd94206700 1 -- 172.21.15.69:0/3524788780 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd8c19b2e0 con 0x7fbd8c103800 2024-04-25T11:25:16.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.118+0000 7fbd82ffd700 1 -- 172.21.15.69:0/3524788780 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd88003e60 con 0x7fbd8c103800 2024-04-25T11:25:16.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.119+0000 7fbd94206700 1 -- 172.21.15.69:0/3524788780 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd8c19b7b0 con 0x7fbd8c103800 2024-04-25T11:25:16.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.119+0000 7fbd94206700 1 -- 172.21.15.69:0/3524788780 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd8c051190 con 0x7fbd8c103800 2024-04-25T11:25:16.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.121+0000 7fbd82ffd700 1 -- 172.21.15.69:0/3524788780 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fbd880051d0 con 0x7fbd8c103800 2024-04-25T11:25:16.114 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:15 smithi069 ceph-mon[29188]: pgmap v334: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:25:16.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.122+0000 7fbd82ffd700 1 --2- 172.21.15.69:0/3524788780 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fbd780afc90 0x7fbd780b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:16.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.122+0000 7fbd82ffd700 1 -- 172.21.15.69:0/3524788780 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7fbd880e0500 con 0x7fbd8c103800 2024-04-25T11:25:16.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.122+0000 7fbd917a1700 1 --2- 172.21.15.69:0/3524788780 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fbd780afc90 0x7fbd780b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:16.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.123+0000 7fbd917a1700 1 --2- 172.21.15.69:0/3524788780 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fbd780afc90 0x7fbd780b21a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c005fd0 tx=0x7fbd7c005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:16.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.126+0000 7fbd82ffd700 1 -- 172.21.15.69:0/3524788780 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbd88013070 con 0x7fbd8c103800 2024-04-25T11:25:16.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:15 smithi152 ceph-mon[32701]: pgmap v334: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:25:16.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.334+0000 7fbd94206700 1 -- 172.21.15.69:0/3524788780 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fbd8c02e510 con 0x7fbd8c103800 2024-04-25T11:25:16.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.334+0000 7fbd82ffd700 1 -- 172.21.15.69:0/3524788780 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbd880a8c50 con 0x7fbd8c103800 2024-04-25T11:25:16.327 INFO:teuthology.orchestra.run.smithi069.stdout:253403070507 2024-04-25T11:25:16.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.336+0000 7fbd94206700 1 -- 172.21.15.69:0/3524788780 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fbd780afc90 msgr2=0x7fbd780b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:16.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.336+0000 7fbd94206700 1 --2- 172.21.15.69:0/3524788780 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fbd780afc90 0x7fbd780b21a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c005fd0 tx=0x7fbd7c005e30 comp rx=0 tx=0).stop 2024-04-25T11:25:16.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.336+0000 7fbd94206700 1 -- 172.21.15.69:0/3524788780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd8c103800 msgr2=0x7fbd8c1a07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:16.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.336+0000 7fbd94206700 1 --2- 172.21.15.69:0/3524788780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd8c103800 0x7fbd8c1a07f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fbd88004790 tx=0x7fbd880047c0 comp rx=0 tx=0).stop 2024-04-25T11:25:16.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.336+0000 7fbd94206700 1 -- 172.21.15.69:0/3524788780 shutdown_connections 2024-04-25T11:25:16.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.336+0000 7fbd94206700 1 --2- 172.21.15.69:0/3524788780 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fbd780afc90 0x7fbd780b21a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:16.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.336+0000 7fbd94206700 1 --2- 172.21.15.69:0/3524788780 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd8c1080a0 0x7fbd8c19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:16.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.336+0000 7fbd94206700 1 --2- 172.21.15.69:0/3524788780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd8c103800 0x7fbd8c1a07f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:16.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.336+0000 7fbd94206700 1 --2- 172.21.15.69:0/3524788780 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbd8c104150 0x7fbd8c1a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:16.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.336+0000 7fbd94206700 1 -- 172.21.15.69:0/3524788780 >> 172.21.15.69:0/3524788780 conn(0x7fbd8c0fe3f0 msgr2=0x7fbd8c102a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:16.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.336+0000 7fbd94206700 1 -- 172.21.15.69:0/3524788780 shutdown_connections 2024-04-25T11:25:16.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:16.337+0000 7fbd94206700 1 -- 172.21.15.69:0/3524788780 wait complete. 2024-04-25T11:25:16.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:15 smithi146 ceph-mon[32821]: pgmap v334: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:25:17.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:16 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/3524788780' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T11:25:17.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:16 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/3524788780' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T11:25:17.404 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:16 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/3524788780' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T11:25:18.163 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:17 smithi069 ceph-mon[29188]: pgmap v335: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:25:18.192 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070500 got 253403070507 for osd.9 2024-04-25T11:25:18.192 DEBUG:teuthology.parallel:result is None 2024-04-25T11:25:18.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:17 smithi152 ceph-mon[32701]: pgmap v335: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:25:18.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:17 smithi146 ceph-mon[32821]: pgmap v335: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:25:18.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:18.928+0000 7f78aaaa1700 1 -- 172.21.15.69:0/3965504421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78a41088c0 msgr2=0x7f78a4108cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:18.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:18.928+0000 7f78aaaa1700 1 --2- 172.21.15.69:0/3965504421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78a41088c0 0x7f78a4108cd0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7898009990 tx=0x7f7898009ca0 comp rx=0 tx=0).stop 2024-04-25T11:25:18.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:18.928+0000 7f78aaaa1700 1 -- 172.21.15.69:0/3965504421 shutdown_connections 2024-04-25T11:25:18.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:18.928+0000 7f78aaaa1700 1 --2- 172.21.15.69:0/3965504421 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78a41031e0 0x7f78a410f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:18.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:18.928+0000 7f78aaaa1700 1 --2- 172.21.15.69:0/3965504421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78a41088c0 0x7f78a4108cd0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:18.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:18.928+0000 7f78aaaa1700 1 --2- 172.21.15.69:0/3965504421 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78a4102890 0x7f78a4102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:18.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:18.928+0000 7f78aaaa1700 1 -- 172.21.15.69:0/3965504421 >> 172.21.15.69:0/3965504421 conn(0x7f78a40fe3f0 msgr2=0x7f78a4100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:18.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:18.928+0000 7f78aaaa1700 1 -- 172.21.15.69:0/3965504421 shutdown_connections 2024-04-25T11:25:18.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:18.928+0000 7f78aaaa1700 1 -- 172.21.15.69:0/3965504421 wait complete. 2024-04-25T11:25:18.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:18.929+0000 7f78aaaa1700 1 Processor -- start 2024-04-25T11:25:18.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:18.929+0000 7f78aaaa1700 1 -- start start 2024-04-25T11:25:18.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:18.929+0000 7f78aaaa1700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78a4102890 0x7f78a41a0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:18.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:18.929+0000 7f78aaaa1700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78a41031e0 0x7f78a41a0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:18.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:18.929+0000 7f78aaaa1700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78a41088c0 0x7f78a419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:18.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:18.929+0000 7f78aaaa1700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f78a4112870 con 0x7f78a41088c0 2024-04-25T11:25:18.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:18.929+0000 7f78aaaa1700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f78a41126f0 con 0x7f78a4102890 2024-04-25T11:25:18.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:18.929+0000 7f78aaaa1700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f78a41129f0 con 0x7f78a41031e0 2024-04-25T11:25:18.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:18.929+0000 7f78a903e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78a41088c0 0x7f78a419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:18.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:18.929+0000 7f78a903e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78a41088c0 0x7f78a419aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45938/0 (socket says 172.21.15.69:45938) 2024-04-25T11:25:18.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:18.929+0000 7f78a903e700 1 -- 172.21.15.69:0/3544832174 learned_addr learned my addr 172.21.15.69:0/3544832174 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:18.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:18.929+0000 7f78a3fff700 1 --2- 172.21.15.69:0/3544832174 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78a41031e0 0x7f78a41a0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:18.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:18.929+0000 7f78a883d700 1 --2- 172.21.15.69:0/3544832174 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78a4102890 0x7f78a41a0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:18.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:18.929+0000 7f78a903e700 1 -- 172.21.15.69:0/3544832174 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78a41031e0 msgr2=0x7f78a41a0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:18.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:18.929+0000 7f78a903e700 1 --2- 172.21.15.69:0/3544832174 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78a41031e0 0x7f78a41a0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:18.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:18.930+0000 7f78a903e700 1 -- 172.21.15.69:0/3544832174 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78a4102890 msgr2=0x7f78a41a0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:18.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:18.930+0000 7f78a903e700 1 --2- 172.21.15.69:0/3544832174 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78a4102890 0x7f78a41a0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:18.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:18.930+0000 7f78a903e700 1 -- 172.21.15.69:0/3544832174 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78a419b0d0 con 0x7f78a41088c0 2024-04-25T11:25:18.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:18.930+0000 7f78a883d700 1 --2- 172.21.15.69:0/3544832174 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78a4102890 0x7f78a41a0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:25:18.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:18.930+0000 7f78a903e700 1 --2- 172.21.15.69:0/3544832174 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78a41088c0 0x7f78a419aa90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f789c007d70 tx=0x7f789c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:18.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:18.930+0000 7f78a1ffb700 1 -- 172.21.15.69:0/3544832174 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789c016020 con 0x7f78a41088c0 2024-04-25T11:25:18.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:18.930+0000 7f78a1ffb700 1 -- 172.21.15.69:0/3544832174 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f789c00ad40 con 0x7f78a41088c0 2024-04-25T11:25:18.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:18.930+0000 7f78a1ffb700 1 -- 172.21.15.69:0/3544832174 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789c0115e0 con 0x7f78a41088c0 2024-04-25T11:25:18.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:18.930+0000 7f78aaaa1700 1 -- 172.21.15.69:0/3544832174 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78a419b3c0 con 0x7f78a41088c0 2024-04-25T11:25:18.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:18.930+0000 7f78aaaa1700 1 -- 172.21.15.69:0/3544832174 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78a419b8b0 con 0x7f78a41088c0 2024-04-25T11:25:18.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:18.931+0000 7f78aaaa1700 1 -- 172.21.15.69:0/3544832174 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78a4051190 con 0x7f78a41088c0 2024-04-25T11:25:18.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:18.933+0000 7f78a1ffb700 1 -- 172.21.15.69:0/3544832174 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f789c019020 con 0x7f78a41088c0 2024-04-25T11:25:18.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:18.933+0000 7f78a1ffb700 1 --2- 172.21.15.69:0/3544832174 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f78940afc10 0x7f78940b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:18.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:18.934+0000 7f78a883d700 1 --2- 172.21.15.69:0/3544832174 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f78940afc10 0x7f78940b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:18.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:18.934+0000 7f78a1ffb700 1 -- 172.21.15.69:0/3544832174 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f789c0df410 con 0x7f78a41088c0 2024-04-25T11:25:18.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:18.934+0000 7f78a883d700 1 --2- 172.21.15.69:0/3544832174 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f78940afc10 0x7f78940b2120 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7898000c00 tx=0x7f7898005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:18.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:18.937+0000 7f78a1ffb700 1 -- 172.21.15.69:0/3544832174 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f789c020020 con 0x7f78a41088c0 2024-04-25T11:25:19.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.139+0000 7f78aaaa1700 1 -- 172.21.15.69:0/3544832174 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f78a419bfc0 con 0x7f78a41088c0 2024-04-25T11:25:19.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.139+0000 7f78a1ffb700 1 -- 172.21.15.69:0/3544832174 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f789c0a7c40 con 0x7f78a41088c0 2024-04-25T11:25:19.132 INFO:teuthology.orchestra.run.smithi069.stdout:201863462964 2024-04-25T11:25:19.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.141+0000 7f78aaaa1700 1 -- 172.21.15.69:0/3544832174 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f78940afc10 msgr2=0x7f78940b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:19.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.141+0000 7f78aaaa1700 1 --2- 172.21.15.69:0/3544832174 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f78940afc10 0x7f78940b2120 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7898000c00 tx=0x7f7898005fd0 comp rx=0 tx=0).stop 2024-04-25T11:25:19.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.141+0000 7f78aaaa1700 1 -- 172.21.15.69:0/3544832174 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78a41088c0 msgr2=0x7f78a419aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:19.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.141+0000 7f78aaaa1700 1 --2- 172.21.15.69:0/3544832174 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78a41088c0 0x7f78a419aa90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f789c007d70 tx=0x7f789c00a340 comp rx=0 tx=0).stop 2024-04-25T11:25:19.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.141+0000 7f78aaaa1700 1 -- 172.21.15.69:0/3544832174 shutdown_connections 2024-04-25T11:25:19.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.141+0000 7f78aaaa1700 1 --2- 172.21.15.69:0/3544832174 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f78940afc10 0x7f78940b2120 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:19.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.141+0000 7f78aaaa1700 1 --2- 172.21.15.69:0/3544832174 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78a41088c0 0x7f78a419aa90 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:19.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.141+0000 7f78aaaa1700 1 --2- 172.21.15.69:0/3544832174 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78a4102890 0x7f78a41a0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:19.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.141+0000 7f78aaaa1700 1 --2- 172.21.15.69:0/3544832174 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78a41031e0 0x7f78a41a0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:19.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.141+0000 7f78aaaa1700 1 -- 172.21.15.69:0/3544832174 >> 172.21.15.69:0/3544832174 conn(0x7f78a40fe3f0 msgr2=0x7f78a41065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:19.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.141+0000 7f78aaaa1700 1 -- 172.21.15.69:0/3544832174 shutdown_connections 2024-04-25T11:25:19.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.141+0000 7f78aaaa1700 1 -- 172.21.15.69:0/3544832174 wait complete. 2024-04-25T11:25:19.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.175+0000 7fabff4c0700 1 -- 172.21.15.69:0/1714623727 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabf8103360 msgr2=0x7fabf8103770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:19.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.175+0000 7fabff4c0700 1 --2- 172.21.15.69:0/1714623727 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabf8103360 0x7fabf8103770 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fabf40099c0 tx=0x7fabf4009cd0 comp rx=0 tx=0).stop 2024-04-25T11:25:19.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.175+0000 7fabff4c0700 1 -- 172.21.15.69:0/1714623727 shutdown_connections 2024-04-25T11:25:19.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.175+0000 7fabff4c0700 1 --2- 172.21.15.69:0/1714623727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabf81085e0 0x7fabf810ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:19.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.175+0000 7fabff4c0700 1 --2- 172.21.15.69:0/1714623727 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabf8103360 0x7fabf8103770 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:19.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.175+0000 7fabff4c0700 1 --2- 172.21.15.69:0/1714623727 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fabf8107c90 0x7fabf81080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:19.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.175+0000 7fabff4c0700 1 -- 172.21.15.69:0/1714623727 >> 172.21.15.69:0/1714623727 conn(0x7fabf80fec90 msgr2=0x7fabf81010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:19.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.175+0000 7fabff4c0700 1 -- 172.21.15.69:0/1714623727 shutdown_connections 2024-04-25T11:25:19.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.175+0000 7fabff4c0700 1 -- 172.21.15.69:0/1714623727 wait complete. 2024-04-25T11:25:19.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.176+0000 7fabff4c0700 1 Processor -- start 2024-04-25T11:25:19.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.176+0000 7fabff4c0700 1 -- start start 2024-04-25T11:25:19.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.176+0000 7fabff4c0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabf8107c90 0x7fabf819bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:19.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.176+0000 7fabff4c0700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabf81085e0 0x7fabf819c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:19.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.176+0000 7fabff4c0700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fabf819c670 0x7fabf81a6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:19.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.176+0000 7fabff4c0700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fabf8113110 con 0x7fabf8107c90 2024-04-25T11:25:19.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.176+0000 7fabff4c0700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fabf8112f90 con 0x7fabf81085e0 2024-04-25T11:25:19.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.176+0000 7fabff4c0700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fabf8113290 con 0x7fabf819c670 2024-04-25T11:25:19.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.176+0000 7fabfd25c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabf8107c90 0x7fabf819bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:19.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.176+0000 7fabfd25c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabf8107c90 0x7fabf819bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45956/0 (socket says 172.21.15.69:45956) 2024-04-25T11:25:19.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.176+0000 7fabfd25c700 1 -- 172.21.15.69:0/749480097 learned_addr learned my addr 172.21.15.69:0/749480097 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:19.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.177+0000 7fabfda5d700 1 --2- 172.21.15.69:0/749480097 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fabf819c670 0x7fabf81a6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:19.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.177+0000 7fabfca5b700 1 --2- 172.21.15.69:0/749480097 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabf81085e0 0x7fabf819c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:19.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.177+0000 7fabfd25c700 1 -- 172.21.15.69:0/749480097 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fabf819c670 msgr2=0x7fabf81a6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:19.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.177+0000 7fabfd25c700 1 --2- 172.21.15.69:0/749480097 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fabf819c670 0x7fabf81a6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:19.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.177+0000 7fabfd25c700 1 -- 172.21.15.69:0/749480097 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabf81085e0 msgr2=0x7fabf819c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:19.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.177+0000 7fabfd25c700 1 --2- 172.21.15.69:0/749480097 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabf81085e0 0x7fabf819c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:19.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.177+0000 7fabfd25c700 1 -- 172.21.15.69:0/749480097 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabf81a7400 con 0x7fabf8107c90 2024-04-25T11:25:19.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.177+0000 7fabfd25c700 1 --2- 172.21.15.69:0/749480097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabf8107c90 0x7fabf819bbf0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fabf40098e0 tx=0x7fabf40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:19.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.177+0000 7fabee7fc700 1 -- 172.21.15.69:0/749480097 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf4009e60 con 0x7fabf8107c90 2024-04-25T11:25:19.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.177+0000 7fabff4c0700 1 -- 172.21.15.69:0/749480097 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabf81a7690 con 0x7fabf8107c90 2024-04-25T11:25:19.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.177+0000 7fabee7fc700 1 -- 172.21.15.69:0/749480097 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fabf4005030 con 0x7fabf8107c90 2024-04-25T11:25:19.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.177+0000 7fabee7fc700 1 -- 172.21.15.69:0/749480097 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf4003e60 con 0x7fabf8107c90 2024-04-25T11:25:19.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.177+0000 7fabff4c0700 1 -- 172.21.15.69:0/749480097 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabf81a7ae0 con 0x7fabf8107c90 2024-04-25T11:25:19.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.178+0000 7fabff4c0700 1 -- 172.21.15.69:0/749480097 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabf8051190 con 0x7fabf8107c90 2024-04-25T11:25:19.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.180+0000 7fabee7fc700 1 -- 172.21.15.69:0/749480097 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fabf40039d0 con 0x7fabf8107c90 2024-04-25T11:25:19.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.180+0000 7fabee7fc700 1 --2- 172.21.15.69:0/749480097 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fabe40afc10 0x7fabe40b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:19.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.180+0000 7fabee7fc700 1 -- 172.21.15.69:0/749480097 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7fabf40dfc00 con 0x7fabf8107c90 2024-04-25T11:25:19.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.180+0000 7fabfca5b700 1 --2- 172.21.15.69:0/749480097 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fabe40afc10 0x7fabe40b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:19.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.181+0000 7fabfca5b700 1 --2- 172.21.15.69:0/749480097 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fabe40afc10 0x7fabe40b2120 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fabf8106c20 tx=0x7fabe8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:19.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.185+0000 7fabee7fc700 1 -- 172.21.15.69:0/749480097 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fabf4013070 con 0x7fabf8107c90 2024-04-25T11:25:19.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.385+0000 7fabff4c0700 1 -- 172.21.15.69:0/749480097 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fabf802e6d0 con 0x7fabf8107c90 2024-04-25T11:25:19.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.385+0000 7fabee7fc700 1 -- 172.21.15.69:0/749480097 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fabf40a83b0 con 0x7fabf8107c90 2024-04-25T11:25:19.378 INFO:teuthology.orchestra.run.smithi069.stdout:107374182473 2024-04-25T11:25:19.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.387+0000 7fabff4c0700 1 -- 172.21.15.69:0/749480097 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fabe40afc10 msgr2=0x7fabe40b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:19.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.387+0000 7fabff4c0700 1 --2- 172.21.15.69:0/749480097 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fabe40afc10 0x7fabe40b2120 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fabf8106c20 tx=0x7fabe8009260 comp rx=0 tx=0).stop 2024-04-25T11:25:19.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.387+0000 7fabff4c0700 1 -- 172.21.15.69:0/749480097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabf8107c90 msgr2=0x7fabf819bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:19.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.387+0000 7fabff4c0700 1 --2- 172.21.15.69:0/749480097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabf8107c90 0x7fabf819bbf0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fabf40098e0 tx=0x7fabf40045e0 comp rx=0 tx=0).stop 2024-04-25T11:25:19.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.387+0000 7fabff4c0700 1 -- 172.21.15.69:0/749480097 shutdown_connections 2024-04-25T11:25:19.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.387+0000 7fabff4c0700 1 --2- 172.21.15.69:0/749480097 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fabe40afc10 0x7fabe40b2120 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:19.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.387+0000 7fabff4c0700 1 --2- 172.21.15.69:0/749480097 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fabf819c670 0x7fabf81a6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:19.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.387+0000 7fabff4c0700 1 --2- 172.21.15.69:0/749480097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fabf8107c90 0x7fabf819bbf0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:19.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.387+0000 7fabff4c0700 1 --2- 172.21.15.69:0/749480097 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fabf81085e0 0x7fabf819c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:19.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.387+0000 7fabff4c0700 1 -- 172.21.15.69:0/749480097 >> 172.21.15.69:0/749480097 conn(0x7fabf80fec90 msgr2=0x7fabf81010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:19.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.387+0000 7fabff4c0700 1 -- 172.21.15.69:0/749480097 shutdown_connections 2024-04-25T11:25:19.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:19.387+0000 7fabff4c0700 1 -- 172.21.15.69:0/749480097 wait complete. 2024-04-25T11:25:19.442 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:19 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/3544832174' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T11:25:19.480 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:19 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/3544832174' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T11:25:19.598 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:19 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/3544832174' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T11:25:20.061 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182469 got 107374182473 for osd.3 2024-04-25T11:25:20.062 DEBUG:teuthology.parallel:result is None 2024-04-25T11:25:20.442 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:20 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/749480097' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T11:25:20.443 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:20 smithi069 ceph-mon[29188]: pgmap v336: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:25:20.479 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:20 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/749480097' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T11:25:20.480 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:20 smithi152 ceph-mon[32701]: pgmap v336: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:25:20.588 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462958 got 201863462964 for osd.7 2024-04-25T11:25:20.588 DEBUG:teuthology.parallel:result is None 2024-04-25T11:25:20.588 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T11:25:20.589 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T11:25:20.598 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:20 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/749480097' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T11:25:20.598 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:20 smithi146 ceph-mon[32821]: pgmap v336: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:25:20.837 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:25:22.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:21 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:25:22.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:21 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:25:22.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:21 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:25:22.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:21 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:25:22.008 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:21 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:25:22.098 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:21 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:25:22.098 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:21 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:25:22.098 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:21 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:25:22.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:21 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:25:22.099 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:21 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:25:22.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:21 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:25:22.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:21 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:25:22.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:21 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:25:22.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:21 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:25:22.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:21 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:25:22.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.561+0000 7fb3eae95700 1 -- 172.21.15.69:0/1881424373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb3e4109730 msgr2=0x7fb3e4111090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:22.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.561+0000 7fb3eae95700 1 --2- 172.21.15.69:0/1881424373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb3e4109730 0x7fb3e4111090 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc00b0d0 tx=0x7fb3dc00b3e0 comp rx=0 tx=0).stop 2024-04-25T11:25:22.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.561+0000 7fb3eae95700 1 -- 172.21.15.69:0/1881424373 shutdown_connections 2024-04-25T11:25:22.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.561+0000 7fb3eae95700 1 --2- 172.21.15.69:0/1881424373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb3e4109730 0x7fb3e4111090 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:22.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.561+0000 7fb3eae95700 1 --2- 172.21.15.69:0/1881424373 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3e41044b0 0x7fb3e41048c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:22.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.561+0000 7fb3eae95700 1 --2- 172.21.15.69:0/1881424373 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3e4108de0 0x7fb3e41091f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:22.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.561+0000 7fb3eae95700 1 -- 172.21.15.69:0/1881424373 >> 172.21.15.69:0/1881424373 conn(0x7fb3e40ffda0 msgr2=0x7fb3e41021f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:22.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.561+0000 7fb3eae95700 1 -- 172.21.15.69:0/1881424373 shutdown_connections 2024-04-25T11:25:22.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.561+0000 7fb3eae95700 1 -- 172.21.15.69:0/1881424373 wait complete. 2024-04-25T11:25:22.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.562+0000 7fb3eae95700 1 Processor -- start 2024-04-25T11:25:22.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.562+0000 7fb3eae95700 1 -- start start 2024-04-25T11:25:22.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.562+0000 7fb3eae95700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3e41044b0 0x7fb3e41a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:22.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.562+0000 7fb3eae95700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb3e4108de0 0x7fb3e41a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:22.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.562+0000 7fb3eae95700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3e4109730 0x7fb3e419a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:22.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.562+0000 7fb3eae95700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e4114210 con 0x7fb3e4108de0 2024-04-25T11:25:22.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.562+0000 7fb3eae95700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e4114090 con 0x7fb3e41044b0 2024-04-25T11:25:22.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.562+0000 7fb3eae95700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e4114390 con 0x7fb3e4109730 2024-04-25T11:25:22.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.562+0000 7fb3e3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb3e4108de0 0x7fb3e41a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:22.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.562+0000 7fb3e3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb3e4108de0 0x7fb3e41a0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34290/0 (socket says 172.21.15.69:34290) 2024-04-25T11:25:22.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.562+0000 7fb3e3fff700 1 -- 172.21.15.69:0/591546379 learned_addr learned my addr 172.21.15.69:0/591546379 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:22.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.562+0000 7fb3e8c31700 1 --2- 172.21.15.69:0/591546379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3e41044b0 0x7fb3e41a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:22.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.562+0000 7fb3e9432700 1 --2- 172.21.15.69:0/591546379 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3e4109730 0x7fb3e419a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:22.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.562+0000 7fb3e3fff700 1 -- 172.21.15.69:0/591546379 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3e4109730 msgr2=0x7fb3e419a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:22.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.562+0000 7fb3e3fff700 1 --2- 172.21.15.69:0/591546379 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3e4109730 0x7fb3e419a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:22.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.562+0000 7fb3e3fff700 1 -- 172.21.15.69:0/591546379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3e41044b0 msgr2=0x7fb3e41a07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:22.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.562+0000 7fb3e3fff700 1 --2- 172.21.15.69:0/591546379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3e41044b0 0x7fb3e41a07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:22.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.562+0000 7fb3e3fff700 1 -- 172.21.15.69:0/591546379 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3e419b010 con 0x7fb3e4108de0 2024-04-25T11:25:22.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.562+0000 7fb3e9432700 1 --2- 172.21.15.69:0/591546379 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3e4109730 0x7fb3e419a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:25:22.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.563+0000 7fb3e3fff700 1 --2- 172.21.15.69:0/591546379 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb3e4108de0 0x7fb3e41a0cf0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb3d000d980 tx=0x7fb3d000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:22.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.563+0000 7fb3e1ffb700 1 -- 172.21.15.69:0/591546379 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d0017070 con 0x7fb3e4108de0 2024-04-25T11:25:22.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.563+0000 7fb3e1ffb700 1 -- 172.21.15.69:0/591546379 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb3d0009dc0 con 0x7fb3e4108de0 2024-04-25T11:25:22.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.563+0000 7fb3eae95700 1 -- 172.21.15.69:0/591546379 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3e419b300 con 0x7fb3e4108de0 2024-04-25T11:25:22.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.563+0000 7fb3e1ffb700 1 -- 172.21.15.69:0/591546379 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d000f3c0 con 0x7fb3e4108de0 2024-04-25T11:25:22.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.563+0000 7fb3eae95700 1 -- 172.21.15.69:0/591546379 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3e41a7640 con 0x7fb3e4108de0 2024-04-25T11:25:22.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.564+0000 7fb3eae95700 1 -- 172.21.15.69:0/591546379 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3e4051190 con 0x7fb3e4108de0 2024-04-25T11:25:22.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.566+0000 7fb3e1ffb700 1 -- 172.21.15.69:0/591546379 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fb3d0009930 con 0x7fb3e4108de0 2024-04-25T11:25:22.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.566+0000 7fb3e1ffb700 1 --2- 172.21.15.69:0/591546379 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fb3d40afc10 0x7fb3d40b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:22.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.566+0000 7fb3e8c31700 1 --2- 172.21.15.69:0/591546379 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fb3d40afc10 0x7fb3d40b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:22.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.566+0000 7fb3e1ffb700 1 -- 172.21.15.69:0/591546379 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7fb3d00deba0 con 0x7fb3e4108de0 2024-04-25T11:25:22.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.567+0000 7fb3e8c31700 1 --2- 172.21.15.69:0/591546379 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fb3d40afc10 0x7fb3d40b2120 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb3e419bb10 tx=0x7fb3d800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:22.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.571+0000 7fb3e1ffb700 1 -- 172.21.15.69:0/591546379 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb3d00a72c0 con 0x7fb3e4108de0 2024-04-25T11:25:22.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.766+0000 7fb3eae95700 1 -- 172.21.15.69:0/591546379 --> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb3e419bf00 con 0x7fb3d40afc10 2024-04-25T11:25:22.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.776+0000 7fb3e1ffb700 1 -- 172.21.15.69:0/591546379 <== mgr.14152 v2:172.21.15.69:6800/4220065409 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340398 (secure 0 0 0) 0x7fb3e419bf00 con 0x7fb3d40afc10 2024-04-25T11:25:22.775 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:25:22.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.780+0000 7fb3eae95700 1 -- 172.21.15.69:0/591546379 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fb3d40afc10 msgr2=0x7fb3d40b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:22.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.780+0000 7fb3eae95700 1 --2- 172.21.15.69:0/591546379 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fb3d40afc10 0x7fb3d40b2120 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb3e419bb10 tx=0x7fb3d800a380 comp rx=0 tx=0).stop 2024-04-25T11:25:22.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.780+0000 7fb3eae95700 1 -- 172.21.15.69:0/591546379 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb3e4108de0 msgr2=0x7fb3e41a0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:22.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.780+0000 7fb3eae95700 1 --2- 172.21.15.69:0/591546379 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb3e4108de0 0x7fb3e41a0cf0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb3d000d980 tx=0x7fb3d000dc90 comp rx=0 tx=0).stop 2024-04-25T11:25:22.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.780+0000 7fb3eae95700 1 -- 172.21.15.69:0/591546379 shutdown_connections 2024-04-25T11:25:22.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.780+0000 7fb3eae95700 1 --2- 172.21.15.69:0/591546379 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fb3d40afc10 0x7fb3d40b2120 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:22.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.780+0000 7fb3eae95700 1 --2- 172.21.15.69:0/591546379 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3e4109730 0x7fb3e419a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:22.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.780+0000 7fb3eae95700 1 --2- 172.21.15.69:0/591546379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3e41044b0 0x7fb3e41a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:22.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.780+0000 7fb3eae95700 1 --2- 172.21.15.69:0/591546379 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb3e4108de0 0x7fb3e41a0cf0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:22.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.780+0000 7fb3eae95700 1 -- 172.21.15.69:0/591546379 >> 172.21.15.69:0/591546379 conn(0x7fb3e40ffda0 msgr2=0x7fb3e410a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:22.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.780+0000 7fb3eae95700 1 -- 172.21.15.69:0/591546379 shutdown_connections 2024-04-25T11:25:22.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:22.780+0000 7fb3eae95700 1 -- 172.21.15.69:0/591546379 wait complete. 2024-04-25T11:25:22.778 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-04-25T11:25:22.942 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:22 smithi069 ceph-mon[29188]: pgmap v337: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:25:23.098 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:22 smithi146 ceph-mon[32821]: pgmap v337: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:25:23.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:22 smithi152 ceph-mon[32701]: pgmap v337: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:25:23.577 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":337,"stamp":"2024-04-25T11:25:21.809007+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658996,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":795,"num_read_kb":509,"num_write":366,"num_write_kb":2290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":2953547,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366472,"kb_used_data":13076,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575352,"statfs":{"total":1151940427776,"available":1151565160448,"internally_reserved":0,"allocated":13389824,"data_stored":7418570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004251"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.879610+0000","last_change":"2024-04-25T11:23:41.646759+0000","last_active":"2024-04-25T11:24:32.879610+0000","last_peered":"2024-04-25T11:24:32.879610+0000","last_clean":"2024-04-25T11:24:32.879610+0000","last_became_active":"2024-04-25T11:23:41.645882+0000","last_became_peered":"2024-04-25T11:23:41.645882+0000","last_unstale":"2024-04-25T11:24:32.879610+0000","last_undegraded":"2024-04-25T11:24:32.879610+0000","last_fullsized":"2024-04-25T11:24:32.879610+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:05.776414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.882273+0000","last_change":"2024-04-25T11:23:41.648561+0000","last_active":"2024-04-25T11:24:32.882273+0000","last_peered":"2024-04-25T11:24:32.882273+0000","last_clean":"2024-04-25T11:24:32.882273+0000","last_became_active":"2024-04-25T11:23:41.647735+0000","last_became_peered":"2024-04-25T11:23:41.647735+0000","last_unstale":"2024-04-25T11:24:32.882273+0000","last_undegraded":"2024-04-25T11:24:32.882273+0000","last_fullsized":"2024-04-25T11:24:32.882273+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:20.303934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.881497+0000","last_change":"2024-04-25T11:23:41.646656+0000","last_active":"2024-04-25T11:24:32.881497+0000","last_peered":"2024-04-25T11:24:32.881497+0000","last_clean":"2024-04-25T11:24:32.881497+0000","last_became_active":"2024-04-25T11:23:41.645487+0000","last_became_peered":"2024-04-25T11:23:41.645487+0000","last_unstale":"2024-04-25T11:24:32.881497+0000","last_undegraded":"2024-04-25T11:24:32.881497+0000","last_fullsized":"2024-04-25T11:24:32.881497+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:32.747478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.884452+0000","last_change":"2024-04-25T11:23:41.651994+0000","last_active":"2024-04-25T11:24:32.884452+0000","last_peered":"2024-04-25T11:24:32.884452+0000","last_clean":"2024-04-25T11:24:32.884452+0000","last_became_active":"2024-04-25T11:23:41.651640+0000","last_became_peered":"2024-04-25T11:23:41.651640+0000","last_unstale":"2024-04-25T11:24:32.884452+0000","last_undegraded":"2024-04-25T11:24:32.884452+0000","last_fullsized":"2024-04-25T11:24:32.884452+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:09.376398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.880729+0000","last_change":"2024-04-25T11:23:41.650954+0000","last_active":"2024-04-25T11:24:32.880729+0000","last_peered":"2024-04-25T11:24:32.880729+0000","last_clean":"2024-04-25T11:24:32.880729+0000","last_became_active":"2024-04-25T11:23:41.650553+0000","last_became_peered":"2024-04-25T11:23:41.650553+0000","last_unstale":"2024-04-25T11:24:32.880729+0000","last_undegraded":"2024-04-25T11:24:32.880729+0000","last_fullsized":"2024-04-25T11:24:32.880729+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:52:34.348370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.878125+0000","last_change":"2024-04-25T11:23:41.650770+0000","last_active":"2024-04-25T11:24:32.878125+0000","last_peered":"2024-04-25T11:24:32.878125+0000","last_clean":"2024-04-25T11:24:32.878125+0000","last_became_active":"2024-04-25T11:23:41.649794+0000","last_became_peered":"2024-04-25T11:23:41.649794+0000","last_unstale":"2024-04-25T11:24:32.878125+0000","last_undegraded":"2024-04-25T11:24:32.878125+0000","last_fullsized":"2024-04-25T11:24:32.878125+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:04.032001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.881238+0000","last_change":"2024-04-25T11:23:41.643753+0000","last_active":"2024-04-25T11:24:32.881238+0000","last_peered":"2024-04-25T11:24:32.881238+0000","last_clean":"2024-04-25T11:24:32.881238+0000","last_became_active":"2024-04-25T11:23:41.642854+0000","last_became_peered":"2024-04-25T11:23:41.642854+0000","last_unstale":"2024-04-25T11:24:32.881238+0000","last_undegraded":"2024-04-25T11:24:32.881238+0000","last_fullsized":"2024-04-25T11:24:32.881238+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:40.868312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.879726+0000","last_change":"2024-04-25T11:23:41.646501+0000","last_active":"2024-04-25T11:24:32.879726+0000","last_peered":"2024-04-25T11:24:32.879726+0000","last_clean":"2024-04-25T11:24:32.879726+0000","last_became_active":"2024-04-25T11:23:41.646178+0000","last_became_peered":"2024-04-25T11:23:41.646178+0000","last_unstale":"2024-04-25T11:24:32.879726+0000","last_undegraded":"2024-04-25T11:24:32.879726+0000","last_fullsized":"2024-04-25T11:24:32.879726+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:34.699979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.882936+0000","last_change":"2024-04-25T11:23:41.647025+0000","last_active":"2024-04-25T11:24:32.882936+0000","last_peered":"2024-04-25T11:24:32.882936+0000","last_clean":"2024-04-25T11:24:32.882936+0000","last_became_active":"2024-04-25T11:23:41.646645+0000","last_became_peered":"2024-04-25T11:23:41.646645+0000","last_unstale":"2024-04-25T11:24:32.882936+0000","last_undegraded":"2024-04-25T11:24:32.882936+0000","last_fullsized":"2024-04-25T11:24:32.882936+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:24:57.108535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.877863+0000","last_change":"2024-04-25T11:23:41.660957+0000","last_active":"2024-04-25T11:24:32.877863+0000","last_peered":"2024-04-25T11:24:32.877863+0000","last_clean":"2024-04-25T11:24:32.877863+0000","last_became_active":"2024-04-25T11:23:41.660529+0000","last_became_peered":"2024-04-25T11:23:41.660529+0000","last_unstale":"2024-04-25T11:24:32.877863+0000","last_undegraded":"2024-04-25T11:24:32.877863+0000","last_fullsized":"2024-04-25T11:24:32.877863+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:51.113991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.881993+0000","last_change":"2024-04-25T11:23:41.648777+0000","last_active":"2024-04-25T11:24:32.881993+0000","last_peered":"2024-04-25T11:24:32.881993+0000","last_clean":"2024-04-25T11:24:32.881993+0000","last_became_active":"2024-04-25T11:23:41.647282+0000","last_became_peered":"2024-04-25T11:23:41.647282+0000","last_unstale":"2024-04-25T11:24:32.881993+0000","last_undegraded":"2024-04-25T11:24:32.881993+0000","last_fullsized":"2024-04-25T11:24:32.881993+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:14.396228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.884161+0000","last_change":"2024-04-25T11:23:41.648855+0000","last_active":"2024-04-25T11:24:32.884161+0000","last_peered":"2024-04-25T11:24:32.884161+0000","last_clean":"2024-04-25T11:24:32.884161+0000","last_became_active":"2024-04-25T11:23:41.648527+0000","last_became_peered":"2024-04-25T11:23:41.648527+0000","last_unstale":"2024-04-25T11:24:32.884161+0000","last_undegraded":"2024-04-25T11:24:32.884161+0000","last_fullsized":"2024-04-25T11:24:32.884161+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:13.569916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.876944+0000","last_change":"2024-04-25T11:23:41.646377+0000","last_active":"2024-04-25T11:24:32.876944+0000","last_peered":"2024-04-25T11:24:32.876944+0000","last_clean":"2024-04-25T11:24:32.876944+0000","last_became_active":"2024-04-25T11:23:41.646063+0000","last_became_peered":"2024-04-25T11:23:41.646063+0000","last_unstale":"2024-04-25T11:24:32.876944+0000","last_undegraded":"2024-04-25T11:24:32.876944+0000","last_fullsized":"2024-04-25T11:24:32.876944+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:50.365317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.881286+0000","last_change":"2024-04-25T11:23:41.648147+0000","last_active":"2024-04-25T11:24:32.881286+0000","last_peered":"2024-04-25T11:24:32.881286+0000","last_clean":"2024-04-25T11:24:32.881286+0000","last_became_active":"2024-04-25T11:23:41.647771+0000","last_became_peered":"2024-04-25T11:23:41.647771+0000","last_unstale":"2024-04-25T11:24:32.881286+0000","last_undegraded":"2024-04-25T11:24:32.881286+0000","last_fullsized":"2024-04-25T11:24:32.881286+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:20.157971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.881212+0000","last_change":"2024-04-25T11:23:41.652784+0000","last_active":"2024-04-25T11:24:32.881212+0000","last_peered":"2024-04-25T11:24:32.881212+0000","last_clean":"2024-04-25T11:24:32.881212+0000","last_became_active":"2024-04-25T11:23:41.652434+0000","last_became_peered":"2024-04-25T11:23:41.652434+0000","last_unstale":"2024-04-25T11:24:32.881212+0000","last_undegraded":"2024-04-25T11:24:32.881212+0000","last_fullsized":"2024-04-25T11:24:32.881212+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:48.468138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.882082+0000","last_change":"2024-04-25T11:23:41.648218+0000","last_active":"2024-04-25T11:24:32.882082+0000","last_peered":"2024-04-25T11:24:32.882082+0000","last_clean":"2024-04-25T11:24:32.882082+0000","last_became_active":"2024-04-25T11:23:41.647871+0000","last_became_peered":"2024-04-25T11:23:41.647871+0000","last_unstale":"2024-04-25T11:24:32.882082+0000","last_undegraded":"2024-04-25T11:24:32.882082+0000","last_fullsized":"2024-04-25T11:24:32.882082+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:49.919090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.878985+0000","last_change":"2024-04-25T11:23:41.647087+0000","last_active":"2024-04-25T11:24:32.878985+0000","last_peered":"2024-04-25T11:24:32.878985+0000","last_clean":"2024-04-25T11:24:32.878985+0000","last_became_active":"2024-04-25T11:23:41.646359+0000","last_became_peered":"2024-04-25T11:23:41.646359+0000","last_unstale":"2024-04-25T11:24:32.878985+0000","last_undegraded":"2024-04-25T11:24:32.878985+0000","last_fullsized":"2024-04-25T11:24:32.878985+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:32.129538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.977554+0000","last_change":"2024-04-25T11:23:41.644143+0000","last_active":"2024-04-25T11:24:32.977554+0000","last_peered":"2024-04-25T11:24:32.977554+0000","last_clean":"2024-04-25T11:24:32.977554+0000","last_became_active":"2024-04-25T11:23:41.643738+0000","last_became_peered":"2024-04-25T11:23:41.643738+0000","last_unstale":"2024-04-25T11:24:32.977554+0000","last_undegraded":"2024-04-25T11:24:32.977554+0000","last_fullsized":"2024-04-25T11:24:32.977554+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:19.786933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.881965+0000","last_change":"2024-04-25T11:23:41.636421+0000","last_active":"2024-04-25T11:24:32.881965+0000","last_peered":"2024-04-25T11:24:32.881965+0000","last_clean":"2024-04-25T11:24:32.881965+0000","last_became_active":"2024-04-25T11:23:41.635392+0000","last_became_peered":"2024-04-25T11:23:41.635392+0000","last_unstale":"2024-04-25T11:24:32.881965+0000","last_undegraded":"2024-04-25T11:24:32.881965+0000","last_fullsized":"2024-04-25T11:24:32.881965+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:59.859340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.883464+0000","last_change":"2024-04-25T11:23:45.675875+0000","last_active":"2024-04-25T11:24:32.883464+0000","last_peered":"2024-04-25T11:24:32.883464+0000","last_clean":"2024-04-25T11:24:32.883464+0000","last_became_active":"2024-04-25T11:23:45.675405+0000","last_became_peered":"2024-04-25T11:23:45.675405+0000","last_unstale":"2024-04-25T11:24:32.883464+0000","last_undegraded":"2024-04-25T11:24:32.883464+0000","last_fullsized":"2024-04-25T11:24:32.883464+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:06.732509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":112,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:25:17.162736+0000","last_change":"2024-04-25T11:23:40.630862+0000","last_active":"2024-04-25T11:25:17.162736+0000","last_peered":"2024-04-25T11:25:17.162736+0000","last_clean":"2024-04-25T11:25:17.162736+0000","last_became_active":"2024-04-25T11:23:40.630498+0000","last_became_peered":"2024-04-25T11:23:40.630498+0000","last_unstale":"2024-04-25T11:25:17.162736+0000","last_undegraded":"2024-04-25T11:25:17.162736+0000","last_fullsized":"2024-04-25T11:25:17.162736+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:07:44.028640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":71,"num_read_kb":71,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.879916+0000","last_change":"2024-04-25T11:23:41.639879+0000","last_active":"2024-04-25T11:24:32.879916+0000","last_peered":"2024-04-25T11:24:32.879916+0000","last_clean":"2024-04-25T11:24:32.879916+0000","last_became_active":"2024-04-25T11:23:41.639359+0000","last_became_peered":"2024-04-25T11:23:41.639359+0000","last_unstale":"2024-04-25T11:24:32.879916+0000","last_undegraded":"2024-04-25T11:24:32.879916+0000","last_fullsized":"2024-04-25T11:24:32.879916+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:47.525089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.882272+0000","last_change":"2024-04-25T11:23:45.672219+0000","last_active":"2024-04-25T11:24:32.882272+0000","last_peered":"2024-04-25T11:24:32.882272+0000","last_clean":"2024-04-25T11:24:32.882272+0000","last_became_active":"2024-04-25T11:23:45.671788+0000","last_became_peered":"2024-04-25T11:23:45.671788+0000","last_unstale":"2024-04-25T11:24:32.882272+0000","last_undegraded":"2024-04-25T11:24:32.882272+0000","last_fullsized":"2024-04-25T11:24:32.882272+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:36.228061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.880268+0000","last_change":"2024-04-25T11:23:40.643435+0000","last_active":"2024-04-25T11:24:32.880268+0000","last_peered":"2024-04-25T11:24:32.880268+0000","last_clean":"2024-04-25T11:24:32.880268+0000","last_became_active":"2024-04-25T11:23:40.642990+0000","last_became_peered":"2024-04-25T11:23:40.642990+0000","last_unstale":"2024-04-25T11:24:32.880268+0000","last_undegraded":"2024-04-25T11:24:32.880268+0000","last_fullsized":"2024-04-25T11:24:32.880268+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:42.723486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.881812+0000","last_change":"2024-04-25T11:23:41.645977+0000","last_active":"2024-04-25T11:24:32.881812+0000","last_peered":"2024-04-25T11:24:32.881812+0000","last_clean":"2024-04-25T11:24:32.881812+0000","last_became_active":"2024-04-25T11:23:41.645562+0000","last_became_peered":"2024-04-25T11:23:41.645562+0000","last_unstale":"2024-04-25T11:24:32.881812+0000","last_undegraded":"2024-04-25T11:24:32.881812+0000","last_fullsized":"2024-04-25T11:24:32.881812+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:04.027324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.883329+0000","last_change":"2024-04-25T11:23:45.672843+0000","last_active":"2024-04-25T11:24:32.883329+0000","last_peered":"2024-04-25T11:24:32.883329+0000","last_clean":"2024-04-25T11:24:32.883329+0000","last_became_active":"2024-04-25T11:23:45.672429+0000","last_became_peered":"2024-04-25T11:23:45.672429+0000","last_unstale":"2024-04-25T11:24:32.883329+0000","last_undegraded":"2024-04-25T11:24:32.883329+0000","last_fullsized":"2024-04-25T11:24:32.883329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:48.818624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.883104+0000","last_change":"2024-04-25T11:23:40.634393+0000","last_active":"2024-04-25T11:24:32.883104+0000","last_peered":"2024-04-25T11:24:32.883104+0000","last_clean":"2024-04-25T11:24:32.883104+0000","last_became_active":"2024-04-25T11:23:40.633518+0000","last_became_peered":"2024-04-25T11:23:40.633518+0000","last_unstale":"2024-04-25T11:24:32.883104+0000","last_undegraded":"2024-04-25T11:24:32.883104+0000","last_fullsized":"2024-04-25T11:24:32.883104+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:59.623135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.883017+0000","last_change":"2024-04-25T11:23:41.656152+0000","last_active":"2024-04-25T11:24:32.883017+0000","last_peered":"2024-04-25T11:24:32.883017+0000","last_clean":"2024-04-25T11:24:32.883017+0000","last_became_active":"2024-04-25T11:23:41.655344+0000","last_became_peered":"2024-04-25T11:23:41.655344+0000","last_unstale":"2024-04-25T11:24:32.883017+0000","last_undegraded":"2024-04-25T11:24:32.883017+0000","last_fullsized":"2024-04-25T11:24:32.883017+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:07.649534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.883028+0000","last_change":"2024-04-25T11:23:45.668602+0000","last_active":"2024-04-25T11:24:32.883028+0000","last_peered":"2024-04-25T11:24:32.883028+0000","last_clean":"2024-04-25T11:24:32.883028+0000","last_became_active":"2024-04-25T11:23:45.668143+0000","last_became_peered":"2024-04-25T11:23:45.668143+0000","last_unstale":"2024-04-25T11:24:32.883028+0000","last_undegraded":"2024-04-25T11:24:32.883028+0000","last_fullsized":"2024-04-25T11:24:32.883028+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:13.463604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.898973+0000","last_change":"2024-04-25T11:24:32.898346+0000","last_active":"2024-04-25T11:24:32.898973+0000","last_peered":"2024-04-25T11:24:32.898973+0000","last_clean":"2024-04-25T11:24:32.898973+0000","last_became_active":"2024-04-25T11:24:32.887657+0000","last_became_peered":"2024-04-25T11:24:32.887657+0000","last_unstale":"2024-04-25T11:24:32.898973+0000","last_undegraded":"2024-04-25T11:24:32.898973+0000","last_fullsized":"2024-04-25T11:24:32.898973+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:48.348173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.886185+0000","last_change":"2024-04-25T11:23:41.657112+0000","last_active":"2024-04-25T11:24:32.886185+0000","last_peered":"2024-04-25T11:24:32.886185+0000","last_clean":"2024-04-25T11:24:32.886185+0000","last_became_active":"2024-04-25T11:23:41.656623+0000","last_became_peered":"2024-04-25T11:23:41.656623+0000","last_unstale":"2024-04-25T11:24:32.886185+0000","last_undegraded":"2024-04-25T11:24:32.886185+0000","last_fullsized":"2024-04-25T11:24:32.886185+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:41.644825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.881725+0000","last_change":"2024-04-25T11:23:45.673373+0000","last_active":"2024-04-25T11:24:32.881725+0000","last_peered":"2024-04-25T11:24:32.881725+0000","last_clean":"2024-04-25T11:24:32.881725+0000","last_became_active":"2024-04-25T11:23:45.672928+0000","last_became_peered":"2024-04-25T11:23:45.672928+0000","last_unstale":"2024-04-25T11:24:32.881725+0000","last_undegraded":"2024-04-25T11:24:32.881725+0000","last_fullsized":"2024-04-25T11:24:32.881725+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:10.553189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"80'6","reported_seq":66,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:25:19.964876+0000","last_change":"2024-04-25T11:24:32.901351+0000","last_active":"2024-04-25T11:25:19.964876+0000","last_peered":"2024-04-25T11:25:19.964876+0000","last_clean":"2024-04-25T11:25:19.964876+0000","last_became_active":"2024-04-25T11:24:32.888110+0000","last_became_peered":"2024-04-25T11:24:32.888110+0000","last_unstale":"2024-04-25T11:25:19.964876+0000","last_undegraded":"2024-04-25T11:25:19.964876+0000","last_fullsized":"2024-04-25T11:25:19.964876+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:54.806224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":49,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.882868+0000","last_change":"2024-04-25T11:23:41.648771+0000","last_active":"2024-04-25T11:24:32.882868+0000","last_peered":"2024-04-25T11:24:32.882868+0000","last_clean":"2024-04-25T11:24:32.882868+0000","last_became_active":"2024-04-25T11:23:41.648059+0000","last_became_peered":"2024-04-25T11:23:41.648059+0000","last_unstale":"2024-04-25T11:24:32.882868+0000","last_undegraded":"2024-04-25T11:24:32.882868+0000","last_fullsized":"2024-04-25T11:24:32.882868+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:27.873310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.882752+0000","last_change":"2024-04-25T11:23:45.672676+0000","last_active":"2024-04-25T11:24:32.882752+0000","last_peered":"2024-04-25T11:24:32.882752+0000","last_clean":"2024-04-25T11:24:32.882752+0000","last_became_active":"2024-04-25T11:23:45.672229+0000","last_became_peered":"2024-04-25T11:23:45.672229+0000","last_unstale":"2024-04-25T11:24:32.882752+0000","last_undegraded":"2024-04-25T11:24:32.882752+0000","last_fullsized":"2024-04-25T11:24:32.882752+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:16.648809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":50,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.894986+0000","last_change":"2024-04-25T11:24:32.894986+0000","last_active":"2024-04-25T11:24:32.894986+0000","last_peered":"2024-04-25T11:24:32.894986+0000","last_clean":"2024-04-25T11:24:32.894986+0000","last_became_active":"2024-04-25T11:24:32.888077+0000","last_became_peered":"2024-04-25T11:24:32.888077+0000","last_unstale":"2024-04-25T11:24:32.894986+0000","last_undegraded":"2024-04-25T11:24:32.894986+0000","last_fullsized":"2024-04-25T11:24:32.894986+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:15.739398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.886977+0000","last_change":"2024-04-25T11:23:41.656031+0000","last_active":"2024-04-25T11:24:32.886977+0000","last_peered":"2024-04-25T11:24:32.886977+0000","last_clean":"2024-04-25T11:24:32.886977+0000","last_became_active":"2024-04-25T11:23:41.655656+0000","last_became_peered":"2024-04-25T11:23:41.655656+0000","last_unstale":"2024-04-25T11:24:32.886977+0000","last_undegraded":"2024-04-25T11:24:32.886977+0000","last_fullsized":"2024-04-25T11:24:32.886977+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:37.991435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.889950+0000","last_change":"2024-04-25T11:24:32.889950+0000","last_active":"2024-04-25T11:24:32.889950+0000","last_peered":"2024-04-25T11:24:32.889950+0000","last_clean":"2024-04-25T11:24:32.889950+0000","last_became_active":"2024-04-25T11:24:32.888985+0000","last_became_peered":"2024-04-25T11:24:32.888985+0000","last_unstale":"2024-04-25T11:24:32.889950+0000","last_undegraded":"2024-04-25T11:24:32.889950+0000","last_fullsized":"2024-04-25T11:24:32.889950+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:28.218707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.876579+0000","last_change":"2024-04-25T11:23:40.631581+0000","last_active":"2024-04-25T11:24:32.876579+0000","last_peered":"2024-04-25T11:24:32.876579+0000","last_clean":"2024-04-25T11:24:32.876579+0000","last_became_active":"2024-04-25T11:23:40.631169+0000","last_became_peered":"2024-04-25T11:23:40.631169+0000","last_unstale":"2024-04-25T11:24:32.876579+0000","last_undegraded":"2024-04-25T11:24:32.876579+0000","last_fullsized":"2024-04-25T11:24:32.876579+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:15.737402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.879601+0000","last_change":"2024-04-25T11:23:41.633740+0000","last_active":"2024-04-25T11:24:32.879601+0000","last_peered":"2024-04-25T11:24:32.879601+0000","last_clean":"2024-04-25T11:24:32.879601+0000","last_became_active":"2024-04-25T11:23:41.633445+0000","last_became_peered":"2024-04-25T11:23:41.633445+0000","last_unstale":"2024-04-25T11:24:32.879601+0000","last_undegraded":"2024-04-25T11:24:32.879601+0000","last_fullsized":"2024-04-25T11:24:32.879601+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:46.103646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.882156+0000","last_change":"2024-04-25T11:23:45.678899+0000","last_active":"2024-04-25T11:24:32.882156+0000","last_peered":"2024-04-25T11:24:32.882156+0000","last_clean":"2024-04-25T11:24:32.882156+0000","last_became_active":"2024-04-25T11:23:45.675305+0000","last_became_peered":"2024-04-25T11:23:45.675305+0000","last_unstale":"2024-04-25T11:24:32.882156+0000","last_undegraded":"2024-04-25T11:24:32.882156+0000","last_fullsized":"2024-04-25T11:24:32.882156+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:49.456551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.885598+0000","last_change":"2024-04-25T11:23:40.638090+0000","last_active":"2024-04-25T11:24:32.885598+0000","last_peered":"2024-04-25T11:24:32.885598+0000","last_clean":"2024-04-25T11:24:32.885598+0000","last_became_active":"2024-04-25T11:23:40.637707+0000","last_became_peered":"2024-04-25T11:23:40.637707+0000","last_unstale":"2024-04-25T11:24:32.885598+0000","last_undegraded":"2024-04-25T11:24:32.885598+0000","last_fullsized":"2024-04-25T11:24:32.885598+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:38:43.413273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.977170+0000","last_change":"2024-04-25T11:23:41.641580+0000","last_active":"2024-04-25T11:24:32.977170+0000","last_peered":"2024-04-25T11:24:32.977170+0000","last_clean":"2024-04-25T11:24:32.977170+0000","last_became_active":"2024-04-25T11:23:41.641199+0000","last_became_peered":"2024-04-25T11:23:41.641199+0000","last_unstale":"2024-04-25T11:24:32.977170+0000","last_undegraded":"2024-04-25T11:24:32.977170+0000","last_fullsized":"2024-04-25T11:24:32.977170+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:34:10.471476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.879873+0000","last_change":"2024-04-25T11:23:45.675573+0000","last_active":"2024-04-25T11:24:32.879873+0000","last_peered":"2024-04-25T11:24:32.879873+0000","last_clean":"2024-04-25T11:24:32.879873+0000","last_became_active":"2024-04-25T11:23:45.675197+0000","last_became_peered":"2024-04-25T11:23:45.675197+0000","last_unstale":"2024-04-25T11:24:32.879873+0000","last_undegraded":"2024-04-25T11:24:32.879873+0000","last_fullsized":"2024-04-25T11:24:32.879873+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:39.740949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":174,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:25:17.961070+0000","last_change":"2024-04-25T11:23:40.635020+0000","last_active":"2024-04-25T11:25:17.961070+0000","last_peered":"2024-04-25T11:25:17.961070+0000","last_clean":"2024-04-25T11:25:17.961070+0000","last_became_active":"2024-04-25T11:23:40.634549+0000","last_became_peered":"2024-04-25T11:23:40.634549+0000","last_unstale":"2024-04-25T11:25:17.961070+0000","last_undegraded":"2024-04-25T11:25:17.961070+0000","last_fullsized":"2024-04-25T11:25:17.961070+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:15:16.289370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":141,"num_read_kb":141,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.883366+0000","last_change":"2024-04-25T11:23:41.654903+0000","last_active":"2024-04-25T11:24:32.883366+0000","last_peered":"2024-04-25T11:24:32.883366+0000","last_clean":"2024-04-25T11:24:32.883366+0000","last_became_active":"2024-04-25T11:23:41.654564+0000","last_became_peered":"2024-04-25T11:23:41.654564+0000","last_unstale":"2024-04-25T11:24:32.883366+0000","last_undegraded":"2024-04-25T11:24:32.883366+0000","last_fullsized":"2024-04-25T11:24:32.883366+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:29.025240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.976910+0000","last_change":"2024-04-25T11:23:45.677683+0000","last_active":"2024-04-25T11:24:32.976910+0000","last_peered":"2024-04-25T11:24:32.976910+0000","last_clean":"2024-04-25T11:24:32.976910+0000","last_became_active":"2024-04-25T11:23:45.677286+0000","last_became_peered":"2024-04-25T11:23:45.677286+0000","last_unstale":"2024-04-25T11:24:32.976910+0000","last_undegraded":"2024-04-25T11:24:32.976910+0000","last_fullsized":"2024-04-25T11:24:32.976910+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:43.236060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.881927+0000","last_change":"2024-04-25T11:23:40.640844+0000","last_active":"2024-04-25T11:24:32.881927+0000","last_peered":"2024-04-25T11:24:32.881927+0000","last_clean":"2024-04-25T11:24:32.881927+0000","last_became_active":"2024-04-25T11:23:40.640434+0000","last_became_peered":"2024-04-25T11:23:40.640434+0000","last_unstale":"2024-04-25T11:24:32.881927+0000","last_undegraded":"2024-04-25T11:24:32.881927+0000","last_fullsized":"2024-04-25T11:24:32.881927+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:16.701418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.882074+0000","last_change":"2024-04-25T11:23:41.651962+0000","last_active":"2024-04-25T11:24:32.882074+0000","last_peered":"2024-04-25T11:24:32.882074+0000","last_clean":"2024-04-25T11:24:32.882074+0000","last_became_active":"2024-04-25T11:23:41.650870+0000","last_became_peered":"2024-04-25T11:23:41.650870+0000","last_unstale":"2024-04-25T11:24:32.882074+0000","last_undegraded":"2024-04-25T11:24:32.882074+0000","last_fullsized":"2024-04-25T11:24:32.882074+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:45.623722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.892455+0000","last_change":"2024-04-25T11:24:32.892455+0000","last_active":"2024-04-25T11:24:32.892455+0000","last_peered":"2024-04-25T11:24:32.892455+0000","last_clean":"2024-04-25T11:24:32.892455+0000","last_became_active":"2024-04-25T11:24:32.891968+0000","last_became_peered":"2024-04-25T11:24:32.891968+0000","last_unstale":"2024-04-25T11:24:32.892455+0000","last_undegraded":"2024-04-25T11:24:32.892455+0000","last_fullsized":"2024-04-25T11:24:32.892455+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:31.962400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.886087+0000","last_change":"2024-04-25T11:23:40.642112+0000","last_active":"2024-04-25T11:24:32.886087+0000","last_peered":"2024-04-25T11:24:32.886087+0000","last_clean":"2024-04-25T11:24:32.886087+0000","last_became_active":"2024-04-25T11:23:40.641745+0000","last_became_peered":"2024-04-25T11:23:40.641745+0000","last_unstale":"2024-04-25T11:24:32.886087+0000","last_undegraded":"2024-04-25T11:24:32.886087+0000","last_fullsized":"2024-04-25T11:24:32.886087+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:14.899432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.883183+0000","last_change":"2024-04-25T11:23:41.656460+0000","last_active":"2024-04-25T11:24:32.883183+0000","last_peered":"2024-04-25T11:24:32.883183+0000","last_clean":"2024-04-25T11:24:32.883183+0000","last_became_active":"2024-04-25T11:23:41.655800+0000","last_became_peered":"2024-04-25T11:23:41.655800+0000","last_unstale":"2024-04-25T11:24:32.883183+0000","last_undegraded":"2024-04-25T11:24:32.883183+0000","last_fullsized":"2024-04-25T11:24:32.883183+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:42.505142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.977193+0000","last_change":"2024-04-25T11:23:45.671650+0000","last_active":"2024-04-25T11:24:32.977193+0000","last_peered":"2024-04-25T11:24:32.977193+0000","last_clean":"2024-04-25T11:24:32.977193+0000","last_became_active":"2024-04-25T11:23:45.671254+0000","last_became_peered":"2024-04-25T11:23:45.671254+0000","last_unstale":"2024-04-25T11:24:32.977193+0000","last_undegraded":"2024-04-25T11:24:32.977193+0000","last_fullsized":"2024-04-25T11:24:32.977193+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:29.404898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"80'34","reported_seq":210,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:25:17.961100+0000","last_change":"2024-04-25T11:23:40.633712+0000","last_active":"2024-04-25T11:25:17.961100+0000","last_peered":"2024-04-25T11:25:17.961100+0000","last_clean":"2024-04-25T11:25:17.961100+0000","last_became_active":"2024-04-25T11:23:40.633274+0000","last_became_peered":"2024-04-25T11:23:40.633274+0000","last_unstale":"2024-04-25T11:25:17.961100+0000","last_undegraded":"2024-04-25T11:25:17.961100+0000","last_fullsized":"2024-04-25T11:25:17.961100+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:47.576213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61190,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":147,"num_read_kb":134,"num_write":36,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.881944+0000","last_change":"2024-04-25T11:23:41.649007+0000","last_active":"2024-04-25T11:24:32.881944+0000","last_peered":"2024-04-25T11:24:32.881944+0000","last_clean":"2024-04-25T11:24:32.881944+0000","last_became_active":"2024-04-25T11:23:41.647778+0000","last_became_peered":"2024-04-25T11:23:41.647778+0000","last_unstale":"2024-04-25T11:24:32.881944+0000","last_undegraded":"2024-04-25T11:24:32.881944+0000","last_fullsized":"2024-04-25T11:24:32.881944+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:33.097482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.886452+0000","last_change":"2024-04-25T11:23:45.671645+0000","last_active":"2024-04-25T11:24:32.886452+0000","last_peered":"2024-04-25T11:24:32.886452+0000","last_clean":"2024-04-25T11:24:32.886452+0000","last_became_active":"2024-04-25T11:23:45.670005+0000","last_became_peered":"2024-04-25T11:23:45.670005+0000","last_unstale":"2024-04-25T11:24:32.886452+0000","last_undegraded":"2024-04-25T11:24:32.886452+0000","last_fullsized":"2024-04-25T11:24:32.886452+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:36.047222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.881964+0000","last_change":"2024-04-25T11:23:40.639922+0000","last_active":"2024-04-25T11:24:32.881964+0000","last_peered":"2024-04-25T11:24:32.881964+0000","last_clean":"2024-04-25T11:24:32.881964+0000","last_became_active":"2024-04-25T11:23:40.639544+0000","last_became_peered":"2024-04-25T11:23:40.639544+0000","last_unstale":"2024-04-25T11:24:32.881964+0000","last_undegraded":"2024-04-25T11:24:32.881964+0000","last_fullsized":"2024-04-25T11:24:32.881964+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:06.281281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.876685+0000","last_change":"2024-04-25T11:23:41.629257+0000","last_active":"2024-04-25T11:24:32.876685+0000","last_peered":"2024-04-25T11:24:32.876685+0000","last_clean":"2024-04-25T11:24:32.876685+0000","last_became_active":"2024-04-25T11:23:41.628933+0000","last_became_peered":"2024-04-25T11:23:41.628933+0000","last_unstale":"2024-04-25T11:24:32.876685+0000","last_undegraded":"2024-04-25T11:24:32.876685+0000","last_fullsized":"2024-04-25T11:24:32.876685+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:04.031468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.878397+0000","last_change":"2024-04-25T11:23:45.680460+0000","last_active":"2024-04-25T11:24:32.878397+0000","last_peered":"2024-04-25T11:24:32.878397+0000","last_clean":"2024-04-25T11:24:32.878397+0000","last_became_active":"2024-04-25T11:23:45.679963+0000","last_became_peered":"2024-04-25T11:23:45.679963+0000","last_unstale":"2024-04-25T11:24:32.878397+0000","last_undegraded":"2024-04-25T11:24:32.878397+0000","last_fullsized":"2024-04-25T11:24:32.878397+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:11.199855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.885689+0000","last_change":"2024-04-25T11:23:40.644026+0000","last_active":"2024-04-25T11:24:32.885689+0000","last_peered":"2024-04-25T11:24:32.885689+0000","last_clean":"2024-04-25T11:24:32.885689+0000","last_became_active":"2024-04-25T11:23:40.643665+0000","last_became_peered":"2024-04-25T11:23:40.643665+0000","last_unstale":"2024-04-25T11:24:32.885689+0000","last_undegraded":"2024-04-25T11:24:32.885689+0000","last_fullsized":"2024-04-25T11:24:32.885689+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:32.950387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.881343+0000","last_change":"2024-04-25T11:23:41.642285+0000","last_active":"2024-04-25T11:24:32.881343+0000","last_peered":"2024-04-25T11:24:32.881343+0000","last_clean":"2024-04-25T11:24:32.881343+0000","last_became_active":"2024-04-25T11:23:41.641712+0000","last_became_peered":"2024-04-25T11:23:41.641712+0000","last_unstale":"2024-04-25T11:24:32.881343+0000","last_undegraded":"2024-04-25T11:24:32.881343+0000","last_fullsized":"2024-04-25T11:24:32.881343+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:22.813546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.977448+0000","last_change":"2024-04-25T11:23:45.677632+0000","last_active":"2024-04-25T11:24:32.977448+0000","last_peered":"2024-04-25T11:24:32.977448+0000","last_clean":"2024-04-25T11:24:32.977448+0000","last_became_active":"2024-04-25T11:23:45.677244+0000","last_became_peered":"2024-04-25T11:23:45.677244+0000","last_unstale":"2024-04-25T11:24:32.977448+0000","last_undegraded":"2024-04-25T11:24:32.977448+0000","last_fullsized":"2024-04-25T11:24:32.977448+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:03.982435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"80'5","reported_seq":107,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:25:17.180078+0000","last_change":"2024-04-25T11:23:40.634661+0000","last_active":"2024-04-25T11:25:17.180078+0000","last_peered":"2024-04-25T11:25:17.180078+0000","last_clean":"2024-04-25T11:25:17.180078+0000","last_became_active":"2024-04-25T11:23:40.634211+0000","last_became_peered":"2024-04-25T11:23:40.634211+0000","last_unstale":"2024-04-25T11:25:17.180078+0000","last_undegraded":"2024-04-25T11:25:17.180078+0000","last_fullsized":"2024-04-25T11:25:17.180078+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:54.928339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.881199+0000","last_change":"2024-04-25T11:23:41.644147+0000","last_active":"2024-04-25T11:24:32.881199+0000","last_peered":"2024-04-25T11:24:32.881199+0000","last_clean":"2024-04-25T11:24:32.881199+0000","last_became_active":"2024-04-25T11:23:41.643103+0000","last_became_peered":"2024-04-25T11:23:41.643103+0000","last_unstale":"2024-04-25T11:24:32.881199+0000","last_undegraded":"2024-04-25T11:24:32.881199+0000","last_fullsized":"2024-04-25T11:24:32.881199+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:00.956551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.877579+0000","last_change":"2024-04-25T11:23:45.671504+0000","last_active":"2024-04-25T11:24:32.877579+0000","last_peered":"2024-04-25T11:24:32.877579+0000","last_clean":"2024-04-25T11:24:32.877579+0000","last_became_active":"2024-04-25T11:23:45.671148+0000","last_became_peered":"2024-04-25T11:23:45.671148+0000","last_unstale":"2024-04-25T11:24:32.877579+0000","last_undegraded":"2024-04-25T11:24:32.877579+0000","last_fullsized":"2024-04-25T11:24:32.877579+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:21.795839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":70,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.906235+0000","last_change":"2024-04-25T11:24:32.906235+0000","last_active":"2024-04-25T11:24:32.906235+0000","last_peered":"2024-04-25T11:24:32.906235+0000","last_clean":"2024-04-25T11:24:32.906235+0000","last_became_active":"2024-04-25T11:24:32.890228+0000","last_became_peered":"2024-04-25T11:24:32.890228+0000","last_unstale":"2024-04-25T11:24:32.906235+0000","last_undegraded":"2024-04-25T11:24:32.906235+0000","last_fullsized":"2024-04-25T11:24:32.906235+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:44.096420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.885831+0000","last_change":"2024-04-25T11:23:41.658283+0000","last_active":"2024-04-25T11:24:32.885831+0000","last_peered":"2024-04-25T11:24:32.885831+0000","last_clean":"2024-04-25T11:24:32.885831+0000","last_became_active":"2024-04-25T11:23:41.657898+0000","last_became_peered":"2024-04-25T11:23:41.657898+0000","last_unstale":"2024-04-25T11:24:32.885831+0000","last_undegraded":"2024-04-25T11:24:32.885831+0000","last_fullsized":"2024-04-25T11:24:32.885831+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:52:55.941232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.889595+0000","last_change":"2024-04-25T11:24:32.888574+0000","last_active":"2024-04-25T11:24:32.889595+0000","last_peered":"2024-04-25T11:24:32.889595+0000","last_clean":"2024-04-25T11:24:32.889595+0000","last_became_active":"2024-04-25T11:24:32.888047+0000","last_became_peered":"2024-04-25T11:24:32.888047+0000","last_unstale":"2024-04-25T11:24:32.889595+0000","last_undegraded":"2024-04-25T11:24:32.889595+0000","last_fullsized":"2024-04-25T11:24:32.889595+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:20.530042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.883512+0000","last_change":"2024-04-25T11:23:40.639442+0000","last_active":"2024-04-25T11:24:32.883512+0000","last_peered":"2024-04-25T11:24:32.883512+0000","last_clean":"2024-04-25T11:24:32.883512+0000","last_became_active":"2024-04-25T11:23:40.639068+0000","last_became_peered":"2024-04-25T11:23:40.639068+0000","last_unstale":"2024-04-25T11:24:32.883512+0000","last_undegraded":"2024-04-25T11:24:32.883512+0000","last_fullsized":"2024-04-25T11:24:32.883512+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:21.491233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.878774+0000","last_change":"2024-04-25T11:23:41.652309+0000","last_active":"2024-04-25T11:24:32.878774+0000","last_peered":"2024-04-25T11:24:32.878774+0000","last_clean":"2024-04-25T11:24:32.878774+0000","last_became_active":"2024-04-25T11:23:41.651927+0000","last_became_peered":"2024-04-25T11:23:41.651927+0000","last_unstale":"2024-04-25T11:24:32.878774+0000","last_undegraded":"2024-04-25T11:24:32.878774+0000","last_fullsized":"2024-04-25T11:24:32.878774+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:51.002480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.882725+0000","last_change":"2024-04-25T11:23:46.384203+0000","last_active":"2024-04-25T11:24:32.882725+0000","last_peered":"2024-04-25T11:24:32.882725+0000","last_clean":"2024-04-25T11:24:32.882725+0000","last_became_active":"2024-04-25T11:23:46.383532+0000","last_became_peered":"2024-04-25T11:23:46.383532+0000","last_unstale":"2024-04-25T11:24:32.882725+0000","last_undegraded":"2024-04-25T11:24:32.882725+0000","last_fullsized":"2024-04-25T11:24:32.882725+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:25.265484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.977503+0000","last_change":"2024-04-25T11:23:40.640082+0000","last_active":"2024-04-25T11:24:32.977503+0000","last_peered":"2024-04-25T11:24:32.977503+0000","last_clean":"2024-04-25T11:24:32.977503+0000","last_became_active":"2024-04-25T11:23:40.639725+0000","last_became_peered":"2024-04-25T11:23:40.639725+0000","last_unstale":"2024-04-25T11:24:32.977503+0000","last_undegraded":"2024-04-25T11:24:32.977503+0000","last_fullsized":"2024-04-25T11:24:32.977503+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:01.244116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.877704+0000","last_change":"2024-04-25T11:23:41.645439+0000","last_active":"2024-04-25T11:24:32.877704+0000","last_peered":"2024-04-25T11:24:32.877704+0000","last_clean":"2024-04-25T11:24:32.877704+0000","last_became_active":"2024-04-25T11:23:41.645056+0000","last_became_peered":"2024-04-25T11:23:41.645056+0000","last_unstale":"2024-04-25T11:24:32.877704+0000","last_undegraded":"2024-04-25T11:24:32.877704+0000","last_fullsized":"2024-04-25T11:24:32.877704+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:43.100321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.876787+0000","last_change":"2024-04-25T11:23:45.672174+0000","last_active":"2024-04-25T11:24:32.876787+0000","last_peered":"2024-04-25T11:24:32.876787+0000","last_clean":"2024-04-25T11:24:32.876787+0000","last_became_active":"2024-04-25T11:23:45.670401+0000","last_became_peered":"2024-04-25T11:23:45.670401+0000","last_unstale":"2024-04-25T11:24:32.876787+0000","last_undegraded":"2024-04-25T11:24:32.876787+0000","last_fullsized":"2024-04-25T11:24:32.876787+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:59.259644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"80'5","reported_seq":105,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:25:18.165023+0000","last_change":"2024-04-25T11:23:40.636704+0000","last_active":"2024-04-25T11:25:18.165023+0000","last_peered":"2024-04-25T11:25:18.165023+0000","last_clean":"2024-04-25T11:25:18.165023+0000","last_became_active":"2024-04-25T11:23:40.636319+0000","last_became_peered":"2024-04-25T11:23:40.636319+0000","last_unstale":"2024-04-25T11:25:18.165023+0000","last_undegraded":"2024-04-25T11:25:18.165023+0000","last_fullsized":"2024-04-25T11:25:18.165023+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:17.199946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.877397+0000","last_change":"2024-04-25T11:23:41.660510+0000","last_active":"2024-04-25T11:24:32.877397+0000","last_peered":"2024-04-25T11:24:32.877397+0000","last_clean":"2024-04-25T11:24:32.877397+0000","last_became_active":"2024-04-25T11:23:41.659837+0000","last_became_peered":"2024-04-25T11:23:41.659837+0000","last_unstale":"2024-04-25T11:24:32.877397+0000","last_undegraded":"2024-04-25T11:24:32.877397+0000","last_fullsized":"2024-04-25T11:24:32.877397+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:33.879487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.877227+0000","last_change":"2024-04-25T11:23:46.383239+0000","last_active":"2024-04-25T11:24:32.877227+0000","last_peered":"2024-04-25T11:24:32.877227+0000","last_clean":"2024-04-25T11:24:32.877227+0000","last_became_active":"2024-04-25T11:23:46.382728+0000","last_became_peered":"2024-04-25T11:23:46.382728+0000","last_unstale":"2024-04-25T11:24:32.877227+0000","last_undegraded":"2024-04-25T11:24:32.877227+0000","last_fullsized":"2024-04-25T11:24:32.877227+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:25.087389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.885805+0000","last_change":"2024-04-25T11:23:41.654723+0000","last_active":"2024-04-25T11:24:32.885805+0000","last_peered":"2024-04-25T11:24:32.885805+0000","last_clean":"2024-04-25T11:24:32.885805+0000","last_became_active":"2024-04-25T11:23:41.654140+0000","last_became_peered":"2024-04-25T11:23:41.654140+0000","last_unstale":"2024-04-25T11:24:32.885805+0000","last_undegraded":"2024-04-25T11:24:32.885805+0000","last_fullsized":"2024-04-25T11:24:32.885805+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:41.981728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":106,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:25:18.180870+0000","last_change":"2024-04-25T11:23:40.641902+0000","last_active":"2024-04-25T11:25:18.180870+0000","last_peered":"2024-04-25T11:25:18.180870+0000","last_clean":"2024-04-25T11:25:18.180870+0000","last_became_active":"2024-04-25T11:23:40.641490+0000","last_became_peered":"2024-04-25T11:23:40.641490+0000","last_unstale":"2024-04-25T11:25:18.180870+0000","last_undegraded":"2024-04-25T11:25:18.180870+0000","last_fullsized":"2024-04-25T11:25:18.180870+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:03.939354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"80'1","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.882503+0000","last_change":"2024-04-25T11:23:45.670485+0000","last_active":"2024-04-25T11:24:32.882503+0000","last_peered":"2024-04-25T11:24:32.882503+0000","last_clean":"2024-04-25T11:24:32.882503+0000","last_became_active":"2024-04-25T11:23:45.669842+0000","last_became_peered":"2024-04-25T11:23:45.669842+0000","last_unstale":"2024-04-25T11:24:32.882503+0000","last_undegraded":"2024-04-25T11:24:32.882503+0000","last_fullsized":"2024-04-25T11:24:32.882503+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:32.801217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.880369+0000","last_change":"2024-04-25T11:23:41.640583+0000","last_active":"2024-04-25T11:24:32.880369+0000","last_peered":"2024-04-25T11:24:32.880369+0000","last_clean":"2024-04-25T11:24:32.880369+0000","last_became_active":"2024-04-25T11:23:41.640262+0000","last_became_peered":"2024-04-25T11:23:41.640262+0000","last_unstale":"2024-04-25T11:24:32.880369+0000","last_undegraded":"2024-04-25T11:24:32.880369+0000","last_fullsized":"2024-04-25T11:24:32.880369+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:09.539504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.883467+0000","last_change":"2024-04-25T11:23:40.628424+0000","last_active":"2024-04-25T11:24:32.883467+0000","last_peered":"2024-04-25T11:24:32.883467+0000","last_clean":"2024-04-25T11:24:32.883467+0000","last_became_active":"2024-04-25T11:23:40.628064+0000","last_became_peered":"2024-04-25T11:23:40.628064+0000","last_unstale":"2024-04-25T11:24:32.883467+0000","last_undegraded":"2024-04-25T11:24:32.883467+0000","last_fullsized":"2024-04-25T11:24:32.883467+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:34.837291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":104,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.880968+0000","last_change":"2024-04-25T11:22:25.135774+0000","last_active":"2024-04-25T11:24:32.880968+0000","last_peered":"2024-04-25T11:24:32.880968+0000","last_clean":"2024-04-25T11:24:32.880968+0000","last_became_active":"2024-04-25T11:22:25.131178+0000","last_became_peered":"2024-04-25T11:22:25.131178+0000","last_unstale":"2024-04-25T11:24:32.880968+0000","last_undegraded":"2024-04-25T11:24:32.880968+0000","last_fullsized":"2024-04-25T11:24:32.880968+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:18:57.891013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:18:57.891013+0000","last_clean_scrub_stamp":"2024-04-25T11:18:57.891013+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:29.317685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.886668+0000","last_change":"2024-04-25T11:23:46.385048+0000","last_active":"2024-04-25T11:24:32.886668+0000","last_peered":"2024-04-25T11:24:32.886668+0000","last_clean":"2024-04-25T11:24:32.886668+0000","last_became_active":"2024-04-25T11:23:46.384014+0000","last_became_peered":"2024-04-25T11:23:46.384014+0000","last_unstale":"2024-04-25T11:24:32.886668+0000","last_undegraded":"2024-04-25T11:24:32.886668+0000","last_fullsized":"2024-04-25T11:24:32.886668+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:26.508616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.876446+0000","last_change":"2024-04-25T11:23:41.650783+0000","last_active":"2024-04-25T11:24:32.876446+0000","last_peered":"2024-04-25T11:24:32.876446+0000","last_clean":"2024-04-25T11:24:32.876446+0000","last_became_active":"2024-04-25T11:23:41.650431+0000","last_became_peered":"2024-04-25T11:23:41.650431+0000","last_unstale":"2024-04-25T11:24:32.876446+0000","last_undegraded":"2024-04-25T11:24:32.876446+0000","last_fullsized":"2024-04-25T11:24:32.876446+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:38.925779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.881061+0000","last_change":"2024-04-25T11:23:40.638879+0000","last_active":"2024-04-25T11:24:32.881061+0000","last_peered":"2024-04-25T11:24:32.881061+0000","last_clean":"2024-04-25T11:24:32.881061+0000","last_became_active":"2024-04-25T11:23:40.638475+0000","last_became_peered":"2024-04-25T11:23:40.638475+0000","last_unstale":"2024-04-25T11:24:32.881061+0000","last_undegraded":"2024-04-25T11:24:32.881061+0000","last_fullsized":"2024-04-25T11:24:32.881061+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:25.591831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.878313+0000","last_change":"2024-04-25T11:23:45.679333+0000","last_active":"2024-04-25T11:24:32.878313+0000","last_peered":"2024-04-25T11:24:32.878313+0000","last_clean":"2024-04-25T11:24:32.878313+0000","last_became_active":"2024-04-25T11:23:45.678622+0000","last_became_peered":"2024-04-25T11:23:45.678622+0000","last_unstale":"2024-04-25T11:24:32.878313+0000","last_undegraded":"2024-04-25T11:24:32.878313+0000","last_fullsized":"2024-04-25T11:24:32.878313+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:08.951281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.882094+0000","last_change":"2024-04-25T11:23:40.642975+0000","last_active":"2024-04-25T11:24:32.882094+0000","last_peered":"2024-04-25T11:24:32.882094+0000","last_clean":"2024-04-25T11:24:32.882094+0000","last_became_active":"2024-04-25T11:23:40.642605+0000","last_became_peered":"2024-04-25T11:23:40.642605+0000","last_unstale":"2024-04-25T11:24:32.882094+0000","last_undegraded":"2024-04-25T11:24:32.882094+0000","last_fullsized":"2024-04-25T11:24:32.882094+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:57.665763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.876918+0000","last_change":"2024-04-25T11:23:41.628396+0000","last_active":"2024-04-25T11:24:32.876918+0000","last_peered":"2024-04-25T11:24:32.876918+0000","last_clean":"2024-04-25T11:24:32.876918+0000","last_became_active":"2024-04-25T11:23:41.627979+0000","last_became_peered":"2024-04-25T11:23:41.627979+0000","last_unstale":"2024-04-25T11:24:32.876918+0000","last_undegraded":"2024-04-25T11:24:32.876918+0000","last_fullsized":"2024-04-25T11:24:32.876918+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:48.471219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.879530+0000","last_change":"2024-04-25T11:23:46.384252+0000","last_active":"2024-04-25T11:24:32.879530+0000","last_peered":"2024-04-25T11:24:32.879530+0000","last_clean":"2024-04-25T11:24:32.879530+0000","last_became_active":"2024-04-25T11:23:46.383408+0000","last_became_peered":"2024-04-25T11:23:46.383408+0000","last_unstale":"2024-04-25T11:24:32.879530+0000","last_undegraded":"2024-04-25T11:24:32.879530+0000","last_fullsized":"2024-04-25T11:24:32.879530+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:19.337043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"80'12","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.977559+0000","last_change":"2024-04-25T11:23:40.643767+0000","last_active":"2024-04-25T11:24:32.977559+0000","last_peered":"2024-04-25T11:24:32.977559+0000","last_clean":"2024-04-25T11:24:32.977559+0000","last_became_active":"2024-04-25T11:23:40.642748+0000","last_became_peered":"2024-04-25T11:23:40.642748+0000","last_unstale":"2024-04-25T11:24:32.977559+0000","last_undegraded":"2024-04-25T11:24:32.977559+0000","last_fullsized":"2024-04-25T11:24:32.977559+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:12.322055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.880652+0000","last_change":"2024-04-25T11:23:41.642766+0000","last_active":"2024-04-25T11:24:32.880652+0000","last_peered":"2024-04-25T11:24:32.880652+0000","last_clean":"2024-04-25T11:24:32.880652+0000","last_became_active":"2024-04-25T11:23:41.642409+0000","last_became_peered":"2024-04-25T11:23:41.642409+0000","last_unstale":"2024-04-25T11:24:32.880652+0000","last_undegraded":"2024-04-25T11:24:32.880652+0000","last_fullsized":"2024-04-25T11:24:32.880652+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:36.139574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.882041+0000","last_change":"2024-04-25T11:23:45.670926+0000","last_active":"2024-04-25T11:24:32.882041+0000","last_peered":"2024-04-25T11:24:32.882041+0000","last_clean":"2024-04-25T11:24:32.882041+0000","last_became_active":"2024-04-25T11:23:45.670444+0000","last_became_peered":"2024-04-25T11:23:45.670444+0000","last_unstale":"2024-04-25T11:24:32.882041+0000","last_undegraded":"2024-04-25T11:24:32.882041+0000","last_fullsized":"2024-04-25T11:24:32.882041+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:40.538155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.884970+0000","last_change":"2024-04-25T11:23:40.647132+0000","last_active":"2024-04-25T11:24:32.884970+0000","last_peered":"2024-04-25T11:24:32.884970+0000","last_clean":"2024-04-25T11:24:32.884970+0000","last_became_active":"2024-04-25T11:23:40.646792+0000","last_became_peered":"2024-04-25T11:23:40.646792+0000","last_unstale":"2024-04-25T11:24:32.884970+0000","last_undegraded":"2024-04-25T11:24:32.884970+0000","last_fullsized":"2024-04-25T11:24:32.884970+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:50.935908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.882364+0000","last_change":"2024-04-25T11:23:41.658368+0000","last_active":"2024-04-25T11:24:32.882364+0000","last_peered":"2024-04-25T11:24:32.882364+0000","last_clean":"2024-04-25T11:24:32.882364+0000","last_became_active":"2024-04-25T11:23:41.657972+0000","last_became_peered":"2024-04-25T11:23:41.657972+0000","last_unstale":"2024-04-25T11:24:32.882364+0000","last_undegraded":"2024-04-25T11:24:32.882364+0000","last_fullsized":"2024-04-25T11:24:32.882364+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:02.717294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.884235+0000","last_change":"2024-04-25T11:23:46.384683+0000","last_active":"2024-04-25T11:24:32.884235+0000","last_peered":"2024-04-25T11:24:32.884235+0000","last_clean":"2024-04-25T11:24:32.884235+0000","last_became_active":"2024-04-25T11:23:46.384230+0000","last_became_peered":"2024-04-25T11:23:46.384230+0000","last_unstale":"2024-04-25T11:24:32.884235+0000","last_undegraded":"2024-04-25T11:24:32.884235+0000","last_fullsized":"2024-04-25T11:24:32.884235+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:06.982061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.882212+0000","last_change":"2024-04-25T11:23:40.642289+0000","last_active":"2024-04-25T11:24:32.882212+0000","last_peered":"2024-04-25T11:24:32.882212+0000","last_clean":"2024-04-25T11:24:32.882212+0000","last_became_active":"2024-04-25T11:23:40.641948+0000","last_became_peered":"2024-04-25T11:23:40.641948+0000","last_unstale":"2024-04-25T11:24:32.882212+0000","last_undegraded":"2024-04-25T11:24:32.882212+0000","last_fullsized":"2024-04-25T11:24:32.882212+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:39.265090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.880445+0000","last_change":"2024-04-25T11:23:41.651792+0000","last_active":"2024-04-25T11:24:32.880445+0000","last_peered":"2024-04-25T11:24:32.880445+0000","last_clean":"2024-04-25T11:24:32.880445+0000","last_became_active":"2024-04-25T11:23:41.651413+0000","last_became_peered":"2024-04-25T11:23:41.651413+0000","last_unstale":"2024-04-25T11:24:32.880445+0000","last_undegraded":"2024-04-25T11:24:32.880445+0000","last_fullsized":"2024-04-25T11:24:32.880445+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:50.805600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.977511+0000","last_change":"2024-04-25T11:23:45.670509+0000","last_active":"2024-04-25T11:24:32.977511+0000","last_peered":"2024-04-25T11:24:32.977511+0000","last_clean":"2024-04-25T11:24:32.977511+0000","last_became_active":"2024-04-25T11:23:45.670145+0000","last_became_peered":"2024-04-25T11:23:45.670145+0000","last_unstale":"2024-04-25T11:24:32.977511+0000","last_undegraded":"2024-04-25T11:24:32.977511+0000","last_fullsized":"2024-04-25T11:24:32.977511+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:03.764278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.879963+0000","last_change":"2024-04-25T11:23:40.644123+0000","last_active":"2024-04-25T11:24:32.879963+0000","last_peered":"2024-04-25T11:24:32.879963+0000","last_clean":"2024-04-25T11:24:32.879963+0000","last_became_active":"2024-04-25T11:23:40.643750+0000","last_became_peered":"2024-04-25T11:23:40.643750+0000","last_unstale":"2024-04-25T11:24:32.879963+0000","last_undegraded":"2024-04-25T11:24:32.879963+0000","last_fullsized":"2024-04-25T11:24:32.879963+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:49.613669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.878031+0000","last_change":"2024-04-25T11:23:41.643606+0000","last_active":"2024-04-25T11:24:32.878031+0000","last_peered":"2024-04-25T11:24:32.878031+0000","last_clean":"2024-04-25T11:24:32.878031+0000","last_became_active":"2024-04-25T11:23:41.643248+0000","last_became_peered":"2024-04-25T11:23:41.643248+0000","last_unstale":"2024-04-25T11:24:32.878031+0000","last_undegraded":"2024-04-25T11:24:32.878031+0000","last_fullsized":"2024-04-25T11:24:32.878031+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:46.200959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"80'1","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.881044+0000","last_change":"2024-04-25T11:23:45.674138+0000","last_active":"2024-04-25T11:24:32.881044+0000","last_peered":"2024-04-25T11:24:32.881044+0000","last_clean":"2024-04-25T11:24:32.881044+0000","last_became_active":"2024-04-25T11:23:45.672549+0000","last_became_peered":"2024-04-25T11:23:45.672549+0000","last_unstale":"2024-04-25T11:24:32.881044+0000","last_undegraded":"2024-04-25T11:24:32.881044+0000","last_fullsized":"2024-04-25T11:24:32.881044+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:11.345176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.881426+0000","last_change":"2024-04-25T11:23:40.636602+0000","last_active":"2024-04-25T11:24:32.881426+0000","last_peered":"2024-04-25T11:24:32.881426+0000","last_clean":"2024-04-25T11:24:32.881426+0000","last_became_active":"2024-04-25T11:23:40.636187+0000","last_became_peered":"2024-04-25T11:23:40.636187+0000","last_unstale":"2024-04-25T11:24:32.881426+0000","last_undegraded":"2024-04-25T11:24:32.881426+0000","last_fullsized":"2024-04-25T11:24:32.881426+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:15.894714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.880078+0000","last_change":"2024-04-25T11:23:41.647345+0000","last_active":"2024-04-25T11:24:32.880078+0000","last_peered":"2024-04-25T11:24:32.880078+0000","last_clean":"2024-04-25T11:24:32.880078+0000","last_became_active":"2024-04-25T11:23:41.646981+0000","last_became_peered":"2024-04-25T11:23:41.646981+0000","last_unstale":"2024-04-25T11:24:32.880078+0000","last_undegraded":"2024-04-25T11:24:32.880078+0000","last_fullsized":"2024-04-25T11:24:32.880078+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:30.096856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.884216+0000","last_change":"2024-04-25T11:23:46.384132+0000","last_active":"2024-04-25T11:24:32.884216+0000","last_peered":"2024-04-25T11:24:32.884216+0000","last_clean":"2024-04-25T11:24:32.884216+0000","last_became_active":"2024-04-25T11:23:46.383501+0000","last_became_peered":"2024-04-25T11:23:46.383501+0000","last_unstale":"2024-04-25T11:24:32.884216+0000","last_undegraded":"2024-04-25T11:24:32.884216+0000","last_fullsized":"2024-04-25T11:24:32.884216+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:22.486326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.880575+0000","last_change":"2024-04-25T11:23:40.634942+0000","last_active":"2024-04-25T11:24:32.880575+0000","last_peered":"2024-04-25T11:24:32.880575+0000","last_clean":"2024-04-25T11:24:32.880575+0000","last_became_active":"2024-04-25T11:23:40.634557+0000","last_became_peered":"2024-04-25T11:23:40.634557+0000","last_unstale":"2024-04-25T11:24:32.880575+0000","last_undegraded":"2024-04-25T11:24:32.880575+0000","last_fullsized":"2024-04-25T11:24:32.880575+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:08.612328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.883122+0000","last_change":"2024-04-25T11:23:41.648985+0000","last_active":"2024-04-25T11:24:32.883122+0000","last_peered":"2024-04-25T11:24:32.883122+0000","last_clean":"2024-04-25T11:24:32.883122+0000","last_became_active":"2024-04-25T11:23:41.645643+0000","last_became_peered":"2024-04-25T11:23:41.645643+0000","last_unstale":"2024-04-25T11:24:32.883122+0000","last_undegraded":"2024-04-25T11:24:32.883122+0000","last_fullsized":"2024-04-25T11:24:32.883122+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:22.289698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.882381+0000","last_change":"2024-04-25T11:23:45.671474+0000","last_active":"2024-04-25T11:24:32.882381+0000","last_peered":"2024-04-25T11:24:32.882381+0000","last_clean":"2024-04-25T11:24:32.882381+0000","last_became_active":"2024-04-25T11:23:45.670976+0000","last_became_peered":"2024-04-25T11:23:45.670976+0000","last_unstale":"2024-04-25T11:24:32.882381+0000","last_undegraded":"2024-04-25T11:24:32.882381+0000","last_fullsized":"2024-04-25T11:24:32.882381+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:13:29.771087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"80'6","reported_seq":107,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:25:16.168298+0000","last_change":"2024-04-25T11:23:40.647950+0000","last_active":"2024-04-25T11:25:16.168298+0000","last_peered":"2024-04-25T11:25:16.168298+0000","last_clean":"2024-04-25T11:25:16.168298+0000","last_became_active":"2024-04-25T11:23:40.647549+0000","last_became_peered":"2024-04-25T11:23:40.647549+0000","last_unstale":"2024-04-25T11:25:16.168298+0000","last_undegraded":"2024-04-25T11:25:16.168298+0000","last_fullsized":"2024-04-25T11:25:16.168298+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:47.975294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":73,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.882177+0000","last_change":"2024-04-25T11:23:41.647670+0000","last_active":"2024-04-25T11:24:32.882177+0000","last_peered":"2024-04-25T11:24:32.882177+0000","last_clean":"2024-04-25T11:24:32.882177+0000","last_became_active":"2024-04-25T11:23:41.646686+0000","last_became_peered":"2024-04-25T11:23:41.646686+0000","last_unstale":"2024-04-25T11:24:32.882177+0000","last_undegraded":"2024-04-25T11:24:32.882177+0000","last_fullsized":"2024-04-25T11:24:32.882177+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:21.260685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.888278+0000","last_change":"2024-04-25T11:24:32.888278+0000","last_active":"2024-04-25T11:24:32.888278+0000","last_peered":"2024-04-25T11:24:32.888278+0000","last_clean":"2024-04-25T11:24:32.888278+0000","last_became_active":"2024-04-25T11:24:32.887747+0000","last_became_peered":"2024-04-25T11:24:32.887747+0000","last_unstale":"2024-04-25T11:24:32.888278+0000","last_undegraded":"2024-04-25T11:24:32.888278+0000","last_fullsized":"2024-04-25T11:24:32.888278+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:03.960560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,10,1,4],"acting":[0,10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.886312+0000","last_change":"2024-04-25T11:23:40.642974+0000","last_active":"2024-04-25T11:24:32.886312+0000","last_peered":"2024-04-25T11:24:32.886312+0000","last_clean":"2024-04-25T11:24:32.886312+0000","last_became_active":"2024-04-25T11:23:40.642628+0000","last_became_peered":"2024-04-25T11:23:40.642628+0000","last_unstale":"2024-04-25T11:24:32.886312+0000","last_undegraded":"2024-04-25T11:24:32.886312+0000","last_fullsized":"2024-04-25T11:24:32.886312+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:03.870867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.882406+0000","last_change":"2024-04-25T11:23:41.643519+0000","last_active":"2024-04-25T11:24:32.882406+0000","last_peered":"2024-04-25T11:24:32.882406+0000","last_clean":"2024-04-25T11:24:32.882406+0000","last_became_active":"2024-04-25T11:23:41.642621+0000","last_became_peered":"2024-04-25T11:23:41.642621+0000","last_unstale":"2024-04-25T11:24:32.882406+0000","last_undegraded":"2024-04-25T11:24:32.882406+0000","last_fullsized":"2024-04-25T11:24:32.882406+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:38.699914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.884821+0000","last_change":"2024-04-25T11:23:45.671017+0000","last_active":"2024-04-25T11:24:32.884821+0000","last_peered":"2024-04-25T11:24:32.884821+0000","last_clean":"2024-04-25T11:24:32.884821+0000","last_became_active":"2024-04-25T11:23:45.670603+0000","last_became_peered":"2024-04-25T11:23:45.670603+0000","last_unstale":"2024-04-25T11:24:32.884821+0000","last_undegraded":"2024-04-25T11:24:32.884821+0000","last_fullsized":"2024-04-25T11:24:32.884821+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:32.385176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.891324+0000","last_change":"2024-04-25T11:24:32.891324+0000","last_active":"2024-04-25T11:24:32.891324+0000","last_peered":"2024-04-25T11:24:32.891324+0000","last_clean":"2024-04-25T11:24:32.891324+0000","last_became_active":"2024-04-25T11:24:32.885073+0000","last_became_peered":"2024-04-25T11:24:32.885073+0000","last_unstale":"2024-04-25T11:24:32.891324+0000","last_undegraded":"2024-04-25T11:24:32.891324+0000","last_fullsized":"2024-04-25T11:24:32.891324+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:38.326741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.881613+0000","last_change":"2024-04-25T11:23:41.634946+0000","last_active":"2024-04-25T11:24:32.881613+0000","last_peered":"2024-04-25T11:24:32.881613+0000","last_clean":"2024-04-25T11:24:32.881613+0000","last_became_active":"2024-04-25T11:23:41.634505+0000","last_became_peered":"2024-04-25T11:23:41.634505+0000","last_unstale":"2024-04-25T11:24:32.881613+0000","last_undegraded":"2024-04-25T11:24:32.881613+0000","last_fullsized":"2024-04-25T11:24:32.881613+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:52.793207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.878128+0000","last_change":"2024-04-25T11:23:41.642079+0000","last_active":"2024-04-25T11:24:32.878128+0000","last_peered":"2024-04-25T11:24:32.878128+0000","last_clean":"2024-04-25T11:24:32.878128+0000","last_became_active":"2024-04-25T11:23:41.641721+0000","last_became_peered":"2024-04-25T11:23:41.641721+0000","last_unstale":"2024-04-25T11:24:32.878128+0000","last_undegraded":"2024-04-25T11:24:32.878128+0000","last_fullsized":"2024-04-25T11:24:32.878128+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:59.968749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.891176+0000","last_change":"2024-04-25T11:24:32.891176+0000","last_active":"2024-04-25T11:24:32.891176+0000","last_peered":"2024-04-25T11:24:32.891176+0000","last_clean":"2024-04-25T11:24:32.891176+0000","last_became_active":"2024-04-25T11:24:32.890758+0000","last_became_peered":"2024-04-25T11:24:32.890758+0000","last_unstale":"2024-04-25T11:24:32.891176+0000","last_undegraded":"2024-04-25T11:24:32.891176+0000","last_fullsized":"2024-04-25T11:24:32.891176+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:25.871589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.881233+0000","last_change":"2024-04-25T11:23:41.641108+0000","last_active":"2024-04-25T11:24:32.881233+0000","last_peered":"2024-04-25T11:24:32.881233+0000","last_clean":"2024-04-25T11:24:32.881233+0000","last_became_active":"2024-04-25T11:23:41.640665+0000","last_became_peered":"2024-04-25T11:23:41.640665+0000","last_unstale":"2024-04-25T11:24:32.881233+0000","last_undegraded":"2024-04-25T11:24:32.881233+0000","last_fullsized":"2024-04-25T11:24:32.881233+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:48.075466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.882690+0000","last_change":"2024-04-25T11:23:41.652322+0000","last_active":"2024-04-25T11:24:32.882690+0000","last_peered":"2024-04-25T11:24:32.882690+0000","last_clean":"2024-04-25T11:24:32.882690+0000","last_became_active":"2024-04-25T11:23:41.651681+0000","last_became_peered":"2024-04-25T11:23:41.651681+0000","last_unstale":"2024-04-25T11:24:32.882690+0000","last_undegraded":"2024-04-25T11:24:32.882690+0000","last_fullsized":"2024-04-25T11:24:32.882690+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:17.574861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.877034+0000","last_change":"2024-04-25T11:23:41.651701+0000","last_active":"2024-04-25T11:24:32.877034+0000","last_peered":"2024-04-25T11:24:32.877034+0000","last_clean":"2024-04-25T11:24:32.877034+0000","last_became_active":"2024-04-25T11:23:41.651362+0000","last_became_peered":"2024-04-25T11:23:41.651362+0000","last_unstale":"2024-04-25T11:24:32.877034+0000","last_undegraded":"2024-04-25T11:24:32.877034+0000","last_fullsized":"2024-04-25T11:24:32.877034+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:50.777396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.878668+0000","last_change":"2024-04-25T11:23:41.640385+0000","last_active":"2024-04-25T11:24:32.878668+0000","last_peered":"2024-04-25T11:24:32.878668+0000","last_clean":"2024-04-25T11:24:32.878668+0000","last_became_active":"2024-04-25T11:23:41.639986+0000","last_became_peered":"2024-04-25T11:23:41.639986+0000","last_unstale":"2024-04-25T11:24:32.878668+0000","last_undegraded":"2024-04-25T11:24:32.878668+0000","last_fullsized":"2024-04-25T11:24:32.878668+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:13.376858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.882697+0000","last_change":"2024-04-25T11:23:41.648075+0000","last_active":"2024-04-25T11:24:32.882697+0000","last_peered":"2024-04-25T11:24:32.882697+0000","last_clean":"2024-04-25T11:24:32.882697+0000","last_became_active":"2024-04-25T11:23:41.647681+0000","last_became_peered":"2024-04-25T11:23:41.647681+0000","last_unstale":"2024-04-25T11:24:32.882697+0000","last_undegraded":"2024-04-25T11:24:32.882697+0000","last_fullsized":"2024-04-25T11:24:32.882697+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:44.916762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.880753+0000","last_change":"2024-04-25T11:23:41.641788+0000","last_active":"2024-04-25T11:24:32.880753+0000","last_peered":"2024-04-25T11:24:32.880753+0000","last_clean":"2024-04-25T11:24:32.880753+0000","last_became_active":"2024-04-25T11:23:41.641407+0000","last_became_peered":"2024-04-25T11:23:41.641407+0000","last_unstale":"2024-04-25T11:24:32.880753+0000","last_undegraded":"2024-04-25T11:24:32.880753+0000","last_fullsized":"2024-04-25T11:24:32.880753+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:33.609749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.885751+0000","last_change":"2024-04-25T11:23:41.647616+0000","last_active":"2024-04-25T11:24:32.885751+0000","last_peered":"2024-04-25T11:24:32.885751+0000","last_clean":"2024-04-25T11:24:32.885751+0000","last_became_active":"2024-04-25T11:23:41.645396+0000","last_became_peered":"2024-04-25T11:23:41.645396+0000","last_unstale":"2024-04-25T11:24:32.885751+0000","last_undegraded":"2024-04-25T11:24:32.885751+0000","last_fullsized":"2024-04-25T11:24:32.885751+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:47.842574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.880114+0000","last_change":"2024-04-25T11:23:41.638407+0000","last_active":"2024-04-25T11:24:32.880114+0000","last_peered":"2024-04-25T11:24:32.880114+0000","last_clean":"2024-04-25T11:24:32.880114+0000","last_became_active":"2024-04-25T11:23:41.638054+0000","last_became_peered":"2024-04-25T11:23:41.638054+0000","last_unstale":"2024-04-25T11:24:32.880114+0000","last_undegraded":"2024-04-25T11:24:32.880114+0000","last_fullsized":"2024-04-25T11:24:32.880114+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:34.306493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.882645+0000","last_change":"2024-04-25T11:23:41.650140+0000","last_active":"2024-04-25T11:24:32.882645+0000","last_peered":"2024-04-25T11:24:32.882645+0000","last_clean":"2024-04-25T11:24:32.882645+0000","last_became_active":"2024-04-25T11:23:41.649487+0000","last_became_peered":"2024-04-25T11:23:41.649487+0000","last_unstale":"2024-04-25T11:24:32.882645+0000","last_undegraded":"2024-04-25T11:24:32.882645+0000","last_fullsized":"2024-04-25T11:24:32.882645+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:55.652921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.876863+0000","last_change":"2024-04-25T11:23:41.640253+0000","last_active":"2024-04-25T11:24:32.876863+0000","last_peered":"2024-04-25T11:24:32.876863+0000","last_clean":"2024-04-25T11:24:32.876863+0000","last_became_active":"2024-04-25T11:23:41.639926+0000","last_became_peered":"2024-04-25T11:23:41.639926+0000","last_unstale":"2024-04-25T11:24:32.876863+0000","last_undegraded":"2024-04-25T11:24:32.876863+0000","last_fullsized":"2024-04-25T11:24:32.876863+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:01.407198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.878973+0000","last_change":"2024-04-25T11:23:41.653244+0000","last_active":"2024-04-25T11:24:32.878973+0000","last_peered":"2024-04-25T11:24:32.878973+0000","last_clean":"2024-04-25T11:24:32.878973+0000","last_became_active":"2024-04-25T11:23:41.652875+0000","last_became_peered":"2024-04-25T11:23:41.652875+0000","last_unstale":"2024-04-25T11:24:32.878973+0000","last_undegraded":"2024-04-25T11:24:32.878973+0000","last_fullsized":"2024-04-25T11:24:32.878973+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:06.897766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68394,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":719,"num_read_kb":445,"num_write":241,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1707,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":205182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710754,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":458067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906982,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1626112,"data_stored":1104087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070508,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":962560,"data_stored":460107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266737,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":950272,"data_stored":455807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462965,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28536,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716616,"statfs":{"total":95995035648,"available":95965814784,"internally_reserved":0,"allocated":1552384,"data_stored":1052886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626492,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":933888,"data_stored":451515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":513735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":958464,"data_stored":453362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":514625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182474,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":946176,"data_stored":451223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986246,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":455701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822720,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36860,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708292,"statfs":{"total":95995035648,"available":95957291008,"internally_reserved":0,"allocated":1552384,"data_stored":1047455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:25:23.583 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T11:25:23.824 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:25:23.942 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:23 smithi069 ceph-mon[29188]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T11:25:24.098 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:23 smithi146 ceph-mon[32821]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T11:25:24.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:23 smithi152 ceph-mon[32701]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T11:25:25.098 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:24 smithi146 ceph-mon[32821]: pgmap v338: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:25:25.103 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:24 smithi069 ceph-mon[29188]: pgmap v338: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:25:25.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:24 smithi152 ceph-mon[32701]: pgmap v338: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:25:25.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:25.795+0000 7fa2c48ae700 1 -- 172.21.15.69:0/2235257911 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2c0104460 msgr2=0x7fa2c0104870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:25.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:25.795+0000 7fa2c48ae700 1 --2- 172.21.15.69:0/2235257911 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2c0104460 0x7fa2c0104870 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa2b800b0d0 tx=0x7fa2b800b3e0 comp rx=0 tx=0).stop 2024-04-25T11:25:25.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:25.795+0000 7fa2c48ae700 1 -- 172.21.15.69:0/2235257911 shutdown_connections 2024-04-25T11:25:25.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:25.795+0000 7fa2c48ae700 1 --2- 172.21.15.69:0/2235257911 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa2c01096e0 0x7fa2c0111040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:25.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:25.795+0000 7fa2c48ae700 1 --2- 172.21.15.69:0/2235257911 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2c0104460 0x7fa2c0104870 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:25.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:25.795+0000 7fa2c48ae700 1 --2- 172.21.15.69:0/2235257911 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2c0108d90 0x7fa2c01091a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:25.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:25.795+0000 7fa2c48ae700 1 -- 172.21.15.69:0/2235257911 >> 172.21.15.69:0/2235257911 conn(0x7fa2c00ffd40 msgr2=0x7fa2c0102160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:25.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:25.795+0000 7fa2c48ae700 1 -- 172.21.15.69:0/2235257911 shutdown_connections 2024-04-25T11:25:25.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:25.795+0000 7fa2c48ae700 1 -- 172.21.15.69:0/2235257911 wait complete. 2024-04-25T11:25:25.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:25.796+0000 7fa2c48ae700 1 Processor -- start 2024-04-25T11:25:25.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:25.796+0000 7fa2c48ae700 1 -- start start 2024-04-25T11:25:25.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:25.796+0000 7fa2c48ae700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2c0104460 0x7fa2c01a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:25.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:25.796+0000 7fa2c48ae700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa2c0108d90 0x7fa2c01a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:25.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:25.796+0000 7fa2c48ae700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2c01096e0 0x7fa2c019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:25.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:25.796+0000 7fa2c48ae700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c01141c0 con 0x7fa2c01096e0 2024-04-25T11:25:25.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:25.796+0000 7fa2c48ae700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c0114040 con 0x7fa2c0104460 2024-04-25T11:25:25.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:25.796+0000 7fa2c48ae700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c0114340 con 0x7fa2c0108d90 2024-04-25T11:25:25.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:25.797+0000 7fa2bed9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2c01096e0 0x7fa2c019aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:25.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:25.797+0000 7fa2be59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2c0104460 0x7fa2c01a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:25.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:25.797+0000 7fa2be59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2c0104460 0x7fa2c01a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.69:60524/0 (socket says 172.21.15.69:60524) 2024-04-25T11:25:25.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:25.797+0000 7fa2be59c700 1 -- 172.21.15.69:0/2763224782 learned_addr learned my addr 172.21.15.69:0/2763224782 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:25.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:25.797+0000 7fa2bdd9b700 1 --2- 172.21.15.69:0/2763224782 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa2c0108d90 0x7fa2c01a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:25.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:25.797+0000 7fa2bed9d700 1 -- 172.21.15.69:0/2763224782 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa2c0108d90 msgr2=0x7fa2c01a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:25.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:25.797+0000 7fa2bed9d700 1 --2- 172.21.15.69:0/2763224782 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa2c0108d90 0x7fa2c01a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:25.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:25.797+0000 7fa2bed9d700 1 -- 172.21.15.69:0/2763224782 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2c0104460 msgr2=0x7fa2c01a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:25.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:25.797+0000 7fa2bed9d700 1 --2- 172.21.15.69:0/2763224782 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2c0104460 0x7fa2c01a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:25.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:25.797+0000 7fa2bed9d700 1 -- 172.21.15.69:0/2763224782 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c019b0e0 con 0x7fa2c01096e0 2024-04-25T11:25:25.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:25.797+0000 7fa2bed9d700 1 --2- 172.21.15.69:0/2763224782 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2c01096e0 0x7fa2c019aaa0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa2b000c970 tx=0x7fa2b000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:25.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:25.797+0000 7fa2af7fe700 1 -- 172.21.15.69:0/2763224782 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b000e880 con 0x7fa2c01096e0 2024-04-25T11:25:25.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:25.797+0000 7fa2c48ae700 1 -- 172.21.15.69:0/2763224782 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2c019b3d0 con 0x7fa2c01096e0 2024-04-25T11:25:25.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:25.797+0000 7fa2af7fe700 1 -- 172.21.15.69:0/2763224782 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa2b000f450 con 0x7fa2c01096e0 2024-04-25T11:25:25.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:25.797+0000 7fa2af7fe700 1 -- 172.21.15.69:0/2763224782 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b0018660 con 0x7fa2c01096e0 2024-04-25T11:25:25.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:25.797+0000 7fa2c48ae700 1 -- 172.21.15.69:0/2763224782 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2c019b8c0 con 0x7fa2c01096e0 2024-04-25T11:25:25.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:25.798+0000 7fa2c48ae700 1 -- 172.21.15.69:0/2763224782 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2c0051190 con 0x7fa2c01096e0 2024-04-25T11:25:25.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:25.801+0000 7fa2af7fe700 1 -- 172.21.15.69:0/2763224782 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fa2b000ea20 con 0x7fa2c01096e0 2024-04-25T11:25:25.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:25.801+0000 7fa2af7fe700 1 --2- 172.21.15.69:0/2763224782 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fa2a80afc10 0x7fa2a80b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:25.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:25.802+0000 7fa2be59c700 1 --2- 172.21.15.69:0/2763224782 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fa2a80afc10 0x7fa2a80b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:25.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:25.802+0000 7fa2af7fe700 1 -- 172.21.15.69:0/2763224782 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7fa2b00df090 con 0x7fa2c01096e0 2024-04-25T11:25:25.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:25.802+0000 7fa2be59c700 1 --2- 172.21.15.69:0/2763224782 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fa2a80afc10 0x7fa2a80b2120 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8009900 tx=0x7fa2b800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:25.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:25.805+0000 7fa2af7fe700 1 -- 172.21.15.69:0/2763224782 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa2b0053ff0 con 0x7fa2c01096e0 2024-04-25T11:25:25.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:25.999+0000 7fa2c48ae700 1 -- 172.21.15.69:0/2763224782 --> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa2c0063ad0 con 0x7fa2a80afc10 2024-04-25T11:25:26.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:26.008+0000 7fa2af7fe700 1 -- 172.21.15.69:0/2763224782 <== mgr.14152 v2:172.21.15.69:6800/4220065409 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340398 (secure 0 0 0) 0x7fa2c0063ad0 con 0x7fa2a80afc10 2024-04-25T11:25:26.008 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:25:26.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:26.011+0000 7fa2c48ae700 1 -- 172.21.15.69:0/2763224782 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fa2a80afc10 msgr2=0x7fa2a80b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:26.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:26.011+0000 7fa2c48ae700 1 --2- 172.21.15.69:0/2763224782 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fa2a80afc10 0x7fa2a80b2120 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8009900 tx=0x7fa2b800b040 comp rx=0 tx=0).stop 2024-04-25T11:25:26.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:26.011+0000 7fa2c48ae700 1 -- 172.21.15.69:0/2763224782 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2c01096e0 msgr2=0x7fa2c019aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:26.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:26.011+0000 7fa2c48ae700 1 --2- 172.21.15.69:0/2763224782 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2c01096e0 0x7fa2c019aaa0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa2b000c970 tx=0x7fa2b000cc80 comp rx=0 tx=0).stop 2024-04-25T11:25:26.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:26.011+0000 7fa2c48ae700 1 -- 172.21.15.69:0/2763224782 shutdown_connections 2024-04-25T11:25:26.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:26.011+0000 7fa2c48ae700 1 --2- 172.21.15.69:0/2763224782 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fa2a80afc10 0x7fa2a80b2120 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:26.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:26.011+0000 7fa2c48ae700 1 --2- 172.21.15.69:0/2763224782 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2c01096e0 0x7fa2c019aaa0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:26.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:26.011+0000 7fa2c48ae700 1 --2- 172.21.15.69:0/2763224782 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2c0104460 0x7fa2c01a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:26.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:26.011+0000 7fa2c48ae700 1 --2- 172.21.15.69:0/2763224782 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa2c0108d90 0x7fa2c01a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:26.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:26.011+0000 7fa2c48ae700 1 -- 172.21.15.69:0/2763224782 >> 172.21.15.69:0/2763224782 conn(0x7fa2c00ffd40 msgr2=0x7fa2c0102160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:26.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:26.011+0000 7fa2c48ae700 1 -- 172.21.15.69:0/2763224782 shutdown_connections 2024-04-25T11:25:26.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:26.011+0000 7fa2c48ae700 1 -- 172.21.15.69:0/2763224782 wait complete. 2024-04-25T11:25:26.014 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-04-25T11:25:26.825 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":339,"stamp":"2024-04-25T11:25:25.809989+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658996,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":840,"num_read_kb":534,"num_write":366,"num_write_kb":2290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":2953547,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366472,"kb_used_data":13076,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575352,"statfs":{"total":1151940427776,"available":1151565160448,"internally_reserved":0,"allocated":13389824,"data_stored":7418570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":140,"num_read_kb":75,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004006"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.879610+0000","last_change":"2024-04-25T11:23:41.646759+0000","last_active":"2024-04-25T11:24:32.879610+0000","last_peered":"2024-04-25T11:24:32.879610+0000","last_clean":"2024-04-25T11:24:32.879610+0000","last_became_active":"2024-04-25T11:23:41.645882+0000","last_became_peered":"2024-04-25T11:23:41.645882+0000","last_unstale":"2024-04-25T11:24:32.879610+0000","last_undegraded":"2024-04-25T11:24:32.879610+0000","last_fullsized":"2024-04-25T11:24:32.879610+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:05.776414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.882273+0000","last_change":"2024-04-25T11:23:41.648561+0000","last_active":"2024-04-25T11:24:32.882273+0000","last_peered":"2024-04-25T11:24:32.882273+0000","last_clean":"2024-04-25T11:24:32.882273+0000","last_became_active":"2024-04-25T11:23:41.647735+0000","last_became_peered":"2024-04-25T11:23:41.647735+0000","last_unstale":"2024-04-25T11:24:32.882273+0000","last_undegraded":"2024-04-25T11:24:32.882273+0000","last_fullsized":"2024-04-25T11:24:32.882273+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:20.303934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.881497+0000","last_change":"2024-04-25T11:23:41.646656+0000","last_active":"2024-04-25T11:24:32.881497+0000","last_peered":"2024-04-25T11:24:32.881497+0000","last_clean":"2024-04-25T11:24:32.881497+0000","last_became_active":"2024-04-25T11:23:41.645487+0000","last_became_peered":"2024-04-25T11:23:41.645487+0000","last_unstale":"2024-04-25T11:24:32.881497+0000","last_undegraded":"2024-04-25T11:24:32.881497+0000","last_fullsized":"2024-04-25T11:24:32.881497+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:32.747478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.884452+0000","last_change":"2024-04-25T11:23:41.651994+0000","last_active":"2024-04-25T11:24:32.884452+0000","last_peered":"2024-04-25T11:24:32.884452+0000","last_clean":"2024-04-25T11:24:32.884452+0000","last_became_active":"2024-04-25T11:23:41.651640+0000","last_became_peered":"2024-04-25T11:23:41.651640+0000","last_unstale":"2024-04-25T11:24:32.884452+0000","last_undegraded":"2024-04-25T11:24:32.884452+0000","last_fullsized":"2024-04-25T11:24:32.884452+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:09.376398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.880729+0000","last_change":"2024-04-25T11:23:41.650954+0000","last_active":"2024-04-25T11:24:32.880729+0000","last_peered":"2024-04-25T11:24:32.880729+0000","last_clean":"2024-04-25T11:24:32.880729+0000","last_became_active":"2024-04-25T11:23:41.650553+0000","last_became_peered":"2024-04-25T11:23:41.650553+0000","last_unstale":"2024-04-25T11:24:32.880729+0000","last_undegraded":"2024-04-25T11:24:32.880729+0000","last_fullsized":"2024-04-25T11:24:32.880729+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:52:34.348370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.878125+0000","last_change":"2024-04-25T11:23:41.650770+0000","last_active":"2024-04-25T11:24:32.878125+0000","last_peered":"2024-04-25T11:24:32.878125+0000","last_clean":"2024-04-25T11:24:32.878125+0000","last_became_active":"2024-04-25T11:23:41.649794+0000","last_became_peered":"2024-04-25T11:23:41.649794+0000","last_unstale":"2024-04-25T11:24:32.878125+0000","last_undegraded":"2024-04-25T11:24:32.878125+0000","last_fullsized":"2024-04-25T11:24:32.878125+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:04.032001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.881238+0000","last_change":"2024-04-25T11:23:41.643753+0000","last_active":"2024-04-25T11:24:32.881238+0000","last_peered":"2024-04-25T11:24:32.881238+0000","last_clean":"2024-04-25T11:24:32.881238+0000","last_became_active":"2024-04-25T11:23:41.642854+0000","last_became_peered":"2024-04-25T11:23:41.642854+0000","last_unstale":"2024-04-25T11:24:32.881238+0000","last_undegraded":"2024-04-25T11:24:32.881238+0000","last_fullsized":"2024-04-25T11:24:32.881238+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:40.868312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.879726+0000","last_change":"2024-04-25T11:23:41.646501+0000","last_active":"2024-04-25T11:24:32.879726+0000","last_peered":"2024-04-25T11:24:32.879726+0000","last_clean":"2024-04-25T11:24:32.879726+0000","last_became_active":"2024-04-25T11:23:41.646178+0000","last_became_peered":"2024-04-25T11:23:41.646178+0000","last_unstale":"2024-04-25T11:24:32.879726+0000","last_undegraded":"2024-04-25T11:24:32.879726+0000","last_fullsized":"2024-04-25T11:24:32.879726+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:34.699979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.882936+0000","last_change":"2024-04-25T11:23:41.647025+0000","last_active":"2024-04-25T11:24:32.882936+0000","last_peered":"2024-04-25T11:24:32.882936+0000","last_clean":"2024-04-25T11:24:32.882936+0000","last_became_active":"2024-04-25T11:23:41.646645+0000","last_became_peered":"2024-04-25T11:23:41.646645+0000","last_unstale":"2024-04-25T11:24:32.882936+0000","last_undegraded":"2024-04-25T11:24:32.882936+0000","last_fullsized":"2024-04-25T11:24:32.882936+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:24:57.108535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.877863+0000","last_change":"2024-04-25T11:23:41.660957+0000","last_active":"2024-04-25T11:24:32.877863+0000","last_peered":"2024-04-25T11:24:32.877863+0000","last_clean":"2024-04-25T11:24:32.877863+0000","last_became_active":"2024-04-25T11:23:41.660529+0000","last_became_peered":"2024-04-25T11:23:41.660529+0000","last_unstale":"2024-04-25T11:24:32.877863+0000","last_undegraded":"2024-04-25T11:24:32.877863+0000","last_fullsized":"2024-04-25T11:24:32.877863+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:51.113991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.881993+0000","last_change":"2024-04-25T11:23:41.648777+0000","last_active":"2024-04-25T11:24:32.881993+0000","last_peered":"2024-04-25T11:24:32.881993+0000","last_clean":"2024-04-25T11:24:32.881993+0000","last_became_active":"2024-04-25T11:23:41.647282+0000","last_became_peered":"2024-04-25T11:23:41.647282+0000","last_unstale":"2024-04-25T11:24:32.881993+0000","last_undegraded":"2024-04-25T11:24:32.881993+0000","last_fullsized":"2024-04-25T11:24:32.881993+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:14.396228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.884161+0000","last_change":"2024-04-25T11:23:41.648855+0000","last_active":"2024-04-25T11:24:32.884161+0000","last_peered":"2024-04-25T11:24:32.884161+0000","last_clean":"2024-04-25T11:24:32.884161+0000","last_became_active":"2024-04-25T11:23:41.648527+0000","last_became_peered":"2024-04-25T11:23:41.648527+0000","last_unstale":"2024-04-25T11:24:32.884161+0000","last_undegraded":"2024-04-25T11:24:32.884161+0000","last_fullsized":"2024-04-25T11:24:32.884161+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:13.569916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.876944+0000","last_change":"2024-04-25T11:23:41.646377+0000","last_active":"2024-04-25T11:24:32.876944+0000","last_peered":"2024-04-25T11:24:32.876944+0000","last_clean":"2024-04-25T11:24:32.876944+0000","last_became_active":"2024-04-25T11:23:41.646063+0000","last_became_peered":"2024-04-25T11:23:41.646063+0000","last_unstale":"2024-04-25T11:24:32.876944+0000","last_undegraded":"2024-04-25T11:24:32.876944+0000","last_fullsized":"2024-04-25T11:24:32.876944+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:50.365317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.881286+0000","last_change":"2024-04-25T11:23:41.648147+0000","last_active":"2024-04-25T11:24:32.881286+0000","last_peered":"2024-04-25T11:24:32.881286+0000","last_clean":"2024-04-25T11:24:32.881286+0000","last_became_active":"2024-04-25T11:23:41.647771+0000","last_became_peered":"2024-04-25T11:23:41.647771+0000","last_unstale":"2024-04-25T11:24:32.881286+0000","last_undegraded":"2024-04-25T11:24:32.881286+0000","last_fullsized":"2024-04-25T11:24:32.881286+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:20.157971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.881212+0000","last_change":"2024-04-25T11:23:41.652784+0000","last_active":"2024-04-25T11:24:32.881212+0000","last_peered":"2024-04-25T11:24:32.881212+0000","last_clean":"2024-04-25T11:24:32.881212+0000","last_became_active":"2024-04-25T11:23:41.652434+0000","last_became_peered":"2024-04-25T11:23:41.652434+0000","last_unstale":"2024-04-25T11:24:32.881212+0000","last_undegraded":"2024-04-25T11:24:32.881212+0000","last_fullsized":"2024-04-25T11:24:32.881212+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:48.468138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.882082+0000","last_change":"2024-04-25T11:23:41.648218+0000","last_active":"2024-04-25T11:24:32.882082+0000","last_peered":"2024-04-25T11:24:32.882082+0000","last_clean":"2024-04-25T11:24:32.882082+0000","last_became_active":"2024-04-25T11:23:41.647871+0000","last_became_peered":"2024-04-25T11:23:41.647871+0000","last_unstale":"2024-04-25T11:24:32.882082+0000","last_undegraded":"2024-04-25T11:24:32.882082+0000","last_fullsized":"2024-04-25T11:24:32.882082+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:49.919090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.878985+0000","last_change":"2024-04-25T11:23:41.647087+0000","last_active":"2024-04-25T11:24:32.878985+0000","last_peered":"2024-04-25T11:24:32.878985+0000","last_clean":"2024-04-25T11:24:32.878985+0000","last_became_active":"2024-04-25T11:23:41.646359+0000","last_became_peered":"2024-04-25T11:23:41.646359+0000","last_unstale":"2024-04-25T11:24:32.878985+0000","last_undegraded":"2024-04-25T11:24:32.878985+0000","last_fullsized":"2024-04-25T11:24:32.878985+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:32.129538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.977554+0000","last_change":"2024-04-25T11:23:41.644143+0000","last_active":"2024-04-25T11:24:32.977554+0000","last_peered":"2024-04-25T11:24:32.977554+0000","last_clean":"2024-04-25T11:24:32.977554+0000","last_became_active":"2024-04-25T11:23:41.643738+0000","last_became_peered":"2024-04-25T11:23:41.643738+0000","last_unstale":"2024-04-25T11:24:32.977554+0000","last_undegraded":"2024-04-25T11:24:32.977554+0000","last_fullsized":"2024-04-25T11:24:32.977554+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:19.786933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.881965+0000","last_change":"2024-04-25T11:23:41.636421+0000","last_active":"2024-04-25T11:24:32.881965+0000","last_peered":"2024-04-25T11:24:32.881965+0000","last_clean":"2024-04-25T11:24:32.881965+0000","last_became_active":"2024-04-25T11:23:41.635392+0000","last_became_peered":"2024-04-25T11:23:41.635392+0000","last_unstale":"2024-04-25T11:24:32.881965+0000","last_undegraded":"2024-04-25T11:24:32.881965+0000","last_fullsized":"2024-04-25T11:24:32.881965+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:59.859340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.883464+0000","last_change":"2024-04-25T11:23:45.675875+0000","last_active":"2024-04-25T11:24:32.883464+0000","last_peered":"2024-04-25T11:24:32.883464+0000","last_clean":"2024-04-25T11:24:32.883464+0000","last_became_active":"2024-04-25T11:23:45.675405+0000","last_became_peered":"2024-04-25T11:23:45.675405+0000","last_unstale":"2024-04-25T11:24:32.883464+0000","last_undegraded":"2024-04-25T11:24:32.883464+0000","last_fullsized":"2024-04-25T11:24:32.883464+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:06.732509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":117,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:25:22.170312+0000","last_change":"2024-04-25T11:23:40.630862+0000","last_active":"2024-04-25T11:25:22.170312+0000","last_peered":"2024-04-25T11:25:22.170312+0000","last_clean":"2024-04-25T11:25:22.170312+0000","last_became_active":"2024-04-25T11:23:40.630498+0000","last_became_peered":"2024-04-25T11:23:40.630498+0000","last_unstale":"2024-04-25T11:25:22.170312+0000","last_undegraded":"2024-04-25T11:25:22.170312+0000","last_fullsized":"2024-04-25T11:25:22.170312+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:07:44.028640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":76,"num_read_kb":76,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.879916+0000","last_change":"2024-04-25T11:23:41.639879+0000","last_active":"2024-04-25T11:24:32.879916+0000","last_peered":"2024-04-25T11:24:32.879916+0000","last_clean":"2024-04-25T11:24:32.879916+0000","last_became_active":"2024-04-25T11:23:41.639359+0000","last_became_peered":"2024-04-25T11:23:41.639359+0000","last_unstale":"2024-04-25T11:24:32.879916+0000","last_undegraded":"2024-04-25T11:24:32.879916+0000","last_fullsized":"2024-04-25T11:24:32.879916+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:47.525089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.882272+0000","last_change":"2024-04-25T11:23:45.672219+0000","last_active":"2024-04-25T11:24:32.882272+0000","last_peered":"2024-04-25T11:24:32.882272+0000","last_clean":"2024-04-25T11:24:32.882272+0000","last_became_active":"2024-04-25T11:23:45.671788+0000","last_became_peered":"2024-04-25T11:23:45.671788+0000","last_unstale":"2024-04-25T11:24:32.882272+0000","last_undegraded":"2024-04-25T11:24:32.882272+0000","last_fullsized":"2024-04-25T11:24:32.882272+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:36.228061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.880268+0000","last_change":"2024-04-25T11:23:40.643435+0000","last_active":"2024-04-25T11:24:32.880268+0000","last_peered":"2024-04-25T11:24:32.880268+0000","last_clean":"2024-04-25T11:24:32.880268+0000","last_became_active":"2024-04-25T11:23:40.642990+0000","last_became_peered":"2024-04-25T11:23:40.642990+0000","last_unstale":"2024-04-25T11:24:32.880268+0000","last_undegraded":"2024-04-25T11:24:32.880268+0000","last_fullsized":"2024-04-25T11:24:32.880268+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:42.723486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.881812+0000","last_change":"2024-04-25T11:23:41.645977+0000","last_active":"2024-04-25T11:24:32.881812+0000","last_peered":"2024-04-25T11:24:32.881812+0000","last_clean":"2024-04-25T11:24:32.881812+0000","last_became_active":"2024-04-25T11:23:41.645562+0000","last_became_peered":"2024-04-25T11:23:41.645562+0000","last_unstale":"2024-04-25T11:24:32.881812+0000","last_undegraded":"2024-04-25T11:24:32.881812+0000","last_fullsized":"2024-04-25T11:24:32.881812+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:04.027324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.883329+0000","last_change":"2024-04-25T11:23:45.672843+0000","last_active":"2024-04-25T11:24:32.883329+0000","last_peered":"2024-04-25T11:24:32.883329+0000","last_clean":"2024-04-25T11:24:32.883329+0000","last_became_active":"2024-04-25T11:23:45.672429+0000","last_became_peered":"2024-04-25T11:23:45.672429+0000","last_unstale":"2024-04-25T11:24:32.883329+0000","last_undegraded":"2024-04-25T11:24:32.883329+0000","last_fullsized":"2024-04-25T11:24:32.883329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:48.818624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.883104+0000","last_change":"2024-04-25T11:23:40.634393+0000","last_active":"2024-04-25T11:24:32.883104+0000","last_peered":"2024-04-25T11:24:32.883104+0000","last_clean":"2024-04-25T11:24:32.883104+0000","last_became_active":"2024-04-25T11:23:40.633518+0000","last_became_peered":"2024-04-25T11:23:40.633518+0000","last_unstale":"2024-04-25T11:24:32.883104+0000","last_undegraded":"2024-04-25T11:24:32.883104+0000","last_fullsized":"2024-04-25T11:24:32.883104+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:59.623135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.883017+0000","last_change":"2024-04-25T11:23:41.656152+0000","last_active":"2024-04-25T11:24:32.883017+0000","last_peered":"2024-04-25T11:24:32.883017+0000","last_clean":"2024-04-25T11:24:32.883017+0000","last_became_active":"2024-04-25T11:23:41.655344+0000","last_became_peered":"2024-04-25T11:23:41.655344+0000","last_unstale":"2024-04-25T11:24:32.883017+0000","last_undegraded":"2024-04-25T11:24:32.883017+0000","last_fullsized":"2024-04-25T11:24:32.883017+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:07.649534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.883028+0000","last_change":"2024-04-25T11:23:45.668602+0000","last_active":"2024-04-25T11:24:32.883028+0000","last_peered":"2024-04-25T11:24:32.883028+0000","last_clean":"2024-04-25T11:24:32.883028+0000","last_became_active":"2024-04-25T11:23:45.668143+0000","last_became_peered":"2024-04-25T11:23:45.668143+0000","last_unstale":"2024-04-25T11:24:32.883028+0000","last_undegraded":"2024-04-25T11:24:32.883028+0000","last_fullsized":"2024-04-25T11:24:32.883028+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:13.463604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.898973+0000","last_change":"2024-04-25T11:24:32.898346+0000","last_active":"2024-04-25T11:24:32.898973+0000","last_peered":"2024-04-25T11:24:32.898973+0000","last_clean":"2024-04-25T11:24:32.898973+0000","last_became_active":"2024-04-25T11:24:32.887657+0000","last_became_peered":"2024-04-25T11:24:32.887657+0000","last_unstale":"2024-04-25T11:24:32.898973+0000","last_undegraded":"2024-04-25T11:24:32.898973+0000","last_fullsized":"2024-04-25T11:24:32.898973+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:48.348173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.886185+0000","last_change":"2024-04-25T11:23:41.657112+0000","last_active":"2024-04-25T11:24:32.886185+0000","last_peered":"2024-04-25T11:24:32.886185+0000","last_clean":"2024-04-25T11:24:32.886185+0000","last_became_active":"2024-04-25T11:23:41.656623+0000","last_became_peered":"2024-04-25T11:23:41.656623+0000","last_unstale":"2024-04-25T11:24:32.886185+0000","last_undegraded":"2024-04-25T11:24:32.886185+0000","last_fullsized":"2024-04-25T11:24:32.886185+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:41.644825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.881725+0000","last_change":"2024-04-25T11:23:45.673373+0000","last_active":"2024-04-25T11:24:32.881725+0000","last_peered":"2024-04-25T11:24:32.881725+0000","last_clean":"2024-04-25T11:24:32.881725+0000","last_became_active":"2024-04-25T11:23:45.672928+0000","last_became_peered":"2024-04-25T11:23:45.672928+0000","last_unstale":"2024-04-25T11:24:32.881725+0000","last_undegraded":"2024-04-25T11:24:32.881725+0000","last_fullsized":"2024-04-25T11:24:32.881725+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:10.553189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"80'6","reported_seq":66,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:25:19.964876+0000","last_change":"2024-04-25T11:24:32.901351+0000","last_active":"2024-04-25T11:25:19.964876+0000","last_peered":"2024-04-25T11:25:19.964876+0000","last_clean":"2024-04-25T11:25:19.964876+0000","last_became_active":"2024-04-25T11:24:32.888110+0000","last_became_peered":"2024-04-25T11:24:32.888110+0000","last_unstale":"2024-04-25T11:25:19.964876+0000","last_undegraded":"2024-04-25T11:25:19.964876+0000","last_fullsized":"2024-04-25T11:25:19.964876+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:54.806224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":49,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.882868+0000","last_change":"2024-04-25T11:23:41.648771+0000","last_active":"2024-04-25T11:24:32.882868+0000","last_peered":"2024-04-25T11:24:32.882868+0000","last_clean":"2024-04-25T11:24:32.882868+0000","last_became_active":"2024-04-25T11:23:41.648059+0000","last_became_peered":"2024-04-25T11:23:41.648059+0000","last_unstale":"2024-04-25T11:24:32.882868+0000","last_undegraded":"2024-04-25T11:24:32.882868+0000","last_fullsized":"2024-04-25T11:24:32.882868+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:27.873310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.882752+0000","last_change":"2024-04-25T11:23:45.672676+0000","last_active":"2024-04-25T11:24:32.882752+0000","last_peered":"2024-04-25T11:24:32.882752+0000","last_clean":"2024-04-25T11:24:32.882752+0000","last_became_active":"2024-04-25T11:23:45.672229+0000","last_became_peered":"2024-04-25T11:23:45.672229+0000","last_unstale":"2024-04-25T11:24:32.882752+0000","last_undegraded":"2024-04-25T11:24:32.882752+0000","last_fullsized":"2024-04-25T11:24:32.882752+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:16.648809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":50,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.894986+0000","last_change":"2024-04-25T11:24:32.894986+0000","last_active":"2024-04-25T11:24:32.894986+0000","last_peered":"2024-04-25T11:24:32.894986+0000","last_clean":"2024-04-25T11:24:32.894986+0000","last_became_active":"2024-04-25T11:24:32.888077+0000","last_became_peered":"2024-04-25T11:24:32.888077+0000","last_unstale":"2024-04-25T11:24:32.894986+0000","last_undegraded":"2024-04-25T11:24:32.894986+0000","last_fullsized":"2024-04-25T11:24:32.894986+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:15.739398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.886977+0000","last_change":"2024-04-25T11:23:41.656031+0000","last_active":"2024-04-25T11:24:32.886977+0000","last_peered":"2024-04-25T11:24:32.886977+0000","last_clean":"2024-04-25T11:24:32.886977+0000","last_became_active":"2024-04-25T11:23:41.655656+0000","last_became_peered":"2024-04-25T11:23:41.655656+0000","last_unstale":"2024-04-25T11:24:32.886977+0000","last_undegraded":"2024-04-25T11:24:32.886977+0000","last_fullsized":"2024-04-25T11:24:32.886977+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:37.991435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.889950+0000","last_change":"2024-04-25T11:24:32.889950+0000","last_active":"2024-04-25T11:24:32.889950+0000","last_peered":"2024-04-25T11:24:32.889950+0000","last_clean":"2024-04-25T11:24:32.889950+0000","last_became_active":"2024-04-25T11:24:32.888985+0000","last_became_peered":"2024-04-25T11:24:32.888985+0000","last_unstale":"2024-04-25T11:24:32.889950+0000","last_undegraded":"2024-04-25T11:24:32.889950+0000","last_fullsized":"2024-04-25T11:24:32.889950+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:28.218707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.876579+0000","last_change":"2024-04-25T11:23:40.631581+0000","last_active":"2024-04-25T11:24:32.876579+0000","last_peered":"2024-04-25T11:24:32.876579+0000","last_clean":"2024-04-25T11:24:32.876579+0000","last_became_active":"2024-04-25T11:23:40.631169+0000","last_became_peered":"2024-04-25T11:23:40.631169+0000","last_unstale":"2024-04-25T11:24:32.876579+0000","last_undegraded":"2024-04-25T11:24:32.876579+0000","last_fullsized":"2024-04-25T11:24:32.876579+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:15.737402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.879601+0000","last_change":"2024-04-25T11:23:41.633740+0000","last_active":"2024-04-25T11:24:32.879601+0000","last_peered":"2024-04-25T11:24:32.879601+0000","last_clean":"2024-04-25T11:24:32.879601+0000","last_became_active":"2024-04-25T11:23:41.633445+0000","last_became_peered":"2024-04-25T11:23:41.633445+0000","last_unstale":"2024-04-25T11:24:32.879601+0000","last_undegraded":"2024-04-25T11:24:32.879601+0000","last_fullsized":"2024-04-25T11:24:32.879601+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:46.103646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.882156+0000","last_change":"2024-04-25T11:23:45.678899+0000","last_active":"2024-04-25T11:24:32.882156+0000","last_peered":"2024-04-25T11:24:32.882156+0000","last_clean":"2024-04-25T11:24:32.882156+0000","last_became_active":"2024-04-25T11:23:45.675305+0000","last_became_peered":"2024-04-25T11:23:45.675305+0000","last_unstale":"2024-04-25T11:24:32.882156+0000","last_undegraded":"2024-04-25T11:24:32.882156+0000","last_fullsized":"2024-04-25T11:24:32.882156+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:49.456551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.885598+0000","last_change":"2024-04-25T11:23:40.638090+0000","last_active":"2024-04-25T11:24:32.885598+0000","last_peered":"2024-04-25T11:24:32.885598+0000","last_clean":"2024-04-25T11:24:32.885598+0000","last_became_active":"2024-04-25T11:23:40.637707+0000","last_became_peered":"2024-04-25T11:23:40.637707+0000","last_unstale":"2024-04-25T11:24:32.885598+0000","last_undegraded":"2024-04-25T11:24:32.885598+0000","last_fullsized":"2024-04-25T11:24:32.885598+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:38:43.413273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.977170+0000","last_change":"2024-04-25T11:23:41.641580+0000","last_active":"2024-04-25T11:24:32.977170+0000","last_peered":"2024-04-25T11:24:32.977170+0000","last_clean":"2024-04-25T11:24:32.977170+0000","last_became_active":"2024-04-25T11:23:41.641199+0000","last_became_peered":"2024-04-25T11:23:41.641199+0000","last_unstale":"2024-04-25T11:24:32.977170+0000","last_undegraded":"2024-04-25T11:24:32.977170+0000","last_fullsized":"2024-04-25T11:24:32.977170+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:34:10.471476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.879873+0000","last_change":"2024-04-25T11:23:45.675573+0000","last_active":"2024-04-25T11:24:32.879873+0000","last_peered":"2024-04-25T11:24:32.879873+0000","last_clean":"2024-04-25T11:24:32.879873+0000","last_became_active":"2024-04-25T11:23:45.675197+0000","last_became_peered":"2024-04-25T11:23:45.675197+0000","last_unstale":"2024-04-25T11:24:32.879873+0000","last_undegraded":"2024-04-25T11:24:32.879873+0000","last_fullsized":"2024-04-25T11:24:32.879873+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:39.740949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":184,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:25:22.969395+0000","last_change":"2024-04-25T11:23:40.635020+0000","last_active":"2024-04-25T11:25:22.969395+0000","last_peered":"2024-04-25T11:25:22.969395+0000","last_clean":"2024-04-25T11:25:22.969395+0000","last_became_active":"2024-04-25T11:23:40.634549+0000","last_became_peered":"2024-04-25T11:23:40.634549+0000","last_unstale":"2024-04-25T11:25:22.969395+0000","last_undegraded":"2024-04-25T11:25:22.969395+0000","last_fullsized":"2024-04-25T11:25:22.969395+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:15:16.289370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":151,"num_read_kb":151,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.883366+0000","last_change":"2024-04-25T11:23:41.654903+0000","last_active":"2024-04-25T11:24:32.883366+0000","last_peered":"2024-04-25T11:24:32.883366+0000","last_clean":"2024-04-25T11:24:32.883366+0000","last_became_active":"2024-04-25T11:23:41.654564+0000","last_became_peered":"2024-04-25T11:23:41.654564+0000","last_unstale":"2024-04-25T11:24:32.883366+0000","last_undegraded":"2024-04-25T11:24:32.883366+0000","last_fullsized":"2024-04-25T11:24:32.883366+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:29.025240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.976910+0000","last_change":"2024-04-25T11:23:45.677683+0000","last_active":"2024-04-25T11:24:32.976910+0000","last_peered":"2024-04-25T11:24:32.976910+0000","last_clean":"2024-04-25T11:24:32.976910+0000","last_became_active":"2024-04-25T11:23:45.677286+0000","last_became_peered":"2024-04-25T11:23:45.677286+0000","last_unstale":"2024-04-25T11:24:32.976910+0000","last_undegraded":"2024-04-25T11:24:32.976910+0000","last_fullsized":"2024-04-25T11:24:32.976910+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:43.236060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.881927+0000","last_change":"2024-04-25T11:23:40.640844+0000","last_active":"2024-04-25T11:24:32.881927+0000","last_peered":"2024-04-25T11:24:32.881927+0000","last_clean":"2024-04-25T11:24:32.881927+0000","last_became_active":"2024-04-25T11:23:40.640434+0000","last_became_peered":"2024-04-25T11:23:40.640434+0000","last_unstale":"2024-04-25T11:24:32.881927+0000","last_undegraded":"2024-04-25T11:24:32.881927+0000","last_fullsized":"2024-04-25T11:24:32.881927+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:16.701418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.882074+0000","last_change":"2024-04-25T11:23:41.651962+0000","last_active":"2024-04-25T11:24:32.882074+0000","last_peered":"2024-04-25T11:24:32.882074+0000","last_clean":"2024-04-25T11:24:32.882074+0000","last_became_active":"2024-04-25T11:23:41.650870+0000","last_became_peered":"2024-04-25T11:23:41.650870+0000","last_unstale":"2024-04-25T11:24:32.882074+0000","last_undegraded":"2024-04-25T11:24:32.882074+0000","last_fullsized":"2024-04-25T11:24:32.882074+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:45.623722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.892455+0000","last_change":"2024-04-25T11:24:32.892455+0000","last_active":"2024-04-25T11:24:32.892455+0000","last_peered":"2024-04-25T11:24:32.892455+0000","last_clean":"2024-04-25T11:24:32.892455+0000","last_became_active":"2024-04-25T11:24:32.891968+0000","last_became_peered":"2024-04-25T11:24:32.891968+0000","last_unstale":"2024-04-25T11:24:32.892455+0000","last_undegraded":"2024-04-25T11:24:32.892455+0000","last_fullsized":"2024-04-25T11:24:32.892455+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:31.962400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.886087+0000","last_change":"2024-04-25T11:23:40.642112+0000","last_active":"2024-04-25T11:24:32.886087+0000","last_peered":"2024-04-25T11:24:32.886087+0000","last_clean":"2024-04-25T11:24:32.886087+0000","last_became_active":"2024-04-25T11:23:40.641745+0000","last_became_peered":"2024-04-25T11:23:40.641745+0000","last_unstale":"2024-04-25T11:24:32.886087+0000","last_undegraded":"2024-04-25T11:24:32.886087+0000","last_fullsized":"2024-04-25T11:24:32.886087+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:14.899432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.883183+0000","last_change":"2024-04-25T11:23:41.656460+0000","last_active":"2024-04-25T11:24:32.883183+0000","last_peered":"2024-04-25T11:24:32.883183+0000","last_clean":"2024-04-25T11:24:32.883183+0000","last_became_active":"2024-04-25T11:23:41.655800+0000","last_became_peered":"2024-04-25T11:23:41.655800+0000","last_unstale":"2024-04-25T11:24:32.883183+0000","last_undegraded":"2024-04-25T11:24:32.883183+0000","last_fullsized":"2024-04-25T11:24:32.883183+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:42.505142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.977193+0000","last_change":"2024-04-25T11:23:45.671650+0000","last_active":"2024-04-25T11:24:32.977193+0000","last_peered":"2024-04-25T11:24:32.977193+0000","last_clean":"2024-04-25T11:24:32.977193+0000","last_became_active":"2024-04-25T11:23:45.671254+0000","last_became_peered":"2024-04-25T11:23:45.671254+0000","last_unstale":"2024-04-25T11:24:32.977193+0000","last_undegraded":"2024-04-25T11:24:32.977193+0000","last_fullsized":"2024-04-25T11:24:32.977193+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:29.404898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"80'34","reported_seq":220,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:25:22.969395+0000","last_change":"2024-04-25T11:23:40.633712+0000","last_active":"2024-04-25T11:25:22.969395+0000","last_peered":"2024-04-25T11:25:22.969395+0000","last_clean":"2024-04-25T11:25:22.969395+0000","last_became_active":"2024-04-25T11:23:40.633274+0000","last_became_peered":"2024-04-25T11:23:40.633274+0000","last_unstale":"2024-04-25T11:25:22.969395+0000","last_undegraded":"2024-04-25T11:25:22.969395+0000","last_fullsized":"2024-04-25T11:25:22.969395+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:47.576213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61190,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":157,"num_read_kb":139,"num_write":36,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.881944+0000","last_change":"2024-04-25T11:23:41.649007+0000","last_active":"2024-04-25T11:24:32.881944+0000","last_peered":"2024-04-25T11:24:32.881944+0000","last_clean":"2024-04-25T11:24:32.881944+0000","last_became_active":"2024-04-25T11:23:41.647778+0000","last_became_peered":"2024-04-25T11:23:41.647778+0000","last_unstale":"2024-04-25T11:24:32.881944+0000","last_undegraded":"2024-04-25T11:24:32.881944+0000","last_fullsized":"2024-04-25T11:24:32.881944+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:33.097482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.886452+0000","last_change":"2024-04-25T11:23:45.671645+0000","last_active":"2024-04-25T11:24:32.886452+0000","last_peered":"2024-04-25T11:24:32.886452+0000","last_clean":"2024-04-25T11:24:32.886452+0000","last_became_active":"2024-04-25T11:23:45.670005+0000","last_became_peered":"2024-04-25T11:23:45.670005+0000","last_unstale":"2024-04-25T11:24:32.886452+0000","last_undegraded":"2024-04-25T11:24:32.886452+0000","last_fullsized":"2024-04-25T11:24:32.886452+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:36.047222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.881964+0000","last_change":"2024-04-25T11:23:40.639922+0000","last_active":"2024-04-25T11:24:32.881964+0000","last_peered":"2024-04-25T11:24:32.881964+0000","last_clean":"2024-04-25T11:24:32.881964+0000","last_became_active":"2024-04-25T11:23:40.639544+0000","last_became_peered":"2024-04-25T11:23:40.639544+0000","last_unstale":"2024-04-25T11:24:32.881964+0000","last_undegraded":"2024-04-25T11:24:32.881964+0000","last_fullsized":"2024-04-25T11:24:32.881964+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:06.281281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.876685+0000","last_change":"2024-04-25T11:23:41.629257+0000","last_active":"2024-04-25T11:24:32.876685+0000","last_peered":"2024-04-25T11:24:32.876685+0000","last_clean":"2024-04-25T11:24:32.876685+0000","last_became_active":"2024-04-25T11:23:41.628933+0000","last_became_peered":"2024-04-25T11:23:41.628933+0000","last_unstale":"2024-04-25T11:24:32.876685+0000","last_undegraded":"2024-04-25T11:24:32.876685+0000","last_fullsized":"2024-04-25T11:24:32.876685+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:04.031468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.878397+0000","last_change":"2024-04-25T11:23:45.680460+0000","last_active":"2024-04-25T11:24:32.878397+0000","last_peered":"2024-04-25T11:24:32.878397+0000","last_clean":"2024-04-25T11:24:32.878397+0000","last_became_active":"2024-04-25T11:23:45.679963+0000","last_became_peered":"2024-04-25T11:23:45.679963+0000","last_unstale":"2024-04-25T11:24:32.878397+0000","last_undegraded":"2024-04-25T11:24:32.878397+0000","last_fullsized":"2024-04-25T11:24:32.878397+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:11.199855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.885689+0000","last_change":"2024-04-25T11:23:40.644026+0000","last_active":"2024-04-25T11:24:32.885689+0000","last_peered":"2024-04-25T11:24:32.885689+0000","last_clean":"2024-04-25T11:24:32.885689+0000","last_became_active":"2024-04-25T11:23:40.643665+0000","last_became_peered":"2024-04-25T11:23:40.643665+0000","last_unstale":"2024-04-25T11:24:32.885689+0000","last_undegraded":"2024-04-25T11:24:32.885689+0000","last_fullsized":"2024-04-25T11:24:32.885689+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:32.950387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.881343+0000","last_change":"2024-04-25T11:23:41.642285+0000","last_active":"2024-04-25T11:24:32.881343+0000","last_peered":"2024-04-25T11:24:32.881343+0000","last_clean":"2024-04-25T11:24:32.881343+0000","last_became_active":"2024-04-25T11:23:41.641712+0000","last_became_peered":"2024-04-25T11:23:41.641712+0000","last_unstale":"2024-04-25T11:24:32.881343+0000","last_undegraded":"2024-04-25T11:24:32.881343+0000","last_fullsized":"2024-04-25T11:24:32.881343+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:22.813546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.977448+0000","last_change":"2024-04-25T11:23:45.677632+0000","last_active":"2024-04-25T11:24:32.977448+0000","last_peered":"2024-04-25T11:24:32.977448+0000","last_clean":"2024-04-25T11:24:32.977448+0000","last_became_active":"2024-04-25T11:23:45.677244+0000","last_became_peered":"2024-04-25T11:23:45.677244+0000","last_unstale":"2024-04-25T11:24:32.977448+0000","last_undegraded":"2024-04-25T11:24:32.977448+0000","last_fullsized":"2024-04-25T11:24:32.977448+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:03.982435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"80'5","reported_seq":112,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:25:22.187596+0000","last_change":"2024-04-25T11:23:40.634661+0000","last_active":"2024-04-25T11:25:22.187596+0000","last_peered":"2024-04-25T11:25:22.187596+0000","last_clean":"2024-04-25T11:25:22.187596+0000","last_became_active":"2024-04-25T11:23:40.634211+0000","last_became_peered":"2024-04-25T11:23:40.634211+0000","last_unstale":"2024-04-25T11:25:22.187596+0000","last_undegraded":"2024-04-25T11:25:22.187596+0000","last_fullsized":"2024-04-25T11:25:22.187596+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:54.928339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":79,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.881199+0000","last_change":"2024-04-25T11:23:41.644147+0000","last_active":"2024-04-25T11:24:32.881199+0000","last_peered":"2024-04-25T11:24:32.881199+0000","last_clean":"2024-04-25T11:24:32.881199+0000","last_became_active":"2024-04-25T11:23:41.643103+0000","last_became_peered":"2024-04-25T11:23:41.643103+0000","last_unstale":"2024-04-25T11:24:32.881199+0000","last_undegraded":"2024-04-25T11:24:32.881199+0000","last_fullsized":"2024-04-25T11:24:32.881199+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:00.956551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.877579+0000","last_change":"2024-04-25T11:23:45.671504+0000","last_active":"2024-04-25T11:24:32.877579+0000","last_peered":"2024-04-25T11:24:32.877579+0000","last_clean":"2024-04-25T11:24:32.877579+0000","last_became_active":"2024-04-25T11:23:45.671148+0000","last_became_peered":"2024-04-25T11:23:45.671148+0000","last_unstale":"2024-04-25T11:24:32.877579+0000","last_undegraded":"2024-04-25T11:24:32.877579+0000","last_fullsized":"2024-04-25T11:24:32.877579+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:21.795839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":70,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.906235+0000","last_change":"2024-04-25T11:24:32.906235+0000","last_active":"2024-04-25T11:24:32.906235+0000","last_peered":"2024-04-25T11:24:32.906235+0000","last_clean":"2024-04-25T11:24:32.906235+0000","last_became_active":"2024-04-25T11:24:32.890228+0000","last_became_peered":"2024-04-25T11:24:32.890228+0000","last_unstale":"2024-04-25T11:24:32.906235+0000","last_undegraded":"2024-04-25T11:24:32.906235+0000","last_fullsized":"2024-04-25T11:24:32.906235+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:44.096420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.885831+0000","last_change":"2024-04-25T11:23:41.658283+0000","last_active":"2024-04-25T11:24:32.885831+0000","last_peered":"2024-04-25T11:24:32.885831+0000","last_clean":"2024-04-25T11:24:32.885831+0000","last_became_active":"2024-04-25T11:23:41.657898+0000","last_became_peered":"2024-04-25T11:23:41.657898+0000","last_unstale":"2024-04-25T11:24:32.885831+0000","last_undegraded":"2024-04-25T11:24:32.885831+0000","last_fullsized":"2024-04-25T11:24:32.885831+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:52:55.941232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.889595+0000","last_change":"2024-04-25T11:24:32.888574+0000","last_active":"2024-04-25T11:24:32.889595+0000","last_peered":"2024-04-25T11:24:32.889595+0000","last_clean":"2024-04-25T11:24:32.889595+0000","last_became_active":"2024-04-25T11:24:32.888047+0000","last_became_peered":"2024-04-25T11:24:32.888047+0000","last_unstale":"2024-04-25T11:24:32.889595+0000","last_undegraded":"2024-04-25T11:24:32.889595+0000","last_fullsized":"2024-04-25T11:24:32.889595+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:20.530042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.883512+0000","last_change":"2024-04-25T11:23:40.639442+0000","last_active":"2024-04-25T11:24:32.883512+0000","last_peered":"2024-04-25T11:24:32.883512+0000","last_clean":"2024-04-25T11:24:32.883512+0000","last_became_active":"2024-04-25T11:23:40.639068+0000","last_became_peered":"2024-04-25T11:23:40.639068+0000","last_unstale":"2024-04-25T11:24:32.883512+0000","last_undegraded":"2024-04-25T11:24:32.883512+0000","last_fullsized":"2024-04-25T11:24:32.883512+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:21.491233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.878774+0000","last_change":"2024-04-25T11:23:41.652309+0000","last_active":"2024-04-25T11:24:32.878774+0000","last_peered":"2024-04-25T11:24:32.878774+0000","last_clean":"2024-04-25T11:24:32.878774+0000","last_became_active":"2024-04-25T11:23:41.651927+0000","last_became_peered":"2024-04-25T11:23:41.651927+0000","last_unstale":"2024-04-25T11:24:32.878774+0000","last_undegraded":"2024-04-25T11:24:32.878774+0000","last_fullsized":"2024-04-25T11:24:32.878774+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:51.002480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.882725+0000","last_change":"2024-04-25T11:23:46.384203+0000","last_active":"2024-04-25T11:24:32.882725+0000","last_peered":"2024-04-25T11:24:32.882725+0000","last_clean":"2024-04-25T11:24:32.882725+0000","last_became_active":"2024-04-25T11:23:46.383532+0000","last_became_peered":"2024-04-25T11:23:46.383532+0000","last_unstale":"2024-04-25T11:24:32.882725+0000","last_undegraded":"2024-04-25T11:24:32.882725+0000","last_fullsized":"2024-04-25T11:24:32.882725+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:25.265484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.977503+0000","last_change":"2024-04-25T11:23:40.640082+0000","last_active":"2024-04-25T11:24:32.977503+0000","last_peered":"2024-04-25T11:24:32.977503+0000","last_clean":"2024-04-25T11:24:32.977503+0000","last_became_active":"2024-04-25T11:23:40.639725+0000","last_became_peered":"2024-04-25T11:23:40.639725+0000","last_unstale":"2024-04-25T11:24:32.977503+0000","last_undegraded":"2024-04-25T11:24:32.977503+0000","last_fullsized":"2024-04-25T11:24:32.977503+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:01.244116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.877704+0000","last_change":"2024-04-25T11:23:41.645439+0000","last_active":"2024-04-25T11:24:32.877704+0000","last_peered":"2024-04-25T11:24:32.877704+0000","last_clean":"2024-04-25T11:24:32.877704+0000","last_became_active":"2024-04-25T11:23:41.645056+0000","last_became_peered":"2024-04-25T11:23:41.645056+0000","last_unstale":"2024-04-25T11:24:32.877704+0000","last_undegraded":"2024-04-25T11:24:32.877704+0000","last_fullsized":"2024-04-25T11:24:32.877704+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:43.100321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.876787+0000","last_change":"2024-04-25T11:23:45.672174+0000","last_active":"2024-04-25T11:24:32.876787+0000","last_peered":"2024-04-25T11:24:32.876787+0000","last_clean":"2024-04-25T11:24:32.876787+0000","last_became_active":"2024-04-25T11:23:45.670401+0000","last_became_peered":"2024-04-25T11:23:45.670401+0000","last_unstale":"2024-04-25T11:24:32.876787+0000","last_undegraded":"2024-04-25T11:24:32.876787+0000","last_fullsized":"2024-04-25T11:24:32.876787+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:59.259644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"80'5","reported_seq":110,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:25:23.172366+0000","last_change":"2024-04-25T11:23:40.636704+0000","last_active":"2024-04-25T11:25:23.172366+0000","last_peered":"2024-04-25T11:25:23.172366+0000","last_clean":"2024-04-25T11:25:23.172366+0000","last_became_active":"2024-04-25T11:23:40.636319+0000","last_became_peered":"2024-04-25T11:23:40.636319+0000","last_unstale":"2024-04-25T11:25:23.172366+0000","last_undegraded":"2024-04-25T11:25:23.172366+0000","last_fullsized":"2024-04-25T11:25:23.172366+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:17.199946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.877397+0000","last_change":"2024-04-25T11:23:41.660510+0000","last_active":"2024-04-25T11:24:32.877397+0000","last_peered":"2024-04-25T11:24:32.877397+0000","last_clean":"2024-04-25T11:24:32.877397+0000","last_became_active":"2024-04-25T11:23:41.659837+0000","last_became_peered":"2024-04-25T11:23:41.659837+0000","last_unstale":"2024-04-25T11:24:32.877397+0000","last_undegraded":"2024-04-25T11:24:32.877397+0000","last_fullsized":"2024-04-25T11:24:32.877397+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:33.879487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.877227+0000","last_change":"2024-04-25T11:23:46.383239+0000","last_active":"2024-04-25T11:24:32.877227+0000","last_peered":"2024-04-25T11:24:32.877227+0000","last_clean":"2024-04-25T11:24:32.877227+0000","last_became_active":"2024-04-25T11:23:46.382728+0000","last_became_peered":"2024-04-25T11:23:46.382728+0000","last_unstale":"2024-04-25T11:24:32.877227+0000","last_undegraded":"2024-04-25T11:24:32.877227+0000","last_fullsized":"2024-04-25T11:24:32.877227+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:25.087389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.885805+0000","last_change":"2024-04-25T11:23:41.654723+0000","last_active":"2024-04-25T11:24:32.885805+0000","last_peered":"2024-04-25T11:24:32.885805+0000","last_clean":"2024-04-25T11:24:32.885805+0000","last_became_active":"2024-04-25T11:23:41.654140+0000","last_became_peered":"2024-04-25T11:23:41.654140+0000","last_unstale":"2024-04-25T11:24:32.885805+0000","last_undegraded":"2024-04-25T11:24:32.885805+0000","last_fullsized":"2024-04-25T11:24:32.885805+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:41.981728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":111,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:25:23.188354+0000","last_change":"2024-04-25T11:23:40.641902+0000","last_active":"2024-04-25T11:25:23.188354+0000","last_peered":"2024-04-25T11:25:23.188354+0000","last_clean":"2024-04-25T11:25:23.188354+0000","last_became_active":"2024-04-25T11:23:40.641490+0000","last_became_peered":"2024-04-25T11:23:40.641490+0000","last_unstale":"2024-04-25T11:25:23.188354+0000","last_undegraded":"2024-04-25T11:25:23.188354+0000","last_fullsized":"2024-04-25T11:25:23.188354+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:03.939354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":78,"num_read_kb":78,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"80'1","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.882503+0000","last_change":"2024-04-25T11:23:45.670485+0000","last_active":"2024-04-25T11:24:32.882503+0000","last_peered":"2024-04-25T11:24:32.882503+0000","last_clean":"2024-04-25T11:24:32.882503+0000","last_became_active":"2024-04-25T11:23:45.669842+0000","last_became_peered":"2024-04-25T11:23:45.669842+0000","last_unstale":"2024-04-25T11:24:32.882503+0000","last_undegraded":"2024-04-25T11:24:32.882503+0000","last_fullsized":"2024-04-25T11:24:32.882503+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:32.801217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.880369+0000","last_change":"2024-04-25T11:23:41.640583+0000","last_active":"2024-04-25T11:24:32.880369+0000","last_peered":"2024-04-25T11:24:32.880369+0000","last_clean":"2024-04-25T11:24:32.880369+0000","last_became_active":"2024-04-25T11:23:41.640262+0000","last_became_peered":"2024-04-25T11:23:41.640262+0000","last_unstale":"2024-04-25T11:24:32.880369+0000","last_undegraded":"2024-04-25T11:24:32.880369+0000","last_fullsized":"2024-04-25T11:24:32.880369+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:09.539504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.883467+0000","last_change":"2024-04-25T11:23:40.628424+0000","last_active":"2024-04-25T11:24:32.883467+0000","last_peered":"2024-04-25T11:24:32.883467+0000","last_clean":"2024-04-25T11:24:32.883467+0000","last_became_active":"2024-04-25T11:23:40.628064+0000","last_became_peered":"2024-04-25T11:23:40.628064+0000","last_unstale":"2024-04-25T11:24:32.883467+0000","last_undegraded":"2024-04-25T11:24:32.883467+0000","last_fullsized":"2024-04-25T11:24:32.883467+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:34.837291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":104,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.880968+0000","last_change":"2024-04-25T11:22:25.135774+0000","last_active":"2024-04-25T11:24:32.880968+0000","last_peered":"2024-04-25T11:24:32.880968+0000","last_clean":"2024-04-25T11:24:32.880968+0000","last_became_active":"2024-04-25T11:22:25.131178+0000","last_became_peered":"2024-04-25T11:22:25.131178+0000","last_unstale":"2024-04-25T11:24:32.880968+0000","last_undegraded":"2024-04-25T11:24:32.880968+0000","last_fullsized":"2024-04-25T11:24:32.880968+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:18:57.891013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:18:57.891013+0000","last_clean_scrub_stamp":"2024-04-25T11:18:57.891013+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:29.317685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.886668+0000","last_change":"2024-04-25T11:23:46.385048+0000","last_active":"2024-04-25T11:24:32.886668+0000","last_peered":"2024-04-25T11:24:32.886668+0000","last_clean":"2024-04-25T11:24:32.886668+0000","last_became_active":"2024-04-25T11:23:46.384014+0000","last_became_peered":"2024-04-25T11:23:46.384014+0000","last_unstale":"2024-04-25T11:24:32.886668+0000","last_undegraded":"2024-04-25T11:24:32.886668+0000","last_fullsized":"2024-04-25T11:24:32.886668+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:26.508616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.876446+0000","last_change":"2024-04-25T11:23:41.650783+0000","last_active":"2024-04-25T11:24:32.876446+0000","last_peered":"2024-04-25T11:24:32.876446+0000","last_clean":"2024-04-25T11:24:32.876446+0000","last_became_active":"2024-04-25T11:23:41.650431+0000","last_became_peered":"2024-04-25T11:23:41.650431+0000","last_unstale":"2024-04-25T11:24:32.876446+0000","last_undegraded":"2024-04-25T11:24:32.876446+0000","last_fullsized":"2024-04-25T11:24:32.876446+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:38.925779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.881061+0000","last_change":"2024-04-25T11:23:40.638879+0000","last_active":"2024-04-25T11:24:32.881061+0000","last_peered":"2024-04-25T11:24:32.881061+0000","last_clean":"2024-04-25T11:24:32.881061+0000","last_became_active":"2024-04-25T11:23:40.638475+0000","last_became_peered":"2024-04-25T11:23:40.638475+0000","last_unstale":"2024-04-25T11:24:32.881061+0000","last_undegraded":"2024-04-25T11:24:32.881061+0000","last_fullsized":"2024-04-25T11:24:32.881061+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:25.591831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.878313+0000","last_change":"2024-04-25T11:23:45.679333+0000","last_active":"2024-04-25T11:24:32.878313+0000","last_peered":"2024-04-25T11:24:32.878313+0000","last_clean":"2024-04-25T11:24:32.878313+0000","last_became_active":"2024-04-25T11:23:45.678622+0000","last_became_peered":"2024-04-25T11:23:45.678622+0000","last_unstale":"2024-04-25T11:24:32.878313+0000","last_undegraded":"2024-04-25T11:24:32.878313+0000","last_fullsized":"2024-04-25T11:24:32.878313+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:08.951281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.882094+0000","last_change":"2024-04-25T11:23:40.642975+0000","last_active":"2024-04-25T11:24:32.882094+0000","last_peered":"2024-04-25T11:24:32.882094+0000","last_clean":"2024-04-25T11:24:32.882094+0000","last_became_active":"2024-04-25T11:23:40.642605+0000","last_became_peered":"2024-04-25T11:23:40.642605+0000","last_unstale":"2024-04-25T11:24:32.882094+0000","last_undegraded":"2024-04-25T11:24:32.882094+0000","last_fullsized":"2024-04-25T11:24:32.882094+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:57.665763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.876918+0000","last_change":"2024-04-25T11:23:41.628396+0000","last_active":"2024-04-25T11:24:32.876918+0000","last_peered":"2024-04-25T11:24:32.876918+0000","last_clean":"2024-04-25T11:24:32.876918+0000","last_became_active":"2024-04-25T11:23:41.627979+0000","last_became_peered":"2024-04-25T11:23:41.627979+0000","last_unstale":"2024-04-25T11:24:32.876918+0000","last_undegraded":"2024-04-25T11:24:32.876918+0000","last_fullsized":"2024-04-25T11:24:32.876918+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:48.471219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.879530+0000","last_change":"2024-04-25T11:23:46.384252+0000","last_active":"2024-04-25T11:24:32.879530+0000","last_peered":"2024-04-25T11:24:32.879530+0000","last_clean":"2024-04-25T11:24:32.879530+0000","last_became_active":"2024-04-25T11:23:46.383408+0000","last_became_peered":"2024-04-25T11:23:46.383408+0000","last_unstale":"2024-04-25T11:24:32.879530+0000","last_undegraded":"2024-04-25T11:24:32.879530+0000","last_fullsized":"2024-04-25T11:24:32.879530+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:19.337043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"80'12","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.977559+0000","last_change":"2024-04-25T11:23:40.643767+0000","last_active":"2024-04-25T11:24:32.977559+0000","last_peered":"2024-04-25T11:24:32.977559+0000","last_clean":"2024-04-25T11:24:32.977559+0000","last_became_active":"2024-04-25T11:23:40.642748+0000","last_became_peered":"2024-04-25T11:23:40.642748+0000","last_unstale":"2024-04-25T11:24:32.977559+0000","last_undegraded":"2024-04-25T11:24:32.977559+0000","last_fullsized":"2024-04-25T11:24:32.977559+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:12.322055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.880652+0000","last_change":"2024-04-25T11:23:41.642766+0000","last_active":"2024-04-25T11:24:32.880652+0000","last_peered":"2024-04-25T11:24:32.880652+0000","last_clean":"2024-04-25T11:24:32.880652+0000","last_became_active":"2024-04-25T11:23:41.642409+0000","last_became_peered":"2024-04-25T11:23:41.642409+0000","last_unstale":"2024-04-25T11:24:32.880652+0000","last_undegraded":"2024-04-25T11:24:32.880652+0000","last_fullsized":"2024-04-25T11:24:32.880652+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:36.139574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.882041+0000","last_change":"2024-04-25T11:23:45.670926+0000","last_active":"2024-04-25T11:24:32.882041+0000","last_peered":"2024-04-25T11:24:32.882041+0000","last_clean":"2024-04-25T11:24:32.882041+0000","last_became_active":"2024-04-25T11:23:45.670444+0000","last_became_peered":"2024-04-25T11:23:45.670444+0000","last_unstale":"2024-04-25T11:24:32.882041+0000","last_undegraded":"2024-04-25T11:24:32.882041+0000","last_fullsized":"2024-04-25T11:24:32.882041+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:40.538155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.884970+0000","last_change":"2024-04-25T11:23:40.647132+0000","last_active":"2024-04-25T11:24:32.884970+0000","last_peered":"2024-04-25T11:24:32.884970+0000","last_clean":"2024-04-25T11:24:32.884970+0000","last_became_active":"2024-04-25T11:23:40.646792+0000","last_became_peered":"2024-04-25T11:23:40.646792+0000","last_unstale":"2024-04-25T11:24:32.884970+0000","last_undegraded":"2024-04-25T11:24:32.884970+0000","last_fullsized":"2024-04-25T11:24:32.884970+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:50.935908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.882364+0000","last_change":"2024-04-25T11:23:41.658368+0000","last_active":"2024-04-25T11:24:32.882364+0000","last_peered":"2024-04-25T11:24:32.882364+0000","last_clean":"2024-04-25T11:24:32.882364+0000","last_became_active":"2024-04-25T11:23:41.657972+0000","last_became_peered":"2024-04-25T11:23:41.657972+0000","last_unstale":"2024-04-25T11:24:32.882364+0000","last_undegraded":"2024-04-25T11:24:32.882364+0000","last_fullsized":"2024-04-25T11:24:32.882364+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:02.717294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.884235+0000","last_change":"2024-04-25T11:23:46.384683+0000","last_active":"2024-04-25T11:24:32.884235+0000","last_peered":"2024-04-25T11:24:32.884235+0000","last_clean":"2024-04-25T11:24:32.884235+0000","last_became_active":"2024-04-25T11:23:46.384230+0000","last_became_peered":"2024-04-25T11:23:46.384230+0000","last_unstale":"2024-04-25T11:24:32.884235+0000","last_undegraded":"2024-04-25T11:24:32.884235+0000","last_fullsized":"2024-04-25T11:24:32.884235+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:06.982061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.882212+0000","last_change":"2024-04-25T11:23:40.642289+0000","last_active":"2024-04-25T11:24:32.882212+0000","last_peered":"2024-04-25T11:24:32.882212+0000","last_clean":"2024-04-25T11:24:32.882212+0000","last_became_active":"2024-04-25T11:23:40.641948+0000","last_became_peered":"2024-04-25T11:23:40.641948+0000","last_unstale":"2024-04-25T11:24:32.882212+0000","last_undegraded":"2024-04-25T11:24:32.882212+0000","last_fullsized":"2024-04-25T11:24:32.882212+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:39.265090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.880445+0000","last_change":"2024-04-25T11:23:41.651792+0000","last_active":"2024-04-25T11:24:32.880445+0000","last_peered":"2024-04-25T11:24:32.880445+0000","last_clean":"2024-04-25T11:24:32.880445+0000","last_became_active":"2024-04-25T11:23:41.651413+0000","last_became_peered":"2024-04-25T11:23:41.651413+0000","last_unstale":"2024-04-25T11:24:32.880445+0000","last_undegraded":"2024-04-25T11:24:32.880445+0000","last_fullsized":"2024-04-25T11:24:32.880445+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:50.805600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.977511+0000","last_change":"2024-04-25T11:23:45.670509+0000","last_active":"2024-04-25T11:24:32.977511+0000","last_peered":"2024-04-25T11:24:32.977511+0000","last_clean":"2024-04-25T11:24:32.977511+0000","last_became_active":"2024-04-25T11:23:45.670145+0000","last_became_peered":"2024-04-25T11:23:45.670145+0000","last_unstale":"2024-04-25T11:24:32.977511+0000","last_undegraded":"2024-04-25T11:24:32.977511+0000","last_fullsized":"2024-04-25T11:24:32.977511+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:03.764278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.879963+0000","last_change":"2024-04-25T11:23:40.644123+0000","last_active":"2024-04-25T11:24:32.879963+0000","last_peered":"2024-04-25T11:24:32.879963+0000","last_clean":"2024-04-25T11:24:32.879963+0000","last_became_active":"2024-04-25T11:23:40.643750+0000","last_became_peered":"2024-04-25T11:23:40.643750+0000","last_unstale":"2024-04-25T11:24:32.879963+0000","last_undegraded":"2024-04-25T11:24:32.879963+0000","last_fullsized":"2024-04-25T11:24:32.879963+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:49.613669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.878031+0000","last_change":"2024-04-25T11:23:41.643606+0000","last_active":"2024-04-25T11:24:32.878031+0000","last_peered":"2024-04-25T11:24:32.878031+0000","last_clean":"2024-04-25T11:24:32.878031+0000","last_became_active":"2024-04-25T11:23:41.643248+0000","last_became_peered":"2024-04-25T11:23:41.643248+0000","last_unstale":"2024-04-25T11:24:32.878031+0000","last_undegraded":"2024-04-25T11:24:32.878031+0000","last_fullsized":"2024-04-25T11:24:32.878031+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:46.200959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"80'1","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.881044+0000","last_change":"2024-04-25T11:23:45.674138+0000","last_active":"2024-04-25T11:24:32.881044+0000","last_peered":"2024-04-25T11:24:32.881044+0000","last_clean":"2024-04-25T11:24:32.881044+0000","last_became_active":"2024-04-25T11:23:45.672549+0000","last_became_peered":"2024-04-25T11:23:45.672549+0000","last_unstale":"2024-04-25T11:24:32.881044+0000","last_undegraded":"2024-04-25T11:24:32.881044+0000","last_fullsized":"2024-04-25T11:24:32.881044+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:11.345176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.881426+0000","last_change":"2024-04-25T11:23:40.636602+0000","last_active":"2024-04-25T11:24:32.881426+0000","last_peered":"2024-04-25T11:24:32.881426+0000","last_clean":"2024-04-25T11:24:32.881426+0000","last_became_active":"2024-04-25T11:23:40.636187+0000","last_became_peered":"2024-04-25T11:23:40.636187+0000","last_unstale":"2024-04-25T11:24:32.881426+0000","last_undegraded":"2024-04-25T11:24:32.881426+0000","last_fullsized":"2024-04-25T11:24:32.881426+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:15.894714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.880078+0000","last_change":"2024-04-25T11:23:41.647345+0000","last_active":"2024-04-25T11:24:32.880078+0000","last_peered":"2024-04-25T11:24:32.880078+0000","last_clean":"2024-04-25T11:24:32.880078+0000","last_became_active":"2024-04-25T11:23:41.646981+0000","last_became_peered":"2024-04-25T11:23:41.646981+0000","last_unstale":"2024-04-25T11:24:32.880078+0000","last_undegraded":"2024-04-25T11:24:32.880078+0000","last_fullsized":"2024-04-25T11:24:32.880078+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:30.096856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.884216+0000","last_change":"2024-04-25T11:23:46.384132+0000","last_active":"2024-04-25T11:24:32.884216+0000","last_peered":"2024-04-25T11:24:32.884216+0000","last_clean":"2024-04-25T11:24:32.884216+0000","last_became_active":"2024-04-25T11:23:46.383501+0000","last_became_peered":"2024-04-25T11:23:46.383501+0000","last_unstale":"2024-04-25T11:24:32.884216+0000","last_undegraded":"2024-04-25T11:24:32.884216+0000","last_fullsized":"2024-04-25T11:24:32.884216+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:22.486326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.880575+0000","last_change":"2024-04-25T11:23:40.634942+0000","last_active":"2024-04-25T11:24:32.880575+0000","last_peered":"2024-04-25T11:24:32.880575+0000","last_clean":"2024-04-25T11:24:32.880575+0000","last_became_active":"2024-04-25T11:23:40.634557+0000","last_became_peered":"2024-04-25T11:23:40.634557+0000","last_unstale":"2024-04-25T11:24:32.880575+0000","last_undegraded":"2024-04-25T11:24:32.880575+0000","last_fullsized":"2024-04-25T11:24:32.880575+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:08.612328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.883122+0000","last_change":"2024-04-25T11:23:41.648985+0000","last_active":"2024-04-25T11:24:32.883122+0000","last_peered":"2024-04-25T11:24:32.883122+0000","last_clean":"2024-04-25T11:24:32.883122+0000","last_became_active":"2024-04-25T11:23:41.645643+0000","last_became_peered":"2024-04-25T11:23:41.645643+0000","last_unstale":"2024-04-25T11:24:32.883122+0000","last_undegraded":"2024-04-25T11:24:32.883122+0000","last_fullsized":"2024-04-25T11:24:32.883122+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:22.289698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.882381+0000","last_change":"2024-04-25T11:23:45.671474+0000","last_active":"2024-04-25T11:24:32.882381+0000","last_peered":"2024-04-25T11:24:32.882381+0000","last_clean":"2024-04-25T11:24:32.882381+0000","last_became_active":"2024-04-25T11:23:45.670976+0000","last_became_peered":"2024-04-25T11:23:45.670976+0000","last_unstale":"2024-04-25T11:24:32.882381+0000","last_undegraded":"2024-04-25T11:24:32.882381+0000","last_fullsized":"2024-04-25T11:24:32.882381+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:13:29.771087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"80'6","reported_seq":112,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:25:21.176264+0000","last_change":"2024-04-25T11:23:40.647950+0000","last_active":"2024-04-25T11:25:21.176264+0000","last_peered":"2024-04-25T11:25:21.176264+0000","last_clean":"2024-04-25T11:25:21.176264+0000","last_became_active":"2024-04-25T11:23:40.647549+0000","last_became_peered":"2024-04-25T11:23:40.647549+0000","last_unstale":"2024-04-25T11:25:21.176264+0000","last_undegraded":"2024-04-25T11:25:21.176264+0000","last_fullsized":"2024-04-25T11:25:21.176264+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:47.975294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":78,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.882177+0000","last_change":"2024-04-25T11:23:41.647670+0000","last_active":"2024-04-25T11:24:32.882177+0000","last_peered":"2024-04-25T11:24:32.882177+0000","last_clean":"2024-04-25T11:24:32.882177+0000","last_became_active":"2024-04-25T11:23:41.646686+0000","last_became_peered":"2024-04-25T11:23:41.646686+0000","last_unstale":"2024-04-25T11:24:32.882177+0000","last_undegraded":"2024-04-25T11:24:32.882177+0000","last_fullsized":"2024-04-25T11:24:32.882177+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:21.260685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.888278+0000","last_change":"2024-04-25T11:24:32.888278+0000","last_active":"2024-04-25T11:24:32.888278+0000","last_peered":"2024-04-25T11:24:32.888278+0000","last_clean":"2024-04-25T11:24:32.888278+0000","last_became_active":"2024-04-25T11:24:32.887747+0000","last_became_peered":"2024-04-25T11:24:32.887747+0000","last_unstale":"2024-04-25T11:24:32.888278+0000","last_undegraded":"2024-04-25T11:24:32.888278+0000","last_fullsized":"2024-04-25T11:24:32.888278+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:03.960560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,10,1,4],"acting":[0,10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.886312+0000","last_change":"2024-04-25T11:23:40.642974+0000","last_active":"2024-04-25T11:24:32.886312+0000","last_peered":"2024-04-25T11:24:32.886312+0000","last_clean":"2024-04-25T11:24:32.886312+0000","last_became_active":"2024-04-25T11:23:40.642628+0000","last_became_peered":"2024-04-25T11:23:40.642628+0000","last_unstale":"2024-04-25T11:24:32.886312+0000","last_undegraded":"2024-04-25T11:24:32.886312+0000","last_fullsized":"2024-04-25T11:24:32.886312+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:03.870867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.882406+0000","last_change":"2024-04-25T11:23:41.643519+0000","last_active":"2024-04-25T11:24:32.882406+0000","last_peered":"2024-04-25T11:24:32.882406+0000","last_clean":"2024-04-25T11:24:32.882406+0000","last_became_active":"2024-04-25T11:23:41.642621+0000","last_became_peered":"2024-04-25T11:23:41.642621+0000","last_unstale":"2024-04-25T11:24:32.882406+0000","last_undegraded":"2024-04-25T11:24:32.882406+0000","last_fullsized":"2024-04-25T11:24:32.882406+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:38.699914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.884821+0000","last_change":"2024-04-25T11:23:45.671017+0000","last_active":"2024-04-25T11:24:32.884821+0000","last_peered":"2024-04-25T11:24:32.884821+0000","last_clean":"2024-04-25T11:24:32.884821+0000","last_became_active":"2024-04-25T11:23:45.670603+0000","last_became_peered":"2024-04-25T11:23:45.670603+0000","last_unstale":"2024-04-25T11:24:32.884821+0000","last_undegraded":"2024-04-25T11:24:32.884821+0000","last_fullsized":"2024-04-25T11:24:32.884821+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:44.644163+0000","last_clean_scrub_stamp":"2024-04-25T11:23:44.644163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:32.385176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":45,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.891324+0000","last_change":"2024-04-25T11:24:32.891324+0000","last_active":"2024-04-25T11:24:32.891324+0000","last_peered":"2024-04-25T11:24:32.891324+0000","last_clean":"2024-04-25T11:24:32.891324+0000","last_became_active":"2024-04-25T11:24:32.885073+0000","last_became_peered":"2024-04-25T11:24:32.885073+0000","last_unstale":"2024-04-25T11:24:32.891324+0000","last_undegraded":"2024-04-25T11:24:32.891324+0000","last_fullsized":"2024-04-25T11:24:32.891324+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:39.606958+0000","last_clean_scrub_stamp":"2024-04-25T11:23:39.606958+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:38.326741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.881613+0000","last_change":"2024-04-25T11:23:41.634946+0000","last_active":"2024-04-25T11:24:32.881613+0000","last_peered":"2024-04-25T11:24:32.881613+0000","last_clean":"2024-04-25T11:24:32.881613+0000","last_became_active":"2024-04-25T11:23:41.634505+0000","last_became_peered":"2024-04-25T11:23:41.634505+0000","last_unstale":"2024-04-25T11:24:32.881613+0000","last_undegraded":"2024-04-25T11:24:32.881613+0000","last_fullsized":"2024-04-25T11:24:32.881613+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:52.793207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.878128+0000","last_change":"2024-04-25T11:23:41.642079+0000","last_active":"2024-04-25T11:24:32.878128+0000","last_peered":"2024-04-25T11:24:32.878128+0000","last_clean":"2024-04-25T11:24:32.878128+0000","last_became_active":"2024-04-25T11:23:41.641721+0000","last_became_peered":"2024-04-25T11:23:41.641721+0000","last_unstale":"2024-04-25T11:24:32.878128+0000","last_undegraded":"2024-04-25T11:24:32.878128+0000","last_fullsized":"2024-04-25T11:24:32.878128+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:59.968749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.891176+0000","last_change":"2024-04-25T11:24:32.891176+0000","last_active":"2024-04-25T11:24:32.891176+0000","last_peered":"2024-04-25T11:24:32.891176+0000","last_clean":"2024-04-25T11:24:32.891176+0000","last_became_active":"2024-04-25T11:24:32.890758+0000","last_became_peered":"2024-04-25T11:24:32.890758+0000","last_unstale":"2024-04-25T11:24:32.891176+0000","last_undegraded":"2024-04-25T11:24:32.891176+0000","last_fullsized":"2024-04-25T11:24:32.891176+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:25.871589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.881233+0000","last_change":"2024-04-25T11:23:41.641108+0000","last_active":"2024-04-25T11:24:32.881233+0000","last_peered":"2024-04-25T11:24:32.881233+0000","last_clean":"2024-04-25T11:24:32.881233+0000","last_became_active":"2024-04-25T11:23:41.640665+0000","last_became_peered":"2024-04-25T11:23:41.640665+0000","last_unstale":"2024-04-25T11:24:32.881233+0000","last_undegraded":"2024-04-25T11:24:32.881233+0000","last_fullsized":"2024-04-25T11:24:32.881233+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:48.075466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.882690+0000","last_change":"2024-04-25T11:23:41.652322+0000","last_active":"2024-04-25T11:24:32.882690+0000","last_peered":"2024-04-25T11:24:32.882690+0000","last_clean":"2024-04-25T11:24:32.882690+0000","last_became_active":"2024-04-25T11:23:41.651681+0000","last_became_peered":"2024-04-25T11:23:41.651681+0000","last_unstale":"2024-04-25T11:24:32.882690+0000","last_undegraded":"2024-04-25T11:24:32.882690+0000","last_fullsized":"2024-04-25T11:24:32.882690+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:17.574861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.877034+0000","last_change":"2024-04-25T11:23:41.651701+0000","last_active":"2024-04-25T11:24:32.877034+0000","last_peered":"2024-04-25T11:24:32.877034+0000","last_clean":"2024-04-25T11:24:32.877034+0000","last_became_active":"2024-04-25T11:23:41.651362+0000","last_became_peered":"2024-04-25T11:23:41.651362+0000","last_unstale":"2024-04-25T11:24:32.877034+0000","last_undegraded":"2024-04-25T11:24:32.877034+0000","last_fullsized":"2024-04-25T11:24:32.877034+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:50.777396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.878668+0000","last_change":"2024-04-25T11:23:41.640385+0000","last_active":"2024-04-25T11:24:32.878668+0000","last_peered":"2024-04-25T11:24:32.878668+0000","last_clean":"2024-04-25T11:24:32.878668+0000","last_became_active":"2024-04-25T11:23:41.639986+0000","last_became_peered":"2024-04-25T11:23:41.639986+0000","last_unstale":"2024-04-25T11:24:32.878668+0000","last_undegraded":"2024-04-25T11:24:32.878668+0000","last_fullsized":"2024-04-25T11:24:32.878668+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:13.376858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.882697+0000","last_change":"2024-04-25T11:23:41.648075+0000","last_active":"2024-04-25T11:24:32.882697+0000","last_peered":"2024-04-25T11:24:32.882697+0000","last_clean":"2024-04-25T11:24:32.882697+0000","last_became_active":"2024-04-25T11:23:41.647681+0000","last_became_peered":"2024-04-25T11:23:41.647681+0000","last_unstale":"2024-04-25T11:24:32.882697+0000","last_undegraded":"2024-04-25T11:24:32.882697+0000","last_fullsized":"2024-04-25T11:24:32.882697+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:44.916762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.880753+0000","last_change":"2024-04-25T11:23:41.641788+0000","last_active":"2024-04-25T11:24:32.880753+0000","last_peered":"2024-04-25T11:24:32.880753+0000","last_clean":"2024-04-25T11:24:32.880753+0000","last_became_active":"2024-04-25T11:23:41.641407+0000","last_became_peered":"2024-04-25T11:23:41.641407+0000","last_unstale":"2024-04-25T11:24:32.880753+0000","last_undegraded":"2024-04-25T11:24:32.880753+0000","last_fullsized":"2024-04-25T11:24:32.880753+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:33.609749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.885751+0000","last_change":"2024-04-25T11:23:41.647616+0000","last_active":"2024-04-25T11:24:32.885751+0000","last_peered":"2024-04-25T11:24:32.885751+0000","last_clean":"2024-04-25T11:24:32.885751+0000","last_became_active":"2024-04-25T11:23:41.645396+0000","last_became_peered":"2024-04-25T11:23:41.645396+0000","last_unstale":"2024-04-25T11:24:32.885751+0000","last_undegraded":"2024-04-25T11:24:32.885751+0000","last_fullsized":"2024-04-25T11:24:32.885751+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:47.842574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.880114+0000","last_change":"2024-04-25T11:23:41.638407+0000","last_active":"2024-04-25T11:24:32.880114+0000","last_peered":"2024-04-25T11:24:32.880114+0000","last_clean":"2024-04-25T11:24:32.880114+0000","last_became_active":"2024-04-25T11:23:41.638054+0000","last_became_peered":"2024-04-25T11:23:41.638054+0000","last_unstale":"2024-04-25T11:24:32.880114+0000","last_undegraded":"2024-04-25T11:24:32.880114+0000","last_fullsized":"2024-04-25T11:24:32.880114+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:34.306493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.882645+0000","last_change":"2024-04-25T11:23:41.650140+0000","last_active":"2024-04-25T11:24:32.882645+0000","last_peered":"2024-04-25T11:24:32.882645+0000","last_clean":"2024-04-25T11:24:32.882645+0000","last_became_active":"2024-04-25T11:23:41.649487+0000","last_became_peered":"2024-04-25T11:23:41.649487+0000","last_unstale":"2024-04-25T11:24:32.882645+0000","last_undegraded":"2024-04-25T11:24:32.882645+0000","last_fullsized":"2024-04-25T11:24:32.882645+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:55.652921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.876863+0000","last_change":"2024-04-25T11:23:41.640253+0000","last_active":"2024-04-25T11:24:32.876863+0000","last_peered":"2024-04-25T11:24:32.876863+0000","last_clean":"2024-04-25T11:24:32.876863+0000","last_became_active":"2024-04-25T11:23:41.639926+0000","last_became_peered":"2024-04-25T11:23:41.639926+0000","last_unstale":"2024-04-25T11:24:32.876863+0000","last_undegraded":"2024-04-25T11:24:32.876863+0000","last_fullsized":"2024-04-25T11:24:32.876863+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:01.407198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T11:24:32.878973+0000","last_change":"2024-04-25T11:23:41.653244+0000","last_active":"2024-04-25T11:24:32.878973+0000","last_peered":"2024-04-25T11:24:32.878973+0000","last_clean":"2024-04-25T11:24:32.878973+0000","last_became_active":"2024-04-25T11:23:41.652875+0000","last_became_peered":"2024-04-25T11:23:41.652875+0000","last_unstale":"2024-04-25T11:24:32.878973+0000","last_undegraded":"2024-04-25T11:24:32.878973+0000","last_fullsized":"2024-04-25T11:24:32.878973+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:23:40.609743+0000","last_clean_scrub_stamp":"2024-04-25T11:23:40.609743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:06.897766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68394,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":764,"num_read_kb":470,"num_write":241,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1707,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":205182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710754,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":458067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906983,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1626112,"data_stored":1104087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070509,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":962560,"data_stored":460107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266737,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":950272,"data_stored":455807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462966,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28536,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716616,"statfs":{"total":95995035648,"available":95965814784,"internally_reserved":0,"allocated":1552384,"data_stored":1052886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626493,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":933888,"data_stored":451515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":513735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":958464,"data_stored":453362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":514625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182475,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":946176,"data_stored":451223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986247,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":455701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822721,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36860,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708292,"statfs":{"total":95995035648,"available":95957291008,"internally_reserved":0,"allocated":1552384,"data_stored":1047455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:25:26.830 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T11:25:26.830 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T11:25:26.830 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-25T11:25:26.831 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-25T11:25:27.077 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:25:27.192 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:26 smithi069 ceph-mon[29188]: pgmap v339: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:25:27.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:26 smithi152 ceph-mon[32701]: pgmap v339: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:25:27.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:26 smithi146 ceph-mon[32821]: pgmap v339: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:25:28.192 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:27 smithi069 ceph-mon[29188]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T11:25:28.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:27 smithi152 ceph-mon[32701]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T11:25:28.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:27 smithi146 ceph-mon[32821]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T11:25:28.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:28.906+0000 7fa0dc86f700 1 -- 172.21.15.69:0/537368532 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0d810e790 msgr2=0x7fa0d8110c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:28.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:28.906+0000 7fa0dc86f700 1 --2- 172.21.15.69:0/537368532 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0d810e790 0x7fa0d8110c70 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa0d000b0a0 tx=0x7fa0d000b3b0 comp rx=0 tx=0).stop 2024-04-25T11:25:28.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:28.906+0000 7fa0dc86f700 1 -- 172.21.15.69:0/537368532 shutdown_connections 2024-04-25T11:25:28.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:28.906+0000 7fa0dc86f700 1 --2- 172.21.15.69:0/537368532 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0d810e790 0x7fa0d8110c70 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:28.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:28.906+0000 7fa0dc86f700 1 --2- 172.21.15.69:0/537368532 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0d8105360 0x7fa0d8105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:28.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:28.906+0000 7fa0dc86f700 1 --2- 172.21.15.69:0/537368532 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0d8105cb0 0x7fa0d810e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:28.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:28.906+0000 7fa0dc86f700 1 -- 172.21.15.69:0/537368532 >> 172.21.15.69:0/537368532 conn(0x7fa0d80fec90 msgr2=0x7fa0d81010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:28.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:28.906+0000 7fa0dc86f700 1 -- 172.21.15.69:0/537368532 shutdown_connections 2024-04-25T11:25:28.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:28.906+0000 7fa0dc86f700 1 -- 172.21.15.69:0/537368532 wait complete. 2024-04-25T11:25:28.900 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:28 smithi069 ceph-mon[29188]: pgmap v340: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:25:28.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:28.907+0000 7fa0dc86f700 1 Processor -- start 2024-04-25T11:25:28.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:28.907+0000 7fa0dc86f700 1 -- start start 2024-04-25T11:25:28.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:28.907+0000 7fa0dc86f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0d8105360 0x7fa0d81a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:28.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:28.907+0000 7fa0dc86f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0d8105cb0 0x7fa0d81a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:28.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:28.907+0000 7fa0dc86f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0d810e790 0x7fa0d819a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:28.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:28.908+0000 7fa0dc86f700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d81141a0 con 0x7fa0d8105cb0 2024-04-25T11:25:28.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:28.908+0000 7fa0dc86f700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d8114020 con 0x7fa0d810e790 2024-04-25T11:25:28.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:28.908+0000 7fa0dc86f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d8114320 con 0x7fa0d8105360 2024-04-25T11:25:28.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:28.908+0000 7fa0d5d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0d8105cb0 0x7fa0d81a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:28.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:28.908+0000 7fa0d5d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0d8105cb0 0x7fa0d81a0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34346/0 (socket says 172.21.15.69:34346) 2024-04-25T11:25:28.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:28.908+0000 7fa0d5d9b700 1 -- 172.21.15.69:0/2011038626 learned_addr learned my addr 172.21.15.69:0/2011038626 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:28.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:28.908+0000 7fa0d659c700 1 --2- 172.21.15.69:0/2011038626 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0d8105360 0x7fa0d81a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:28.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:28.908+0000 7fa0d6d9d700 1 --2- 172.21.15.69:0/2011038626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0d810e790 0x7fa0d819a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:28.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:28.908+0000 7fa0d5d9b700 1 -- 172.21.15.69:0/2011038626 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0d8105360 msgr2=0x7fa0d81a0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:28.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:28.908+0000 7fa0d5d9b700 1 --2- 172.21.15.69:0/2011038626 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0d8105360 0x7fa0d81a0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:28.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:28.908+0000 7fa0d5d9b700 1 -- 172.21.15.69:0/2011038626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0d810e790 msgr2=0x7fa0d819a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:28.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:28.908+0000 7fa0d5d9b700 1 --2- 172.21.15.69:0/2011038626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0d810e790 0x7fa0d819a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:28.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:28.908+0000 7fa0d5d9b700 1 -- 172.21.15.69:0/2011038626 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0d819aff0 con 0x7fa0d8105cb0 2024-04-25T11:25:28.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:28.908+0000 7fa0d6d9d700 1 --2- 172.21.15.69:0/2011038626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0d810e790 0x7fa0d819a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:25:28.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:28.908+0000 7fa0d5d9b700 1 --2- 172.21.15.69:0/2011038626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0d8105cb0 0x7fa0d81a0c60 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc00d950 tx=0x7fa0cc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:28.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:28.908+0000 7fa0c77fe700 1 -- 172.21.15.69:0/2011038626 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc016070 con 0x7fa0d8105cb0 2024-04-25T11:25:28.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:28.908+0000 7fa0dc86f700 1 -- 172.21.15.69:0/2011038626 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0d819b2e0 con 0x7fa0d8105cb0 2024-04-25T11:25:28.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:28.908+0000 7fa0c77fe700 1 -- 172.21.15.69:0/2011038626 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0cc009dd0 con 0x7fa0d8105cb0 2024-04-25T11:25:28.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:28.908+0000 7fa0dc86f700 1 -- 172.21.15.69:0/2011038626 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0d81a75b0 con 0x7fa0d8105cb0 2024-04-25T11:25:28.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:28.909+0000 7fa0c77fe700 1 -- 172.21.15.69:0/2011038626 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc00f3c0 con 0x7fa0d8105cb0 2024-04-25T11:25:28.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:28.910+0000 7fa0dc86f700 1 -- 172.21.15.69:0/2011038626 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0d8051190 con 0x7fa0d8105cb0 2024-04-25T11:25:28.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:28.911+0000 7fa0c77fe700 1 -- 172.21.15.69:0/2011038626 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fa0cc019070 con 0x7fa0d8105cb0 2024-04-25T11:25:28.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:28.912+0000 7fa0c77fe700 1 --2- 172.21.15.69:0/2011038626 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fa0c00afc10 0x7fa0c00b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:28.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:28.912+0000 7fa0d659c700 1 --2- 172.21.15.69:0/2011038626 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fa0c00afc10 0x7fa0c00b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:28.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:28.912+0000 7fa0c77fe700 1 -- 172.21.15.69:0/2011038626 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7fa0cc0df1b0 con 0x7fa0d8105cb0 2024-04-25T11:25:28.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:28.912+0000 7fa0d659c700 1 --2- 172.21.15.69:0/2011038626 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fa0c00afc10 0x7fa0c00b2120 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa0d819ba80 tx=0x7fa0c800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:28.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:28.915+0000 7fa0c77fe700 1 -- 172.21.15.69:0/2011038626 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa0cc0a78d0 con 0x7fa0d8105cb0 2024-04-25T11:25:29.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.162+0000 7fa0dc86f700 1 -- 172.21.15.69:0/2011038626 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fa0d802e510 con 0x7fa0d8105cb0 2024-04-25T11:25:29.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.162+0000 7fa0c77fe700 1 -- 172.21.15.69:0/2011038626 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fa0cc0a71d0 con 0x7fa0d8105cb0 2024-04-25T11:25:29.155 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:25:29.155 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T11:25:29.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.165+0000 7fa0dc86f700 1 -- 172.21.15.69:0/2011038626 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fa0c00afc10 msgr2=0x7fa0c00b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:29.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.165+0000 7fa0dc86f700 1 --2- 172.21.15.69:0/2011038626 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fa0c00afc10 0x7fa0c00b2120 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa0d819ba80 tx=0x7fa0c800b410 comp rx=0 tx=0).stop 2024-04-25T11:25:29.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.165+0000 7fa0dc86f700 1 -- 172.21.15.69:0/2011038626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0d8105cb0 msgr2=0x7fa0d81a0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:29.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.165+0000 7fa0dc86f700 1 --2- 172.21.15.69:0/2011038626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0d8105cb0 0x7fa0d81a0c60 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc00d950 tx=0x7fa0cc00dc60 comp rx=0 tx=0).stop 2024-04-25T11:25:29.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.165+0000 7fa0dc86f700 1 -- 172.21.15.69:0/2011038626 shutdown_connections 2024-04-25T11:25:29.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.165+0000 7fa0dc86f700 1 --2- 172.21.15.69:0/2011038626 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fa0c00afc10 0x7fa0c00b2120 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:29.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.165+0000 7fa0dc86f700 1 --2- 172.21.15.69:0/2011038626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0d810e790 0x7fa0d819a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:29.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.165+0000 7fa0dc86f700 1 --2- 172.21.15.69:0/2011038626 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0d8105360 0x7fa0d81a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:29.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.165+0000 7fa0dc86f700 1 --2- 172.21.15.69:0/2011038626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0d8105cb0 0x7fa0d81a0c60 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:29.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.165+0000 7fa0dc86f700 1 -- 172.21.15.69:0/2011038626 >> 172.21.15.69:0/2011038626 conn(0x7fa0d80fec90 msgr2=0x7fa0d81009f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:29.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.165+0000 7fa0dc86f700 1 -- 172.21.15.69:0/2011038626 shutdown_connections 2024-04-25T11:25:29.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.165+0000 7fa0dc86f700 1 -- 172.21.15.69:0/2011038626 wait complete. 2024-04-25T11:25:29.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:28 smithi152 ceph-mon[32701]: pgmap v340: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:25:29.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:28 smithi146 ceph-mon[32821]: pgmap v340: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:25:29.812 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-25T11:25:29.812 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:25:29.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.935+0000 7ffab6cf9640 1 -- 172.21.15.69:0/2973175682 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffab0102360 msgr2=0x7ffab01090f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:29.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.935+0000 7ffab6cf9640 1 --2- 172.21.15.69:0/2973175682 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffab0102360 0x7ffab01090f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c009900 tx=0x7ffa9c02a410 comp rx=0 tx=0).stop 2024-04-25T11:25:29.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.935+0000 7ffab6cf9640 1 -- 172.21.15.69:0/2973175682 shutdown_connections 2024-04-25T11:25:29.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.935+0000 7ffab6cf9640 1 --2- 172.21.15.69:0/2973175682 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffab0109780 0x7ffab010bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:29.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.935+0000 7ffab6cf9640 1 --2- 172.21.15.69:0/2973175682 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffab0102360 0x7ffab01090f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:29.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.935+0000 7ffab6cf9640 1 --2- 172.21.15.69:0/2973175682 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffab0101e60 0x7ffab0106b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:29.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.935+0000 7ffab6cf9640 1 -- 172.21.15.69:0/2973175682 >> 172.21.15.69:0/2973175682 conn(0x7ffab00fd370 msgr2=0x7ffab00ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:29.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.935+0000 7ffab6cf9640 1 -- 172.21.15.69:0/2973175682 shutdown_connections 2024-04-25T11:25:29.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.935+0000 7ffab6cf9640 1 -- 172.21.15.69:0/2973175682 wait complete. 2024-04-25T11:25:29.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.936+0000 7ffab6cf9640 1 Processor -- start 2024-04-25T11:25:29.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.936+0000 7ffab6cf9640 1 -- start start 2024-04-25T11:25:29.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.936+0000 7ffab6cf9640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffab0101e60 0x7ffab019a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:29.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.936+0000 7ffab6cf9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffab0102360 0x7ffab019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:29.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.936+0000 7ffab6cf9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffab0109780 0x7ffab01a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:29.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.936+0000 7ffab6cf9640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ffab010f4a0 con 0x7ffab0102360 2024-04-25T11:25:29.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.936+0000 7ffab6cf9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ffab010f320 con 0x7ffab0109780 2024-04-25T11:25:29.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.936+0000 7ffab6cf9640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ffab010f620 con 0x7ffab0101e60 2024-04-25T11:25:29.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.936+0000 7ffaaffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffab0102360 0x7ffab019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:29.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.936+0000 7ffaaffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffab0102360 0x7ffab019a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34358/0 (socket says 172.21.15.69:34358) 2024-04-25T11:25:29.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.936+0000 7ffaaffff640 1 -- 172.21.15.69:0/1730802718 learned_addr learned my addr 172.21.15.69:0/1730802718 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:29.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.936+0000 7ffab4a95640 1 --2- 172.21.15.69:0/1730802718 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffab0101e60 0x7ffab019a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:29.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.936+0000 7ffaaffff640 1 -- 172.21.15.69:0/1730802718 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffab0101e60 msgr2=0x7ffab019a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:29.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.936+0000 7ffab5296640 1 --2- 172.21.15.69:0/1730802718 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffab0109780 0x7ffab01a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:29.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.936+0000 7ffaaffff640 1 --2- 172.21.15.69:0/1730802718 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffab0101e60 0x7ffab019a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:29.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.936+0000 7ffaaffff640 1 -- 172.21.15.69:0/1730802718 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffab0109780 msgr2=0x7ffab01a15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:29.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.936+0000 7ffaaffff640 1 --2- 172.21.15.69:0/1730802718 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffab0109780 0x7ffab01a15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:29.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.936+0000 7ffaaffff640 1 -- 172.21.15.69:0/1730802718 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffab01a1c40 con 0x7ffab0102360 2024-04-25T11:25:29.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.936+0000 7ffaaffff640 1 --2- 172.21.15.69:0/1730802718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffab0102360 0x7ffab019a640 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c009230 tx=0x7ffa9c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:29.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.937+0000 7ffaadffb640 1 -- 172.21.15.69:0/1730802718 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c03f030 con 0x7ffab0102360 2024-04-25T11:25:29.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.937+0000 7ffaadffb640 1 -- 172.21.15.69:0/1730802718 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffa9c0043d0 con 0x7ffab0102360 2024-04-25T11:25:29.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.937+0000 7ffaadffb640 1 -- 172.21.15.69:0/1730802718 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c035070 con 0x7ffab0102360 2024-04-25T11:25:29.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.937+0000 7ffab6cf9640 1 -- 172.21.15.69:0/1730802718 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffab01a1ed0 con 0x7ffab0102360 2024-04-25T11:25:29.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.937+0000 7ffab6cf9640 1 -- 172.21.15.69:0/1730802718 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffab01a2330 con 0x7ffab0102360 2024-04-25T11:25:29.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.938+0000 7ffab6cf9640 1 -- 172.21.15.69:0/1730802718 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffab00fee90 con 0x7ffab0102360 2024-04-25T11:25:29.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.940+0000 7ffaadffb640 1 -- 172.21.15.69:0/1730802718 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7ffa9c042030 con 0x7ffab0102360 2024-04-25T11:25:29.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.941+0000 7ffaadffb640 1 --2- 172.21.15.69:0/1730802718 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ffaa00afc00 0x7ffaa00b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:29.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.941+0000 7ffab4a95640 1 --2- 172.21.15.69:0/1730802718 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ffaa00afc00 0x7ffaa00b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:29.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.941+0000 7ffaadffb640 1 -- 172.21.15.69:0/1730802718 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7ffa9c0ffa30 con 0x7ffab0102360 2024-04-25T11:25:29.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.941+0000 7ffab4a95640 1 --2- 172.21.15.69:0/1730802718 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ffaa00afc00 0x7ffaa00b2110 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4009aa0 tx=0x7ffaa4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:29.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:29.944+0000 7ffaadffb640 1 -- 172.21.15.69:0/1730802718 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffa9c0c8150 con 0x7ffab0102360 2024-04-25T11:25:30.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.122+0000 7ffab6cf9640 1 -- 172.21.15.69:0/1730802718 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffab00637a0 con 0x7ffab0102360 2024-04-25T11:25:30.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.123+0000 7ffaadffb640 1 -- 172.21.15.69:0/1730802718 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+24658 (secure 0 0 0) 0x7ffa9c0c7a50 con 0x7ffab0102360 2024-04-25T11:25:30.116 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:25:30.117 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":82,"fsid":"f7ed73dc-02f4-11ef-bc93-c7b262605968","created":"2024-04-25T11:15:37.965281+0000","modified":"2024-04-25T11:24:32.872870+0000","last_up_change":"2024-04-25T11:22:50.691869+0000","last_in_change":"2024-04-25T11:22:30.271959+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:18:57.781263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:23:38.685110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:23:39.943803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T11:23:43.650521+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6a0d179-4998-4754-bf82-2ea3dac54e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6803","nonce":3756025235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6805","nonce":3756025235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6809","nonce":3756025235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6807","nonce":3756025235}]},"public_addr":"172.21.15.69:6803/3756025235","cluster_addr":"172.21.15.69:6805/3756025235","heartbeat_back_addr":"172.21.15.69:6809/3756025235","heartbeat_front_addr":"172.21.15.69:6807/3756025235","state":["exists","up"]},{"osd":1,"uuid":"c0c66930-4233-493f-b2bb-99c473994b59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6801","nonce":3166305213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6803","nonce":3166305213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6807","nonce":3166305213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6805","nonce":3166305213}]},"public_addr":"172.21.15.146:6801/3166305213","cluster_addr":"172.21.15.146:6803/3166305213","heartbeat_back_addr":"172.21.15.146:6807/3166305213","heartbeat_front_addr":"172.21.15.146:6805/3166305213","state":["exists","up"]},{"osd":2,"uuid":"d4ef735b-9786-4cb9-83ef-7c3f0691a729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6801","nonce":1474319175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6803","nonce":1474319175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6807","nonce":1474319175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6805","nonce":1474319175}]},"public_addr":"172.21.15.152:6801/1474319175","cluster_addr":"172.21.15.152:6803/1474319175","heartbeat_back_addr":"172.21.15.152:6807/1474319175","heartbeat_front_addr":"172.21.15.152:6805/1474319175","state":["exists","up"]},{"osd":3,"uuid":"0ab85fb8-5bbb-4ffc-bddf-cae7644a64ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6811","nonce":1809071296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6813","nonce":1809071296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6817","nonce":1809071296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6815","nonce":1809071296}]},"public_addr":"172.21.15.69:6811/1809071296","cluster_addr":"172.21.15.69:6813/1809071296","heartbeat_back_addr":"172.21.15.69:6817/1809071296","heartbeat_front_addr":"172.21.15.69:6815/1809071296","state":["exists","up"]},{"osd":4,"uuid":"273e0592-bbbc-4188-8eeb-46a3879cf632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6809","nonce":1898520972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6811","nonce":1898520972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6815","nonce":1898520972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6813","nonce":1898520972}]},"public_addr":"172.21.15.146:6809/1898520972","cluster_addr":"172.21.15.146:6811/1898520972","heartbeat_back_addr":"172.21.15.146:6815/1898520972","heartbeat_front_addr":"172.21.15.146:6813/1898520972","state":["exists","up"]},{"osd":5,"uuid":"bbeae945-9fa2-48f9-a1d6-8bf18dcaef19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6809","nonce":1996555108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6811","nonce":1996555108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6815","nonce":1996555108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6813","nonce":1996555108}]},"public_addr":"172.21.15.152:6809/1996555108","cluster_addr":"172.21.15.152:6811/1996555108","heartbeat_back_addr":"172.21.15.152:6815/1996555108","heartbeat_front_addr":"172.21.15.152:6813/1996555108","state":["exists","up"]},{"osd":6,"uuid":"dbb61253-30bc-418e-bd75-118c80bbf931","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6819","nonce":491215762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6821","nonce":491215762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6825","nonce":491215762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6823","nonce":491215762}]},"public_addr":"172.21.15.69:6819/491215762","cluster_addr":"172.21.15.69:6821/491215762","heartbeat_back_addr":"172.21.15.69:6825/491215762","heartbeat_front_addr":"172.21.15.69:6823/491215762","state":["exists","up"]},{"osd":7,"uuid":"75f23854-e283-4e3e-94cf-7b4079d83876","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6817","nonce":2330294690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6819","nonce":2330294690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6823","nonce":2330294690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6821","nonce":2330294690}]},"public_addr":"172.21.15.146:6817/2330294690","cluster_addr":"172.21.15.146:6819/2330294690","heartbeat_back_addr":"172.21.15.146:6823/2330294690","heartbeat_front_addr":"172.21.15.146:6821/2330294690","state":["exists","up"]},{"osd":8,"uuid":"1aa1060c-0c14-4abd-be56-5ebfeb78759c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6817","nonce":950669381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6819","nonce":950669381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6823","nonce":950669381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6821","nonce":950669381}]},"public_addr":"172.21.15.152:6817/950669381","cluster_addr":"172.21.15.152:6819/950669381","heartbeat_back_addr":"172.21.15.152:6823/950669381","heartbeat_front_addr":"172.21.15.152:6821/950669381","state":["exists","up"]},{"osd":9,"uuid":"3f4bf800-4090-4138-9bb3-d001f13f450c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6827","nonce":2414186758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6829","nonce":2414186758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6833","nonce":2414186758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6831","nonce":2414186758}]},"public_addr":"172.21.15.69:6827/2414186758","cluster_addr":"172.21.15.69:6829/2414186758","heartbeat_back_addr":"172.21.15.69:6833/2414186758","heartbeat_front_addr":"172.21.15.69:6831/2414186758","state":["exists","up"]},{"osd":10,"uuid":"19e967e5-51ce-4506-80c9-0a95441baf99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6825","nonce":707256400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6827","nonce":707256400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6831","nonce":707256400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6829","nonce":707256400}]},"public_addr":"172.21.15.146:6825/707256400","cluster_addr":"172.21.15.146:6827/707256400","heartbeat_back_addr":"172.21.15.146:6831/707256400","heartbeat_front_addr":"172.21.15.146:6829/707256400","state":["exists","up"]},{"osd":11,"uuid":"7788c2de-6930-4808-be0f-f1fd1ae0cb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6825","nonce":2934336833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6827","nonce":2934336833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6831","nonce":2934336833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6829","nonce":2934336833}]},"public_addr":"172.21.15.152:6825/2934336833","cluster_addr":"172.21.15.152:6827/2934336833","heartbeat_back_addr":"172.21.15.152:6831/2934336833","heartbeat_front_addr":"172.21.15.152:6829/2934336833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:07.497448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:31.227494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:55.027978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:20.713437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:46.214704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:20:11.199983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:20:37.177070+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:02.951240+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:28.963680+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:56.095807+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:22:22.221744+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:22:48.929334+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1b","mappings":[{"from":5,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/1372727546":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6801/1097124842":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6800/1508198215":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3103465990":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3540132960":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3899912190":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6800/1097124842":"2024-04-26T11:16:31.688159+0000","172.21.15.69:0/1199907202":"2024-04-26T11:16:04.860792+0000","172.21.15.69:6801/1508198215":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/2879102529":"2024-04-26T11:16:31.688159+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:25:30.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.125+0000 7ffab6cf9640 1 -- 172.21.15.69:0/1730802718 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ffaa00afc00 msgr2=0x7ffaa00b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:30.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.125+0000 7ffab6cf9640 1 --2- 172.21.15.69:0/1730802718 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ffaa00afc00 0x7ffaa00b2110 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4009aa0 tx=0x7ffaa4005f90 comp rx=0 tx=0).stop 2024-04-25T11:25:30.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.125+0000 7ffab6cf9640 1 -- 172.21.15.69:0/1730802718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffab0102360 msgr2=0x7ffab019a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:30.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.125+0000 7ffab6cf9640 1 --2- 172.21.15.69:0/1730802718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffab0102360 0x7ffab019a640 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c009230 tx=0x7ffa9c005e80 comp rx=0 tx=0).stop 2024-04-25T11:25:30.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.126+0000 7ffab6cf9640 1 -- 172.21.15.69:0/1730802718 shutdown_connections 2024-04-25T11:25:30.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.126+0000 7ffab6cf9640 1 --2- 172.21.15.69:0/1730802718 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ffaa00afc00 0x7ffaa00b2110 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:30.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.126+0000 7ffab6cf9640 1 --2- 172.21.15.69:0/1730802718 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffab0109780 0x7ffab01a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:30.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.126+0000 7ffab6cf9640 1 --2- 172.21.15.69:0/1730802718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffab0102360 0x7ffab019a640 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:30.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.126+0000 7ffab6cf9640 1 --2- 172.21.15.69:0/1730802718 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffab0101e60 0x7ffab019a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:30.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.126+0000 7ffab6cf9640 1 -- 172.21.15.69:0/1730802718 >> 172.21.15.69:0/1730802718 conn(0x7ffab00fd370 msgr2=0x7ffab0106750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:30.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.126+0000 7ffab6cf9640 1 -- 172.21.15.69:0/1730802718 shutdown_connections 2024-04-25T11:25:30.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.126+0000 7ffab6cf9640 1 -- 172.21.15.69:0/1730802718 wait complete. 2024-04-25T11:25:30.129 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:29 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2011038626' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T11:25:30.130 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:18:57.781263+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:23:38.685110+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:23:39.943803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T11:23:43.650521+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T11:25:30.131 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:25:30.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:29 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/2011038626' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T11:25:30.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.249+0000 7f7818940640 1 -- 172.21.15.69:0/887409471 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7814103c40 msgr2=0x7f7814104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:30.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.249+0000 7f7818940640 1 --2- 172.21.15.69:0/887409471 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7814103c40 0x7f7814104050 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f77fc009900 tx=0x7f77fc02a410 comp rx=0 tx=0).stop 2024-04-25T11:25:30.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.249+0000 7f7818940640 1 -- 172.21.15.69:0/887409471 shutdown_connections 2024-04-25T11:25:30.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.249+0000 7f7818940640 1 --2- 172.21.15.69:0/887409471 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7814108940 0x7f781410ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:30.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.249+0000 7f7818940640 1 --2- 172.21.15.69:0/887409471 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7814103c40 0x7f7814104050 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:30.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.249+0000 7f7818940640 1 --2- 172.21.15.69:0/887409471 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7814102a00 0x7f7814103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:30.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.249+0000 7f7818940640 1 -- 172.21.15.69:0/887409471 >> 172.21.15.69:0/887409471 conn(0x7f78140fdbf0 msgr2=0x7f78141000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:30.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.249+0000 7f7818940640 1 -- 172.21.15.69:0/887409471 shutdown_connections 2024-04-25T11:25:30.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.249+0000 7f7818940640 1 -- 172.21.15.69:0/887409471 wait complete. 2024-04-25T11:25:30.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.250+0000 7f7818940640 1 Processor -- start 2024-04-25T11:25:30.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.250+0000 7f7818940640 1 -- start start 2024-04-25T11:25:30.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.250+0000 7f7818940640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7814102a00 0x7f7814075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:30.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.250+0000 7f7818940640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7814103c40 0x7f781419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:30.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.250+0000 7f7818940640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7814108940 0x7f78141a1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:30.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.250+0000 7f7818940640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f781410e3b0 con 0x7f7814102a00 2024-04-25T11:25:30.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.250+0000 7f7818940640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f781410e230 con 0x7f7814108940 2024-04-25T11:25:30.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.250+0000 7f7818940640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f781410e530 con 0x7f7814103c40 2024-04-25T11:25:30.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.250+0000 7f781259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7814102a00 0x7f7814075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:30.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.250+0000 7f781259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7814102a00 0x7f7814075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34370/0 (socket says 172.21.15.69:34370) 2024-04-25T11:25:30.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.250+0000 7f781259c640 1 -- 172.21.15.69:0/1594868144 learned_addr learned my addr 172.21.15.69:0/1594868144 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:30.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.250+0000 7f7812d9d640 1 --2- 172.21.15.69:0/1594868144 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7814108940 0x7f78141a1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:30.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.250+0000 7f780bfff640 1 --2- 172.21.15.69:0/1594868144 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7814103c40 0x7f781419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:30.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.250+0000 7f781259c640 1 -- 172.21.15.69:0/1594868144 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7814103c40 msgr2=0x7f781419ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:30.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.250+0000 7f781259c640 1 --2- 172.21.15.69:0/1594868144 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7814103c40 0x7f781419ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:30.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.250+0000 7f781259c640 1 -- 172.21.15.69:0/1594868144 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7814108940 msgr2=0x7f78141a1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:30.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.250+0000 7f781259c640 1 --2- 172.21.15.69:0/1594868144 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7814108940 0x7f78141a1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:30.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.250+0000 7f781259c640 1 -- 172.21.15.69:0/1594868144 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78141a1c00 con 0x7f7814102a00 2024-04-25T11:25:30.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.250+0000 7f780bfff640 1 --2- 172.21.15.69:0/1594868144 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7814103c40 0x7f781419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:25:30.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.250+0000 7f781259c640 1 --2- 172.21.15.69:0/1594868144 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7814102a00 0x7f7814075840 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f780400c970 tx=0x7f780400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:30.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.251+0000 7f780b7fe640 1 -- 172.21.15.69:0/1594868144 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7804004220 con 0x7f7814102a00 2024-04-25T11:25:30.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.251+0000 7f780b7fe640 1 -- 172.21.15.69:0/1594868144 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7804007e40 con 0x7f7814102a00 2024-04-25T11:25:30.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.251+0000 7f7818940640 1 -- 172.21.15.69:0/1594868144 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78141a1ef0 con 0x7f7814102a00 2024-04-25T11:25:30.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.251+0000 7f780b7fe640 1 -- 172.21.15.69:0/1594868144 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780400f450 con 0x7f7814102a00 2024-04-25T11:25:30.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.251+0000 7f7818940640 1 -- 172.21.15.69:0/1594868144 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78141a2350 con 0x7f7814102a00 2024-04-25T11:25:30.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.252+0000 7f7818940640 1 -- 172.21.15.69:0/1594868144 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f781406a190 con 0x7f7814102a00 2024-04-25T11:25:30.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.253+0000 7f780b7fe640 1 -- 172.21.15.69:0/1594868144 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f78040079b0 con 0x7f7814102a00 2024-04-25T11:25:30.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.254+0000 7f780b7fe640 1 --2- 172.21.15.69:0/1594868144 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f78000afb80 0x7f78000b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:30.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.254+0000 7f780bfff640 1 --2- 172.21.15.69:0/1594868144 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f78000afb80 0x7f78000b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:30.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.254+0000 7f780b7fe640 1 -- 172.21.15.69:0/1594868144 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f78040df1a0 con 0x7f7814102a00 2024-04-25T11:25:30.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.254+0000 7f780bfff640 1 --2- 172.21.15.69:0/1594868144 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f78000afb80 0x7f78000b2090 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f77fc0096c0 tx=0x7f77fc005de0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:30.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.257+0000 7f780b7fe640 1 -- 172.21.15.69:0/1594868144 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f781406a190 con 0x7f7814102a00 2024-04-25T11:25:30.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:29 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/2011038626' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T11:25:30.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.436+0000 7f7818940640 1 -- 172.21.15.69:0/1594868144 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f781402dbf0 con 0x7f7814102a00 2024-04-25T11:25:30.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.437+0000 7f780b7fe640 1 -- 172.21.15.69:0/1594868144 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f7804014030 con 0x7f7814102a00 2024-04-25T11:25:30.430 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-04-25T11:25:30.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.439+0000 7f7818940640 1 -- 172.21.15.69:0/1594868144 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f78000afb80 msgr2=0x7f78000b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:30.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.439+0000 7f7818940640 1 --2- 172.21.15.69:0/1594868144 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f78000afb80 0x7f78000b2090 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f77fc0096c0 tx=0x7f77fc005de0 comp rx=0 tx=0).stop 2024-04-25T11:25:30.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.439+0000 7f7818940640 1 -- 172.21.15.69:0/1594868144 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7814102a00 msgr2=0x7f7814075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:30.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.439+0000 7f7818940640 1 --2- 172.21.15.69:0/1594868144 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7814102a00 0x7f7814075840 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f780400c970 tx=0x7f780400ce30 comp rx=0 tx=0).stop 2024-04-25T11:25:30.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.439+0000 7f7818940640 1 -- 172.21.15.69:0/1594868144 shutdown_connections 2024-04-25T11:25:30.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.439+0000 7f7818940640 1 --2- 172.21.15.69:0/1594868144 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f78000afb80 0x7f78000b2090 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:30.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.439+0000 7f7818940640 1 --2- 172.21.15.69:0/1594868144 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7814108940 0x7f78141a1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:30.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.439+0000 7f7818940640 1 --2- 172.21.15.69:0/1594868144 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7814103c40 0x7f781419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:30.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.439+0000 7f7818940640 1 --2- 172.21.15.69:0/1594868144 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7814102a00 0x7f7814075840 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:30.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.439+0000 7f7818940640 1 -- 172.21.15.69:0/1594868144 >> 172.21.15.69:0/1594868144 conn(0x7f78140fdbf0 msgr2=0x7f7814104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:30.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.439+0000 7f7818940640 1 -- 172.21.15.69:0/1594868144 shutdown_connections 2024-04-25T11:25:30.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.439+0000 7f7818940640 1 -- 172.21.15.69:0/1594868144 wait complete. 2024-04-25T11:25:30.444 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:25:30.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.561+0000 7fe0f6b3f640 1 -- 172.21.15.69:0/3964029082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0f0106f40 msgr2=0x7fe0f0109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:30.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.561+0000 7fe0f6b3f640 1 --2- 172.21.15.69:0/3964029082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0f0106f40 0x7fe0f0109360 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4009900 tx=0x7fe0e402a410 comp rx=0 tx=0).stop 2024-04-25T11:25:30.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.562+0000 7fe0f6b3f640 1 -- 172.21.15.69:0/3964029082 shutdown_connections 2024-04-25T11:25:30.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.562+0000 7fe0f6b3f640 1 --2- 172.21.15.69:0/3964029082 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0f01098a0 0x7fe0f010bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:30.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.562+0000 7fe0f6b3f640 1 --2- 172.21.15.69:0/3964029082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0f0106f40 0x7fe0f0109360 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:30.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.562+0000 7fe0f6b3f640 1 --2- 172.21.15.69:0/3964029082 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0f01045e0 0x7fe0f0106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:30.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.562+0000 7fe0f6b3f640 1 -- 172.21.15.69:0/3964029082 >> 172.21.15.69:0/3964029082 conn(0x7fe0f00fdbc0 msgr2=0x7fe0f0100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:30.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.562+0000 7fe0f6b3f640 1 -- 172.21.15.69:0/3964029082 shutdown_connections 2024-04-25T11:25:30.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.562+0000 7fe0f6b3f640 1 -- 172.21.15.69:0/3964029082 wait complete. 2024-04-25T11:25:30.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.562+0000 7fe0f6b3f640 1 Processor -- start 2024-04-25T11:25:30.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.562+0000 7fe0f6b3f640 1 -- start start 2024-04-25T11:25:30.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.562+0000 7fe0f6b3f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0f01045e0 0x7fe0f019a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:30.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.562+0000 7fe0f6b3f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0f0106f40 0x7fe0f019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:30.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.562+0000 7fe0f6b3f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0f01098a0 0x7fe0f01a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:30.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.562+0000 7fe0f6b3f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f010f4b0 con 0x7fe0f01045e0 2024-04-25T11:25:30.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.562+0000 7fe0f6b3f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f010f330 con 0x7fe0f0106f40 2024-04-25T11:25:30.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.562+0000 7fe0f6b3f640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f010f630 con 0x7fe0f01098a0 2024-04-25T11:25:30.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.563+0000 7fe0f48db640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0f01045e0 0x7fe0f019a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:30.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.563+0000 7fe0f48db640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0f01045e0 0x7fe0f019a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34382/0 (socket says 172.21.15.69:34382) 2024-04-25T11:25:30.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.563+0000 7fe0f48db640 1 -- 172.21.15.69:0/1396399562 learned_addr learned my addr 172.21.15.69:0/1396399562 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:30.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.563+0000 7fe0effff640 1 --2- 172.21.15.69:0/1396399562 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0f0106f40 0x7fe0f019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:30.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.563+0000 7fe0f50dc640 1 --2- 172.21.15.69:0/1396399562 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0f01098a0 0x7fe0f01a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:30.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.563+0000 7fe0f48db640 1 -- 172.21.15.69:0/1396399562 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0f01098a0 msgr2=0x7fe0f01a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:30.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.563+0000 7fe0f48db640 1 --2- 172.21.15.69:0/1396399562 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0f01098a0 0x7fe0f01a1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:30.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.563+0000 7fe0f48db640 1 -- 172.21.15.69:0/1396399562 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0f0106f40 msgr2=0x7fe0f019a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:30.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.563+0000 7fe0f48db640 1 --2- 172.21.15.69:0/1396399562 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0f0106f40 0x7fe0f019a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:30.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.563+0000 7fe0f48db640 1 -- 172.21.15.69:0/1396399562 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0f01a1c90 con 0x7fe0f01045e0 2024-04-25T11:25:30.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.563+0000 7fe0f48db640 1 --2- 172.21.15.69:0/1396399562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0f01045e0 0x7fe0f019a150 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe0e000c970 tx=0x7fe0e000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:30.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.563+0000 7fe0edffb640 1 -- 172.21.15.69:0/1396399562 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e0007bb0 con 0x7fe0f01045e0 2024-04-25T11:25:30.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.563+0000 7fe0edffb640 1 -- 172.21.15.69:0/1396399562 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe0e0007d50 con 0x7fe0f01045e0 2024-04-25T11:25:30.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.563+0000 7fe0f6b3f640 1 -- 172.21.15.69:0/1396399562 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0f01a1f80 con 0x7fe0f01045e0 2024-04-25T11:25:30.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.563+0000 7fe0edffb640 1 -- 172.21.15.69:0/1396399562 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e0004e10 con 0x7fe0f01045e0 2024-04-25T11:25:30.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.564+0000 7fe0f6b3f640 1 -- 172.21.15.69:0/1396399562 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0f01a23e0 con 0x7fe0f01045e0 2024-04-25T11:25:30.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.564+0000 7fe0f6b3f640 1 -- 172.21.15.69:0/1396399562 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0f006a160 con 0x7fe0f01045e0 2024-04-25T11:25:30.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.566+0000 7fe0edffb640 1 -- 172.21.15.69:0/1396399562 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fe0e0005000 con 0x7fe0f01045e0 2024-04-25T11:25:30.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.566+0000 7fe0edffb640 1 --2- 172.21.15.69:0/1396399562 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fe0d80afb80 0x7fe0d80b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:30.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.566+0000 7fe0effff640 1 --2- 172.21.15.69:0/1396399562 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fe0d80afb80 0x7fe0d80b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:30.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.566+0000 7fe0edffb640 1 -- 172.21.15.69:0/1396399562 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7fe0e00df150 con 0x7fe0f01045e0 2024-04-25T11:25:30.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.567+0000 7fe0effff640 1 --2- 172.21.15.69:0/1396399562 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fe0d80afb80 0x7fe0d80b2090 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe0f019b1f0 tx=0x7fe0e4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:30.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.571+0000 7fe0edffb640 1 -- 172.21.15.69:0/1396399562 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe0e0014030 con 0x7fe0f01045e0 2024-04-25T11:25:30.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.749+0000 7fe0f6b3f640 1 -- 172.21.15.69:0/1396399562 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe0f002dbd0 con 0x7fe0f01045e0 2024-04-25T11:25:30.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.749+0000 7fe0edffb640 1 -- 172.21.15.69:0/1396399562 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) ==== 104+0+11 (secure 0 0 0) 0x7fe0e00a7220 con 0x7fe0f01045e0 2024-04-25T11:25:30.742 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-25T11:25:30.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.751+0000 7fe0f6b3f640 1 -- 172.21.15.69:0/1396399562 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fe0d80afb80 msgr2=0x7fe0d80b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:30.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.751+0000 7fe0f6b3f640 1 --2- 172.21.15.69:0/1396399562 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fe0d80afb80 0x7fe0d80b2090 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe0f019b1f0 tx=0x7fe0e4000bc0 comp rx=0 tx=0).stop 2024-04-25T11:25:30.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.751+0000 7fe0f6b3f640 1 -- 172.21.15.69:0/1396399562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0f01045e0 msgr2=0x7fe0f019a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:30.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.751+0000 7fe0f6b3f640 1 --2- 172.21.15.69:0/1396399562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0f01045e0 0x7fe0f019a150 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe0e000c970 tx=0x7fe0e000ce30 comp rx=0 tx=0).stop 2024-04-25T11:25:30.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.751+0000 7fe0f6b3f640 1 -- 172.21.15.69:0/1396399562 shutdown_connections 2024-04-25T11:25:30.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.751+0000 7fe0f6b3f640 1 --2- 172.21.15.69:0/1396399562 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fe0d80afb80 0x7fe0d80b2090 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:30.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.751+0000 7fe0f6b3f640 1 --2- 172.21.15.69:0/1396399562 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0f01098a0 0x7fe0f01a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:30.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.751+0000 7fe0f6b3f640 1 --2- 172.21.15.69:0/1396399562 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0f0106f40 0x7fe0f019a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:30.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.751+0000 7fe0f6b3f640 1 --2- 172.21.15.69:0/1396399562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0f01045e0 0x7fe0f019a150 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:30.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.751+0000 7fe0f6b3f640 1 -- 172.21.15.69:0/1396399562 >> 172.21.15.69:0/1396399562 conn(0x7fe0f00fdbc0 msgr2=0x7fe0f0106630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:30.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.751+0000 7fe0f6b3f640 1 -- 172.21.15.69:0/1396399562 shutdown_connections 2024-04-25T11:25:30.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.751+0000 7fe0f6b3f640 1 -- 172.21.15.69:0/1396399562 wait complete. 2024-04-25T11:25:30.755 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:25:30.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.874+0000 7fedd13a8640 1 -- 172.21.15.69:0/3320336998 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedcc1029e0 msgr2=0x7fedcc1036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:30.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.874+0000 7fedd13a8640 1 --2- 172.21.15.69:0/3320336998 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedcc1029e0 0x7fedcc1036e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fedbc009ca0 tx=0x7fedbc02a8c0 comp rx=0 tx=0).stop 2024-04-25T11:25:30.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.874+0000 7fedd13a8640 1 -- 172.21.15.69:0/3320336998 shutdown_connections 2024-04-25T11:25:30.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.874+0000 7fedd13a8640 1 --2- 172.21.15.69:0/3320336998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fedcc108920 0x7fedcc10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:30.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.874+0000 7fedd13a8640 1 --2- 172.21.15.69:0/3320336998 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fedcc103c20 0x7fedcc104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:30.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.874+0000 7fedd13a8640 1 --2- 172.21.15.69:0/3320336998 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedcc1029e0 0x7fedcc1036e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:30.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.874+0000 7fedd13a8640 1 -- 172.21.15.69:0/3320336998 >> 172.21.15.69:0/3320336998 conn(0x7fedcc0fdbd0 msgr2=0x7fedcc100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:30.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.874+0000 7fedd13a8640 1 -- 172.21.15.69:0/3320336998 shutdown_connections 2024-04-25T11:25:30.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.874+0000 7fedd13a8640 1 -- 172.21.15.69:0/3320336998 wait complete. 2024-04-25T11:25:30.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.874+0000 7fedd13a8640 1 Processor -- start 2024-04-25T11:25:30.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.874+0000 7fedd13a8640 1 -- start start 2024-04-25T11:25:30.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.875+0000 7fedd13a8640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedcc1029e0 0x7fedcc19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:30.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.875+0000 7fedd13a8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fedcc103c20 0x7fedcc19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:30.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.875+0000 7fedd13a8640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fedcc108920 0x7fedcc1a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:30.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.875+0000 7fedd13a8640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fedcc10f520 con 0x7fedcc1029e0 2024-04-25T11:25:30.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.875+0000 7fedd13a8640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fedcc10f3a0 con 0x7fedcc103c20 2024-04-25T11:25:30.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.875+0000 7fedd13a8640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fedcc10f6a0 con 0x7fedcc108920 2024-04-25T11:25:30.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.875+0000 7fedcaffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedcc1029e0 0x7fedcc19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:30.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.875+0000 7fedcaffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedcc1029e0 0x7fedcc19a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34398/0 (socket says 172.21.15.69:34398) 2024-04-25T11:25:30.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.875+0000 7fedcaffd640 1 -- 172.21.15.69:0/2319699479 learned_addr learned my addr 172.21.15.69:0/2319699479 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:30.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.875+0000 7fedca7fc640 1 --2- 172.21.15.69:0/2319699479 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fedcc103c20 0x7fedcc19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:30.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.875+0000 7fedcb7fe640 1 --2- 172.21.15.69:0/2319699479 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fedcc108920 0x7fedcc1a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:30.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.875+0000 7fedcaffd640 1 -- 172.21.15.69:0/2319699479 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fedcc108920 msgr2=0x7fedcc1a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:30.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.875+0000 7fedcaffd640 1 --2- 172.21.15.69:0/2319699479 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fedcc108920 0x7fedcc1a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:30.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.875+0000 7fedcaffd640 1 -- 172.21.15.69:0/2319699479 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fedcc103c20 msgr2=0x7fedcc19aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:30.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.875+0000 7fedcaffd640 1 --2- 172.21.15.69:0/2319699479 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fedcc103c20 0x7fedcc19aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:30.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.875+0000 7fedcaffd640 1 -- 172.21.15.69:0/2319699479 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedcc1a1f90 con 0x7fedcc1029e0 2024-04-25T11:25:30.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.875+0000 7fedcaffd640 1 --2- 172.21.15.69:0/2319699479 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedcc1029e0 0x7fedcc19a4c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fedbc004af0 tx=0x7fedbc003a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:30.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.875+0000 7fedb3fff640 1 -- 172.21.15.69:0/2319699479 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedbc004040 con 0x7fedcc1029e0 2024-04-25T11:25:30.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.875+0000 7fedb3fff640 1 -- 172.21.15.69:0/2319699479 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fedbc004670 con 0x7fedcc1029e0 2024-04-25T11:25:30.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.875+0000 7fedd13a8640 1 -- 172.21.15.69:0/2319699479 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedcc1a2220 con 0x7fedcc1029e0 2024-04-25T11:25:30.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.875+0000 7fedb3fff640 1 -- 172.21.15.69:0/2319699479 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedbc030ce0 con 0x7fedcc1029e0 2024-04-25T11:25:30.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.875+0000 7fedd13a8640 1 -- 172.21.15.69:0/2319699479 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedcc1a2770 con 0x7fedcc1029e0 2024-04-25T11:25:30.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.876+0000 7fedd13a8640 1 -- 172.21.15.69:0/2319699479 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedcc107730 con 0x7fedcc1029e0 2024-04-25T11:25:30.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.878+0000 7fedb3fff640 1 -- 172.21.15.69:0/2319699479 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7fedbc0041e0 con 0x7fedcc1029e0 2024-04-25T11:25:30.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.879+0000 7fedb3fff640 1 --2- 172.21.15.69:0/2319699479 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fedb40afb80 0x7fedb40b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:30.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.879+0000 7fedca7fc640 1 --2- 172.21.15.69:0/2319699479 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fedb40afb80 0x7fedb40b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:30.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.879+0000 7fedb3fff640 1 -- 172.21.15.69:0/2319699479 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7fedbc1007e0 con 0x7fedcc1029e0 2024-04-25T11:25:30.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.879+0000 7fedca7fc640 1 --2- 172.21.15.69:0/2319699479 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fedb40afb80 0x7fedb40b2090 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fedc00058c0 tx=0x7fedc000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:30.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:30.883+0000 7fedb3fff640 1 -- 172.21.15.69:0/2319699479 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fedbc0c9010 con 0x7fedcc1029e0 2024-04-25T11:25:31.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.061+0000 7fedd13a8640 1 -- 172.21.15.69:0/2319699479 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fedcc02dc50 con 0x7fedcc1029e0 2024-04-25T11:25:31.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.062+0000 7fedb3fff640 1 -- 172.21.15.69:0/2319699479 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) ==== 100+0+11 (secure 0 0 0) 0x7fedbc004820 con 0x7fedcc1029e0 2024-04-25T11:25:31.055 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-04-25T11:25:31.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.064+0000 7fedd13a8640 1 -- 172.21.15.69:0/2319699479 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fedb40afb80 msgr2=0x7fedb40b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:31.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.064+0000 7fedd13a8640 1 --2- 172.21.15.69:0/2319699479 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fedb40afb80 0x7fedb40b2090 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fedc00058c0 tx=0x7fedc000a2d0 comp rx=0 tx=0).stop 2024-04-25T11:25:31.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.064+0000 7fedd13a8640 1 -- 172.21.15.69:0/2319699479 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedcc1029e0 msgr2=0x7fedcc19a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:31.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.064+0000 7fedd13a8640 1 --2- 172.21.15.69:0/2319699479 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedcc1029e0 0x7fedcc19a4c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fedbc004af0 tx=0x7fedbc003a70 comp rx=0 tx=0).stop 2024-04-25T11:25:31.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.064+0000 7fedd13a8640 1 -- 172.21.15.69:0/2319699479 shutdown_connections 2024-04-25T11:25:31.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.064+0000 7fedd13a8640 1 --2- 172.21.15.69:0/2319699479 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fedb40afb80 0x7fedb40b2090 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:31.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.064+0000 7fedd13a8640 1 --2- 172.21.15.69:0/2319699479 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fedcc108920 0x7fedcc1a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:31.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.064+0000 7fedd13a8640 1 --2- 172.21.15.69:0/2319699479 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fedcc103c20 0x7fedcc19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:31.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.064+0000 7fedd13a8640 1 --2- 172.21.15.69:0/2319699479 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fedcc1029e0 0x7fedcc19a4c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:31.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.064+0000 7fedd13a8640 1 -- 172.21.15.69:0/2319699479 >> 172.21.15.69:0/2319699479 conn(0x7fedcc0fdbd0 msgr2=0x7fedcc104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:31.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.064+0000 7fedd13a8640 1 -- 172.21.15.69:0/2319699479 shutdown_connections 2024-04-25T11:25:31.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.064+0000 7fedd13a8640 1 -- 172.21.15.69:0/2319699479 wait complete. 2024-04-25T11:25:31.067 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:30 smithi069 ceph-mon[29188]: pgmap v341: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:25:31.067 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:30 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/1730802718' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:25:31.067 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:30 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/1594868144' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:25:31.067 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:30 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/1396399562' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:25:31.068 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T11:25:31.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.186+0000 7f38b9813640 1 -- 172.21.15.69:0/4270939544 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38b41029e0 msgr2=0x7f38b41036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:31.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.186+0000 7f38b9813640 1 --2- 172.21.15.69:0/4270939544 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38b41029e0 0x7f38b41036e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f38a4009900 tx=0x7f38a402a410 comp rx=0 tx=0).stop 2024-04-25T11:25:31.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.186+0000 7f38b9813640 1 -- 172.21.15.69:0/4270939544 shutdown_connections 2024-04-25T11:25:31.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.186+0000 7f38b9813640 1 --2- 172.21.15.69:0/4270939544 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38b4108920 0x7f38b410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:31.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.186+0000 7f38b9813640 1 --2- 172.21.15.69:0/4270939544 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38b4103c20 0x7f38b4104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:31.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.186+0000 7f38b9813640 1 --2- 172.21.15.69:0/4270939544 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38b41029e0 0x7f38b41036e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:31.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.186+0000 7f38b9813640 1 -- 172.21.15.69:0/4270939544 >> 172.21.15.69:0/4270939544 conn(0x7f38b40fdbd0 msgr2=0x7f38b4100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:31.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.186+0000 7f38b9813640 1 -- 172.21.15.69:0/4270939544 shutdown_connections 2024-04-25T11:25:31.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.186+0000 7f38b9813640 1 -- 172.21.15.69:0/4270939544 wait complete. 2024-04-25T11:25:31.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.187+0000 7f38b9813640 1 Processor -- start 2024-04-25T11:25:31.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.187+0000 7f38b9813640 1 -- start start 2024-04-25T11:25:31.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.187+0000 7f38b9813640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38b41029e0 0x7f38b419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:31.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.187+0000 7f38b9813640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38b4103c20 0x7f38b419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:31.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.187+0000 7f38b9813640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38b4108920 0x7f38b41a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:31.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.187+0000 7f38b9813640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f38b410f4c0 con 0x7f38b4108920 2024-04-25T11:25:31.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.187+0000 7f38b9813640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f38b410f340 con 0x7f38b4103c20 2024-04-25T11:25:31.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.187+0000 7f38b9813640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f38b410f640 con 0x7f38b41029e0 2024-04-25T11:25:31.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.187+0000 7f38b37fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38b4108920 0x7f38b41a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:31.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.187+0000 7f38b37fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38b4108920 0x7f38b41a1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34414/0 (socket says 172.21.15.69:34414) 2024-04-25T11:25:31.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.187+0000 7f38b37fe640 1 -- 172.21.15.69:0/2522833109 learned_addr learned my addr 172.21.15.69:0/2522833109 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:31.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.187+0000 7f38aa7fc640 1 --2- 172.21.15.69:0/2522833109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38b4103c20 0x7f38b419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:31.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.187+0000 7f38b2ffd640 1 --2- 172.21.15.69:0/2522833109 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38b41029e0 0x7f38b419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:31.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.188+0000 7f38b37fe640 1 -- 172.21.15.69:0/2522833109 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38b41029e0 msgr2=0x7f38b419a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:31.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.188+0000 7f38b37fe640 1 --2- 172.21.15.69:0/2522833109 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38b41029e0 0x7f38b419a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:31.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.188+0000 7f38b37fe640 1 -- 172.21.15.69:0/2522833109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38b4103c20 msgr2=0x7f38b419a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:31.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.188+0000 7f38b37fe640 1 --2- 172.21.15.69:0/2522833109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38b4103c20 0x7f38b419a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:31.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.188+0000 7f38b37fe640 1 -- 172.21.15.69:0/2522833109 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38b41a1fb0 con 0x7f38b4108920 2024-04-25T11:25:31.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.188+0000 7f38b37fe640 1 --2- 172.21.15.69:0/2522833109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38b4108920 0x7f38b41a1940 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f38ac00eaa0 tx=0x7f38ac00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:31.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.188+0000 7f38b8811640 1 -- 172.21.15.69:0/2522833109 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac00cdb0 con 0x7f38b4108920 2024-04-25T11:25:31.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.188+0000 7f38b8811640 1 -- 172.21.15.69:0/2522833109 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f38ac005020 con 0x7f38b4108920 2024-04-25T11:25:31.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.188+0000 7f38b9813640 1 -- 172.21.15.69:0/2522833109 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38b41a22a0 con 0x7f38b4108920 2024-04-25T11:25:31.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.188+0000 7f38b8811640 1 -- 172.21.15.69:0/2522833109 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac0116d0 con 0x7f38b4108920 2024-04-25T11:25:31.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.188+0000 7f38b9813640 1 -- 172.21.15.69:0/2522833109 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38b41a27f0 con 0x7f38b4108920 2024-04-25T11:25:31.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.189+0000 7f38b9813640 1 -- 172.21.15.69:0/2522833109 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38b4107730 con 0x7f38b4108920 2024-04-25T11:25:31.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.190+0000 7f38b8811640 1 -- 172.21.15.69:0/2522833109 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f38ac0051d0 con 0x7f38b4108920 2024-04-25T11:25:31.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.191+0000 7f38b8811640 1 --2- 172.21.15.69:0/2522833109 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f38a00afb80 0x7f38a00b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:31.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.191+0000 7f38b2ffd640 1 --2- 172.21.15.69:0/2522833109 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f38a00afb80 0x7f38a00b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:31.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.191+0000 7f38b8811640 1 -- 172.21.15.69:0/2522833109 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f38ac0df600 con 0x7f38b4108920 2024-04-25T11:25:31.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.191+0000 7f38b2ffd640 1 --2- 172.21.15.69:0/2522833109 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f38a00afb80 0x7f38a00b2090 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f38a4005760 tx=0x7f38a4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:31.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.195+0000 7f38b8811640 1 -- 172.21.15.69:0/2522833109 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f38ac016030 con 0x7f38b4108920 2024-04-25T11:25:31.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:30 smithi152 ceph-mon[32701]: pgmap v341: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:25:31.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:30 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/1730802718' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:25:31.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:30 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/1594868144' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:25:31.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:30 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/1396399562' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:25:31.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:30 smithi146 ceph-mon[32821]: pgmap v341: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:25:31.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:30 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/1730802718' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:25:31.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:30 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/1594868144' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:25:31.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:30 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/1396399562' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:25:31.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.372+0000 7f38b9813640 1 -- 172.21.15.69:0/2522833109 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f38b402dd30 con 0x7f38b4108920 2024-04-25T11:25:31.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.373+0000 7f38b8811640 1 -- 172.21.15.69:0/2522833109 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v82) ==== 103+0+11 (secure 0 0 0) 0x7f38ac0a76d0 con 0x7f38b4108920 2024-04-25T11:25:31.365 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-25T11:25:31.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.375+0000 7f38b9813640 1 -- 172.21.15.69:0/2522833109 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f38a00afb80 msgr2=0x7f38a00b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:31.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.375+0000 7f38b9813640 1 --2- 172.21.15.69:0/2522833109 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f38a00afb80 0x7f38a00b2090 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f38a4005760 tx=0x7f38a4000bc0 comp rx=0 tx=0).stop 2024-04-25T11:25:31.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.375+0000 7f38b9813640 1 -- 172.21.15.69:0/2522833109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38b4108920 msgr2=0x7f38b41a1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:31.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.375+0000 7f38b9813640 1 --2- 172.21.15.69:0/2522833109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38b4108920 0x7f38b41a1940 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f38ac00eaa0 tx=0x7f38ac00ef60 comp rx=0 tx=0).stop 2024-04-25T11:25:31.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.375+0000 7f38b9813640 1 -- 172.21.15.69:0/2522833109 shutdown_connections 2024-04-25T11:25:31.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.375+0000 7f38b9813640 1 --2- 172.21.15.69:0/2522833109 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f38a00afb80 0x7f38a00b2090 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:31.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.375+0000 7f38b9813640 1 --2- 172.21.15.69:0/2522833109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38b4108920 0x7f38b41a1940 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:31.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.375+0000 7f38b9813640 1 --2- 172.21.15.69:0/2522833109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38b4103c20 0x7f38b419a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:31.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.375+0000 7f38b9813640 1 --2- 172.21.15.69:0/2522833109 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38b41029e0 0x7f38b419a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:31.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.375+0000 7f38b9813640 1 -- 172.21.15.69:0/2522833109 >> 172.21.15.69:0/2522833109 conn(0x7f38b40fdbd0 msgr2=0x7f38b4104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:31.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.375+0000 7f38b9813640 1 -- 172.21.15.69:0/2522833109 shutdown_connections 2024-04-25T11:25:31.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.375+0000 7f38b9813640 1 -- 172.21.15.69:0/2522833109 wait complete. 2024-04-25T11:25:31.379 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:25:31.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.495+0000 7f22c7e6e640 1 -- 172.21.15.69:0/2626367394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22c0102f50 msgr2=0x7f22c0103360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:31.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.495+0000 7f22c7e6e640 1 --2- 172.21.15.69:0/2626367394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22c0102f50 0x7f22c0103360 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f22b0009900 tx=0x7f22b002a410 comp rx=0 tx=0).stop 2024-04-25T11:25:31.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.495+0000 7f22c7e6e640 1 -- 172.21.15.69:0/2626367394 shutdown_connections 2024-04-25T11:25:31.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.495+0000 7f22c7e6e640 1 --2- 172.21.15.69:0/2626367394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22c0104470 0x7f22c010abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:31.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.495+0000 7f22c7e6e640 1 --2- 172.21.15.69:0/2626367394 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22c0103930 0x7f22c0103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:31.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.495+0000 7f22c7e6e640 1 --2- 172.21.15.69:0/2626367394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22c0102f50 0x7f22c0103360 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:31.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.495+0000 7f22c7e6e640 1 -- 172.21.15.69:0/2626367394 >> 172.21.15.69:0/2626367394 conn(0x7f22c00fd3b0 msgr2=0x7f22c00ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:31.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.495+0000 7f22c7e6e640 1 -- 172.21.15.69:0/2626367394 shutdown_connections 2024-04-25T11:25:31.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.495+0000 7f22c7e6e640 1 -- 172.21.15.69:0/2626367394 wait complete. 2024-04-25T11:25:31.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.496+0000 7f22c7e6e640 1 Processor -- start 2024-04-25T11:25:31.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.496+0000 7f22c7e6e640 1 -- start start 2024-04-25T11:25:31.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.496+0000 7f22c7e6e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22c0102f50 0x7f22c019a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:31.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.496+0000 7f22c7e6e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22c0103930 0x7f22c019a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:31.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.496+0000 7f22c7e6e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22c0104470 0x7f22c01a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:31.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.496+0000 7f22c7e6e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f22c010e390 con 0x7f22c0102f50 2024-04-25T11:25:31.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.496+0000 7f22c7e6e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f22c010e210 con 0x7f22c0103930 2024-04-25T11:25:31.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.496+0000 7f22c7e6e640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f22c010e510 con 0x7f22c0104470 2024-04-25T11:25:31.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.496+0000 7f22c5c0a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22c0102f50 0x7f22c019a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:31.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.496+0000 7f22c5c0a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22c0102f50 0x7f22c019a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34426/0 (socket says 172.21.15.69:34426) 2024-04-25T11:25:31.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.496+0000 7f22c5c0a640 1 -- 172.21.15.69:0/3557655681 learned_addr learned my addr 172.21.15.69:0/3557655681 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:31.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.496+0000 7f22c5409640 1 --2- 172.21.15.69:0/3557655681 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22c0103930 0x7f22c019a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:31.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.496+0000 7f22c5c0a640 1 -- 172.21.15.69:0/3557655681 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22c0104470 msgr2=0x7f22c01a1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:31.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.496+0000 7f22c640b640 1 --2- 172.21.15.69:0/3557655681 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22c0104470 0x7f22c01a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:31.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.496+0000 7f22c5c0a640 1 --2- 172.21.15.69:0/3557655681 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22c0104470 0x7f22c01a1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:31.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.496+0000 7f22c5c0a640 1 -- 172.21.15.69:0/3557655681 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22c0103930 msgr2=0x7f22c019a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:31.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.496+0000 7f22c5c0a640 1 --2- 172.21.15.69:0/3557655681 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22c0103930 0x7f22c019a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:31.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.496+0000 7f22c5c0a640 1 -- 172.21.15.69:0/3557655681 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22c01a1c80 con 0x7f22c0102f50 2024-04-25T11:25:31.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.496+0000 7f22c5409640 1 --2- 172.21.15.69:0/3557655681 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22c0103930 0x7f22c019a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:25:31.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.496+0000 7f22c640b640 1 --2- 172.21.15.69:0/3557655681 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22c0104470 0x7f22c01a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:25:31.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.496+0000 7f22c5c0a640 1 --2- 172.21.15.69:0/3557655681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22c0102f50 0x7f22c019a140 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f22b0005760 tx=0x7f22b0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:31.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.497+0000 7f22b6ffd640 1 -- 172.21.15.69:0/3557655681 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b003f030 con 0x7f22c0102f50 2024-04-25T11:25:31.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.497+0000 7f22b6ffd640 1 -- 172.21.15.69:0/3557655681 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22b0003a80 con 0x7f22c0102f50 2024-04-25T11:25:31.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.497+0000 7f22c7e6e640 1 -- 172.21.15.69:0/3557655681 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22c01a1f10 con 0x7f22c0102f50 2024-04-25T11:25:31.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.497+0000 7f22b6ffd640 1 -- 172.21.15.69:0/3557655681 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b0035070 con 0x7f22c0102f50 2024-04-25T11:25:31.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.497+0000 7f22c7e6e640 1 -- 172.21.15.69:0/3557655681 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22c01a2320 con 0x7f22c0102f50 2024-04-25T11:25:31.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.498+0000 7f22c7e6e640 1 -- 172.21.15.69:0/3557655681 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22c01a2610 con 0x7f22c0102f50 2024-04-25T11:25:31.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.499+0000 7f22b6ffd640 1 -- 172.21.15.69:0/3557655681 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f22b0042030 con 0x7f22c0102f50 2024-04-25T11:25:31.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.499+0000 7f22b6ffd640 1 --2- 172.21.15.69:0/3557655681 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f22ac0afb80 0x7f22ac0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:31.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.500+0000 7f22c5409640 1 --2- 172.21.15.69:0/3557655681 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f22ac0afb80 0x7f22ac0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:31.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.500+0000 7f22b6ffd640 1 -- 172.21.15.69:0/3557655681 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f22b00ffcd0 con 0x7f22c0102f50 2024-04-25T11:25:31.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.500+0000 7f22c5409640 1 --2- 172.21.15.69:0/3557655681 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f22ac0afb80 0x7f22ac0b2090 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f22bc0058c0 tx=0x7f22bc005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:31.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.503+0000 7f22b6ffd640 1 -- 172.21.15.69:0/3557655681 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f22b00c83f0 con 0x7f22c0102f50 2024-04-25T11:25:31.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.719+0000 7f22c7e6e640 1 -- 172.21.15.69:0/3557655681 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22c01a27e0 con 0x7f22c0102f50 2024-04-25T11:25:31.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.719+0000 7f22b6ffd640 1 -- 172.21.15.69:0/3557655681 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11944 (secure 0 0 0) 0x7f22b00c7cf0 con 0x7f22c0102f50 2024-04-25T11:25:31.712 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:25:31.712 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3399424094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3399424094},{"type":"v1","addr":"172.21.15.69:6835","nonce":3399424094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1463217771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1463217771},{"type":"v1","addr":"172.21.15.69:6837","nonce":1463217771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T11:23:40.984930+0000","modified":"2024-04-25T11:24:09.061188+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24485,"mds_2":24475,"mds_3":24458,"mds_4":14625},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6839/966316001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":966316001},{"type":"v1","addr":"172.21.15.69:6839","nonce":966316001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.69:6841/2060225181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":2060225181},{"type":"v1","addr":"172.21.15.69:6841","nonce":2060225181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6833/4197482473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":4197482473},{"type":"v1","addr":"172.21.15.152:6833","nonce":4197482473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/4267885323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4267885323},{"type":"v1","addr":"172.21.15.146:6833","nonce":4267885323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6835/3188843362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3188843362},{"type":"v1","addr":"172.21.15.152:6835","nonce":3188843362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6837/563274982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":563274982},{"type":"v1","addr":"172.21.15.152:6837","nonce":563274982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.146:6835/1876533699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1876533699},{"type":"v1","addr":"172.21.15.146:6835","nonce":1876533699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.152:6839/3008615657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":3008615657},{"type":"v1","addr":"172.21.15.152:6839","nonce":3008615657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.146:6837/2730317134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2730317134},{"type":"v1","addr":"172.21.15.146:6837","nonce":2730317134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6839/2013283448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":2013283448},{"type":"v1","addr":"172.21.15.146:6839","nonce":2013283448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24485,24515,24475,24458,14625]},"id":1}]} 2024-04-25T11:25:31.713 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 28 2024-04-25T11:25:31.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.721+0000 7f22c7e6e640 1 -- 172.21.15.69:0/3557655681 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f22ac0afb80 msgr2=0x7f22ac0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:31.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.721+0000 7f22c7e6e640 1 --2- 172.21.15.69:0/3557655681 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f22ac0afb80 0x7f22ac0b2090 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f22bc0058c0 tx=0x7f22bc005d50 comp rx=0 tx=0).stop 2024-04-25T11:25:31.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.721+0000 7f22c7e6e640 1 -- 172.21.15.69:0/3557655681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22c0102f50 msgr2=0x7f22c019a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:31.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.721+0000 7f22c7e6e640 1 --2- 172.21.15.69:0/3557655681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22c0102f50 0x7f22c019a140 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f22b0005760 tx=0x7f22b0004270 comp rx=0 tx=0).stop 2024-04-25T11:25:31.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.721+0000 7f22c7e6e640 1 -- 172.21.15.69:0/3557655681 shutdown_connections 2024-04-25T11:25:31.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.721+0000 7f22c7e6e640 1 --2- 172.21.15.69:0/3557655681 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f22ac0afb80 0x7f22ac0b2090 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:31.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.721+0000 7f22c7e6e640 1 --2- 172.21.15.69:0/3557655681 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22c0104470 0x7f22c01a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:31.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.721+0000 7f22c7e6e640 1 --2- 172.21.15.69:0/3557655681 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22c0103930 0x7f22c019a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:31.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.721+0000 7f22c7e6e640 1 --2- 172.21.15.69:0/3557655681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22c0102f50 0x7f22c019a140 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:31.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.721+0000 7f22c7e6e640 1 -- 172.21.15.69:0/3557655681 >> 172.21.15.69:0/3557655681 conn(0x7f22c00fd3b0 msgr2=0x7f22c0109d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:31.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.721+0000 7f22c7e6e640 1 -- 172.21.15.69:0/3557655681 shutdown_connections 2024-04-25T11:25:31.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.722+0000 7f22c7e6e640 1 -- 172.21.15.69:0/3557655681 wait complete. 2024-04-25T11:25:31.726 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:25:31.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.865+0000 7f34517c0640 1 -- 172.21.15.69:0/1260948209 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f344c106c20 msgr2=0x7f344c109100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:31.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.865+0000 7f34517c0640 1 --2- 172.21.15.69:0/1260948209 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f344c106c20 0x7f344c109100 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3440009900 tx=0x7f344002a410 comp rx=0 tx=0).stop 2024-04-25T11:25:31.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.865+0000 7f34517c0640 1 -- 172.21.15.69:0/1260948209 shutdown_connections 2024-04-25T11:25:31.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.865+0000 7f34517c0640 1 --2- 172.21.15.69:0/1260948209 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f344c109790 0x7f344c10bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:31.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.865+0000 7f34517c0640 1 --2- 172.21.15.69:0/1260948209 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f344c106c20 0x7f344c109100 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:31.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.865+0000 7f34517c0640 1 --2- 172.21.15.69:0/1260948209 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f344c101e40 0x7f344c102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:31.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.865+0000 7f34517c0640 1 -- 172.21.15.69:0/1260948209 >> 172.21.15.69:0/1260948209 conn(0x7f344c0fd370 msgr2=0x7f344c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:31.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.865+0000 7f34517c0640 1 -- 172.21.15.69:0/1260948209 shutdown_connections 2024-04-25T11:25:31.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.865+0000 7f34517c0640 1 -- 172.21.15.69:0/1260948209 wait complete. 2024-04-25T11:25:31.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.866+0000 7f34517c0640 1 Processor -- start 2024-04-25T11:25:31.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.866+0000 7f34517c0640 1 -- start start 2024-04-25T11:25:31.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.866+0000 7f34517c0640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f344c101e40 0x7f344c19a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:31.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.866+0000 7f34517c0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f344c106c20 0x7f344c19a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:31.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.866+0000 7f34517c0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f344c109790 0x7f344c1a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:31.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.866+0000 7f34517c0640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f344c10f5a0 con 0x7f344c106c20 2024-04-25T11:25:31.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.866+0000 7f34517c0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f344c10f420 con 0x7f344c109790 2024-04-25T11:25:31.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.866+0000 7f34517c0640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f344c10f720 con 0x7f344c101e40 2024-04-25T11:25:31.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.866+0000 7f344a7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f344c106c20 0x7f344c19a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:31.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.866+0000 7f344a7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f344c106c20 0x7f344c19a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34438/0 (socket says 172.21.15.69:34438) 2024-04-25T11:25:31.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.866+0000 7f344a7fc640 1 -- 172.21.15.69:0/475634253 learned_addr learned my addr 172.21.15.69:0/475634253 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:31.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.867+0000 7f344affd640 1 --2- 172.21.15.69:0/475634253 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f344c101e40 0x7f344c19a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:31.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.867+0000 7f344b7fe640 1 --2- 172.21.15.69:0/475634253 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f344c109790 0x7f344c1a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:31.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.867+0000 7f344a7fc640 1 -- 172.21.15.69:0/475634253 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f344c101e40 msgr2=0x7f344c19a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:31.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.867+0000 7f344a7fc640 1 --2- 172.21.15.69:0/475634253 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f344c101e40 0x7f344c19a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:31.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.867+0000 7f344a7fc640 1 -- 172.21.15.69:0/475634253 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f344c109790 msgr2=0x7f344c1a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:31.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.867+0000 7f344a7fc640 1 --2- 172.21.15.69:0/475634253 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f344c109790 0x7f344c1a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:31.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.867+0000 7f344a7fc640 1 -- 172.21.15.69:0/475634253 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f344c1a1d90 con 0x7f344c106c20 2024-04-25T11:25:31.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.867+0000 7f344affd640 1 --2- 172.21.15.69:0/475634253 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f344c101e40 0x7f344c19a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:25:31.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.867+0000 7f344b7fe640 1 --2- 172.21.15.69:0/475634253 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f344c109790 0x7f344c1a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:25:31.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.867+0000 7f344a7fc640 1 --2- 172.21.15.69:0/475634253 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f344c106c20 0x7f344c19a790 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3440009230 tx=0x7f34400042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:31.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.867+0000 7f3433fff640 1 -- 172.21.15.69:0/475634253 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344003f030 con 0x7f344c106c20 2024-04-25T11:25:31.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.867+0000 7f3433fff640 1 -- 172.21.15.69:0/475634253 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3440003c90 con 0x7f344c106c20 2024-04-25T11:25:31.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.867+0000 7f34517c0640 1 -- 172.21.15.69:0/475634253 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f344c1a2020 con 0x7f344c106c20 2024-04-25T11:25:31.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.867+0000 7f3433fff640 1 -- 172.21.15.69:0/475634253 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3440035070 con 0x7f344c106c20 2024-04-25T11:25:31.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.867+0000 7f34517c0640 1 -- 172.21.15.69:0/475634253 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f344c1a2590 con 0x7f344c106c20 2024-04-25T11:25:31.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.868+0000 7f34517c0640 1 -- 172.21.15.69:0/475634253 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f344c0fee90 con 0x7f344c106c20 2024-04-25T11:25:31.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.869+0000 7f3433fff640 1 -- 172.21.15.69:0/475634253 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 147571+0+0 (secure 0 0 0) 0x7f3440042030 con 0x7f344c106c20 2024-04-25T11:25:31.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.870+0000 7f3433fff640 1 --2- 172.21.15.69:0/475634253 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f34340afb80 0x7f34340b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:31.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.871+0000 7f344affd640 1 --2- 172.21.15.69:0/475634253 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f34340afb80 0x7f34340b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:31.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.871+0000 7f3433fff640 1 -- 172.21.15.69:0/475634253 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f34400ffd80 con 0x7f344c106c20 2024-04-25T11:25:31.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.871+0000 7f344affd640 1 --2- 172.21.15.69:0/475634253 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f34340afb80 0x7f34340b2090 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f344c104b10 tx=0x7f343c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:31.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.875+0000 7f3433fff640 1 -- 172.21.15.69:0/475634253 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f34400c84a0 con 0x7f344c106c20 2024-04-25T11:25:31.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:31.906+0000 7f3433fff640 1 -- 172.21.15.69:0/475634253 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f34400c4030 con 0x7f344c106c20 2024-04-25T11:25:32.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.066+0000 7f34517c0640 1 -- 172.21.15.69:0/475634253 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f344c0637a0 con 0x7f344c106c20 2024-04-25T11:25:32.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.067+0000 7f3433fff640 1 -- 172.21.15.69:0/475634253 <== mon.0 v2:172.21.15.69:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+24658 (secure 0 0 0) 0x7f34400f5d50 con 0x7f344c106c20 2024-04-25T11:25:32.060 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:25:32.061 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":82,"fsid":"f7ed73dc-02f4-11ef-bc93-c7b262605968","created":"2024-04-25T11:15:37.965281+0000","modified":"2024-04-25T11:24:32.872870+0000","last_up_change":"2024-04-25T11:22:50.691869+0000","last_in_change":"2024-04-25T11:22:30.271959+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:18:57.781263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:23:38.685110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:23:39.943803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T11:23:43.650521+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6a0d179-4998-4754-bf82-2ea3dac54e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6803","nonce":3756025235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6805","nonce":3756025235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6809","nonce":3756025235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6807","nonce":3756025235}]},"public_addr":"172.21.15.69:6803/3756025235","cluster_addr":"172.21.15.69:6805/3756025235","heartbeat_back_addr":"172.21.15.69:6809/3756025235","heartbeat_front_addr":"172.21.15.69:6807/3756025235","state":["exists","up"]},{"osd":1,"uuid":"c0c66930-4233-493f-b2bb-99c473994b59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6801","nonce":3166305213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6803","nonce":3166305213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6807","nonce":3166305213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6805","nonce":3166305213}]},"public_addr":"172.21.15.146:6801/3166305213","cluster_addr":"172.21.15.146:6803/3166305213","heartbeat_back_addr":"172.21.15.146:6807/3166305213","heartbeat_front_addr":"172.21.15.146:6805/3166305213","state":["exists","up"]},{"osd":2,"uuid":"d4ef735b-9786-4cb9-83ef-7c3f0691a729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6801","nonce":1474319175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6803","nonce":1474319175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6807","nonce":1474319175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6805","nonce":1474319175}]},"public_addr":"172.21.15.152:6801/1474319175","cluster_addr":"172.21.15.152:6803/1474319175","heartbeat_back_addr":"172.21.15.152:6807/1474319175","heartbeat_front_addr":"172.21.15.152:6805/1474319175","state":["exists","up"]},{"osd":3,"uuid":"0ab85fb8-5bbb-4ffc-bddf-cae7644a64ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6811","nonce":1809071296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6813","nonce":1809071296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6817","nonce":1809071296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6815","nonce":1809071296}]},"public_addr":"172.21.15.69:6811/1809071296","cluster_addr":"172.21.15.69:6813/1809071296","heartbeat_back_addr":"172.21.15.69:6817/1809071296","heartbeat_front_addr":"172.21.15.69:6815/1809071296","state":["exists","up"]},{"osd":4,"uuid":"273e0592-bbbc-4188-8eeb-46a3879cf632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6809","nonce":1898520972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6811","nonce":1898520972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6815","nonce":1898520972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6813","nonce":1898520972}]},"public_addr":"172.21.15.146:6809/1898520972","cluster_addr":"172.21.15.146:6811/1898520972","heartbeat_back_addr":"172.21.15.146:6815/1898520972","heartbeat_front_addr":"172.21.15.146:6813/1898520972","state":["exists","up"]},{"osd":5,"uuid":"bbeae945-9fa2-48f9-a1d6-8bf18dcaef19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6809","nonce":1996555108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6811","nonce":1996555108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6815","nonce":1996555108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6813","nonce":1996555108}]},"public_addr":"172.21.15.152:6809/1996555108","cluster_addr":"172.21.15.152:6811/1996555108","heartbeat_back_addr":"172.21.15.152:6815/1996555108","heartbeat_front_addr":"172.21.15.152:6813/1996555108","state":["exists","up"]},{"osd":6,"uuid":"dbb61253-30bc-418e-bd75-118c80bbf931","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6819","nonce":491215762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6821","nonce":491215762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6825","nonce":491215762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6823","nonce":491215762}]},"public_addr":"172.21.15.69:6819/491215762","cluster_addr":"172.21.15.69:6821/491215762","heartbeat_back_addr":"172.21.15.69:6825/491215762","heartbeat_front_addr":"172.21.15.69:6823/491215762","state":["exists","up"]},{"osd":7,"uuid":"75f23854-e283-4e3e-94cf-7b4079d83876","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6817","nonce":2330294690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6819","nonce":2330294690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6823","nonce":2330294690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6821","nonce":2330294690}]},"public_addr":"172.21.15.146:6817/2330294690","cluster_addr":"172.21.15.146:6819/2330294690","heartbeat_back_addr":"172.21.15.146:6823/2330294690","heartbeat_front_addr":"172.21.15.146:6821/2330294690","state":["exists","up"]},{"osd":8,"uuid":"1aa1060c-0c14-4abd-be56-5ebfeb78759c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6817","nonce":950669381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6819","nonce":950669381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6823","nonce":950669381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6821","nonce":950669381}]},"public_addr":"172.21.15.152:6817/950669381","cluster_addr":"172.21.15.152:6819/950669381","heartbeat_back_addr":"172.21.15.152:6823/950669381","heartbeat_front_addr":"172.21.15.152:6821/950669381","state":["exists","up"]},{"osd":9,"uuid":"3f4bf800-4090-4138-9bb3-d001f13f450c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6827","nonce":2414186758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6829","nonce":2414186758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6833","nonce":2414186758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6831","nonce":2414186758}]},"public_addr":"172.21.15.69:6827/2414186758","cluster_addr":"172.21.15.69:6829/2414186758","heartbeat_back_addr":"172.21.15.69:6833/2414186758","heartbeat_front_addr":"172.21.15.69:6831/2414186758","state":["exists","up"]},{"osd":10,"uuid":"19e967e5-51ce-4506-80c9-0a95441baf99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6825","nonce":707256400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6827","nonce":707256400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6831","nonce":707256400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6829","nonce":707256400}]},"public_addr":"172.21.15.146:6825/707256400","cluster_addr":"172.21.15.146:6827/707256400","heartbeat_back_addr":"172.21.15.146:6831/707256400","heartbeat_front_addr":"172.21.15.146:6829/707256400","state":["exists","up"]},{"osd":11,"uuid":"7788c2de-6930-4808-be0f-f1fd1ae0cb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6825","nonce":2934336833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6827","nonce":2934336833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6831","nonce":2934336833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6829","nonce":2934336833}]},"public_addr":"172.21.15.152:6825/2934336833","cluster_addr":"172.21.15.152:6827/2934336833","heartbeat_back_addr":"172.21.15.152:6831/2934336833","heartbeat_front_addr":"172.21.15.152:6829/2934336833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:07.497448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:31.227494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:55.027978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:20.713437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:46.214704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:20:11.199983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:20:37.177070+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:02.951240+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:28.963680+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:56.095807+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:22:22.221744+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:22:48.929334+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1b","mappings":[{"from":5,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/1372727546":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6801/1097124842":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6800/1508198215":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3103465990":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3540132960":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3899912190":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6800/1097124842":"2024-04-26T11:16:31.688159+0000","172.21.15.69:0/1199907202":"2024-04-26T11:16:04.860792+0000","172.21.15.69:6801/1508198215":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/2879102529":"2024-04-26T11:16:31.688159+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:25:32.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.069+0000 7f34517c0640 1 -- 172.21.15.69:0/475634253 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f34340afb80 msgr2=0x7f34340b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:32.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.069+0000 7f34517c0640 1 --2- 172.21.15.69:0/475634253 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f34340afb80 0x7f34340b2090 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f344c104b10 tx=0x7f343c008040 comp rx=0 tx=0).stop 2024-04-25T11:25:32.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.069+0000 7f34517c0640 1 -- 172.21.15.69:0/475634253 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f344c106c20 msgr2=0x7f344c19a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:32.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.069+0000 7f34517c0640 1 --2- 172.21.15.69:0/475634253 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f344c106c20 0x7f344c19a790 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3440009230 tx=0x7f34400042f0 comp rx=0 tx=0).stop 2024-04-25T11:25:32.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.069+0000 7f34517c0640 1 -- 172.21.15.69:0/475634253 shutdown_connections 2024-04-25T11:25:32.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.069+0000 7f34517c0640 1 --2- 172.21.15.69:0/475634253 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f34340afb80 0x7f34340b2090 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:32.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.069+0000 7f34517c0640 1 --2- 172.21.15.69:0/475634253 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f344c109790 0x7f344c1a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:32.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.069+0000 7f34517c0640 1 --2- 172.21.15.69:0/475634253 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f344c106c20 0x7f344c19a790 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:32.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.069+0000 7f34517c0640 1 --2- 172.21.15.69:0/475634253 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f344c101e40 0x7f344c19a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:32.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.069+0000 7f34517c0640 1 -- 172.21.15.69:0/475634253 >> 172.21.15.69:0/475634253 conn(0x7f344c0fd370 msgr2=0x7f344c10a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:32.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.070+0000 7f34517c0640 1 -- 172.21.15.69:0/475634253 shutdown_connections 2024-04-25T11:25:32.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.070+0000 7f34517c0640 1 -- 172.21.15.69:0/475634253 wait complete. 2024-04-25T11:25:32.073 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:31 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2319699479' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:25:32.073 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:31 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2522833109' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T11:25:32.073 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:31 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/3557655681' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:25:32.074 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:18:57.781263+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:23:38.685110+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:23:39.943803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T11:23:43.650521+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T11:25:32.075 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:25:32.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.192+0000 7f0cd3df3640 1 -- 172.21.15.69:0/414858566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ccc103ae0 msgr2=0x7f0ccc1047e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:32.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.192+0000 7f0cd3df3640 1 --2- 172.21.15.69:0/414858566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ccc103ae0 0x7f0ccc1047e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8009900 tx=0x7f0cc802a410 comp rx=0 tx=0).stop 2024-04-25T11:25:32.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.192+0000 7f0cd3df3640 1 -- 172.21.15.69:0/414858566 shutdown_connections 2024-04-25T11:25:32.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.192+0000 7f0cd3df3640 1 --2- 172.21.15.69:0/414858566 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ccc109a20 0x7f0ccc10be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:32.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.192+0000 7f0cd3df3640 1 --2- 172.21.15.69:0/414858566 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ccc104d20 0x7f0ccc105130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:32.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.192+0000 7f0cd3df3640 1 --2- 172.21.15.69:0/414858566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ccc103ae0 0x7f0ccc1047e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:32.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.192+0000 7f0cd3df3640 1 -- 172.21.15.69:0/414858566 >> 172.21.15.69:0/414858566 conn(0x7f0ccc0fecd0 msgr2=0x7f0ccc101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:32.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.192+0000 7f0cd3df3640 1 -- 172.21.15.69:0/414858566 shutdown_connections 2024-04-25T11:25:32.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.192+0000 7f0cd3df3640 1 -- 172.21.15.69:0/414858566 wait complete. 2024-04-25T11:25:32.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.192+0000 7f0cd3df3640 1 Processor -- start 2024-04-25T11:25:32.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.192+0000 7f0cd3df3640 1 -- start start 2024-04-25T11:25:32.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.192+0000 7f0cd3df3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ccc103ae0 0x7f0ccc076850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:32.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.193+0000 7f0cd3df3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ccc104d20 0x7f0ccc074ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:32.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.193+0000 7f0cd3df3640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ccc109a20 0x7f0ccc075430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:32.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.193+0000 7f0cd3df3640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc10f4f0 con 0x7f0ccc103ae0 2024-04-25T11:25:32.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.193+0000 7f0cd3df3640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc10f370 con 0x7f0ccc104d20 2024-04-25T11:25:32.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.193+0000 7f0cd3df3640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc10f670 con 0x7f0ccc109a20 2024-04-25T11:25:32.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.193+0000 7f0cd1b8f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ccc103ae0 0x7f0ccc076850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:32.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.193+0000 7f0cd1b8f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ccc103ae0 0x7f0ccc076850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34444/0 (socket says 172.21.15.69:34444) 2024-04-25T11:25:32.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.193+0000 7f0cd1b8f640 1 -- 172.21.15.69:0/3542860219 learned_addr learned my addr 172.21.15.69:0/3542860219 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:32.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.193+0000 7f0cd2390640 1 --2- 172.21.15.69:0/3542860219 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ccc109a20 0x7f0ccc075430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:32.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.193+0000 7f0cd138e640 1 --2- 172.21.15.69:0/3542860219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ccc104d20 0x7f0ccc074ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:32.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.193+0000 7f0cd1b8f640 1 -- 172.21.15.69:0/3542860219 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ccc109a20 msgr2=0x7f0ccc075430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:32.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.193+0000 7f0cd1b8f640 1 --2- 172.21.15.69:0/3542860219 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ccc109a20 0x7f0ccc075430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:32.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.193+0000 7f0cd1b8f640 1 -- 172.21.15.69:0/3542860219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ccc104d20 msgr2=0x7f0ccc074ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:32.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.193+0000 7f0cd1b8f640 1 --2- 172.21.15.69:0/3542860219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ccc104d20 0x7f0ccc074ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:32.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.193+0000 7f0cd1b8f640 1 -- 172.21.15.69:0/3542860219 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ccc075ad0 con 0x7f0ccc103ae0 2024-04-25T11:25:32.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.193+0000 7f0cd138e640 1 --2- 172.21.15.69:0/3542860219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ccc104d20 0x7f0ccc074ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:25:32.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.193+0000 7f0cd1b8f640 1 --2- 172.21.15.69:0/3542860219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ccc103ae0 0x7f0ccc076850 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8005ab0 tx=0x7f0cc8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:32.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.193+0000 7f0cc2ffd640 1 -- 172.21.15.69:0/3542860219 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc803f070 con 0x7f0ccc103ae0 2024-04-25T11:25:32.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.193+0000 7f0cd3df3640 1 -- 172.21.15.69:0/3542860219 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ccc1a42a0 con 0x7f0ccc103ae0 2024-04-25T11:25:32.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.193+0000 7f0cc2ffd640 1 -- 172.21.15.69:0/3542860219 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0cc8003c80 con 0x7f0ccc103ae0 2024-04-25T11:25:32.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.193+0000 7f0cc2ffd640 1 -- 172.21.15.69:0/3542860219 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc8035070 con 0x7f0ccc103ae0 2024-04-25T11:25:32.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.193+0000 7f0cd3df3640 1 -- 172.21.15.69:0/3542860219 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ccc1a46d0 con 0x7f0ccc103ae0 2024-04-25T11:25:32.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.194+0000 7f0cd3df3640 1 -- 172.21.15.69:0/3542860219 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ccc108830 con 0x7f0ccc103ae0 2024-04-25T11:25:32.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.196+0000 7f0cc2ffd640 1 -- 172.21.15.69:0/3542860219 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f0cc803b610 con 0x7f0ccc103ae0 2024-04-25T11:25:32.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.196+0000 7f0cc2ffd640 1 --2- 172.21.15.69:0/3542860219 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f0cb80afaf0 0x7f0cb80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:32.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.196+0000 7f0cd138e640 1 --2- 172.21.15.69:0/3542860219 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f0cb80afaf0 0x7f0cb80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:32.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.197+0000 7f0cc2ffd640 1 -- 172.21.15.69:0/3542860219 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f0cc80fff80 con 0x7f0ccc103ae0 2024-04-25T11:25:32.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.197+0000 7f0cd138e640 1 --2- 172.21.15.69:0/3542860219 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f0cb80afaf0 0x7f0cb80b2000 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc005fd0 tx=0x7f0cbc005f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:32.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.200+0000 7f0cc2ffd640 1 -- 172.21.15.69:0/3542860219 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0cc80c8800 con 0x7f0ccc103ae0 2024-04-25T11:25:32.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:31 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/2319699479' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:25:32.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:31 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/2522833109' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T11:25:32.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:31 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/3557655681' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:25:32.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:31 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/2319699479' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:25:32.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:31 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/2522833109' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T11:25:32.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:31 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/3557655681' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:25:32.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.377+0000 7f0cd3df3640 1 -- 172.21.15.69:0/3542860219 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0ccc02dce0 con 0x7f0ccc103ae0 2024-04-25T11:25:32.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.377+0000 7f0cc2ffd640 1 -- 172.21.15.69:0/3542860219 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f0cc80c8100 con 0x7f0ccc103ae0 2024-04-25T11:25:32.370 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-04-25T11:25:32.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.379+0000 7f0cd3df3640 1 -- 172.21.15.69:0/3542860219 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f0cb80afaf0 msgr2=0x7f0cb80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:32.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.379+0000 7f0cd3df3640 1 --2- 172.21.15.69:0/3542860219 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f0cb80afaf0 0x7f0cb80b2000 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc005fd0 tx=0x7f0cbc005f40 comp rx=0 tx=0).stop 2024-04-25T11:25:32.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.379+0000 7f0cd3df3640 1 -- 172.21.15.69:0/3542860219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ccc103ae0 msgr2=0x7f0ccc076850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:32.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.379+0000 7f0cd3df3640 1 --2- 172.21.15.69:0/3542860219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ccc103ae0 0x7f0ccc076850 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8005ab0 tx=0x7f0cc8004270 comp rx=0 tx=0).stop 2024-04-25T11:25:32.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.379+0000 7f0cd3df3640 1 -- 172.21.15.69:0/3542860219 shutdown_connections 2024-04-25T11:25:32.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.379+0000 7f0cd3df3640 1 --2- 172.21.15.69:0/3542860219 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f0cb80afaf0 0x7f0cb80b2000 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:32.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.379+0000 7f0cd3df3640 1 --2- 172.21.15.69:0/3542860219 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ccc109a20 0x7f0ccc075430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:32.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.379+0000 7f0cd3df3640 1 --2- 172.21.15.69:0/3542860219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ccc104d20 0x7f0ccc074ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:32.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.379+0000 7f0cd3df3640 1 --2- 172.21.15.69:0/3542860219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ccc103ae0 0x7f0ccc076850 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:32.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.379+0000 7f0cd3df3640 1 -- 172.21.15.69:0/3542860219 >> 172.21.15.69:0/3542860219 conn(0x7f0ccc0fecd0 msgr2=0x7f0ccc105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:32.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.379+0000 7f0cd3df3640 1 -- 172.21.15.69:0/3542860219 shutdown_connections 2024-04-25T11:25:32.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.379+0000 7f0cd3df3640 1 -- 172.21.15.69:0/3542860219 wait complete. 2024-04-25T11:25:32.383 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:25:32.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.502+0000 7f2f104c6640 1 -- 172.21.15.69:0/3336892581 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f08104d30 msgr2=0x7f2f08105140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:32.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.502+0000 7f2f104c6640 1 --2- 172.21.15.69:0/3336892581 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f08104d30 0x7f2f08105140 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2f04009900 tx=0x7f2f0402a410 comp rx=0 tx=0).stop 2024-04-25T11:25:32.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.502+0000 7f2f104c6640 1 -- 172.21.15.69:0/3336892581 shutdown_connections 2024-04-25T11:25:32.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.502+0000 7f2f104c6640 1 --2- 172.21.15.69:0/3336892581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f08109a30 0x7f2f0810be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:32.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.502+0000 7f2f104c6640 1 --2- 172.21.15.69:0/3336892581 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f08104d30 0x7f2f08105140 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:32.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.502+0000 7f2f104c6640 1 --2- 172.21.15.69:0/3336892581 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f08103af0 0x7f2f081047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:32.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.502+0000 7f2f104c6640 1 -- 172.21.15.69:0/3336892581 >> 172.21.15.69:0/3336892581 conn(0x7f2f080fece0 msgr2=0x7f2f08101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:32.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.502+0000 7f2f104c6640 1 -- 172.21.15.69:0/3336892581 shutdown_connections 2024-04-25T11:25:32.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.502+0000 7f2f104c6640 1 -- 172.21.15.69:0/3336892581 wait complete. 2024-04-25T11:25:32.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.503+0000 7f2f104c6640 1 Processor -- start 2024-04-25T11:25:32.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.503+0000 7f2f104c6640 1 -- start start 2024-04-25T11:25:32.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.503+0000 7f2f104c6640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f08103af0 0x7f2f0819a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:32.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.503+0000 7f2f104c6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f08104d30 0x7f2f0819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:32.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.503+0000 7f2f104c6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f08109a30 0x7f2f081a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:32.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.503+0000 7f2f104c6640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0810f4c0 con 0x7f2f08104d30 2024-04-25T11:25:32.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.503+0000 7f2f104c6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0810f340 con 0x7f2f08109a30 2024-04-25T11:25:32.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.503+0000 7f2f104c6640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0810f640 con 0x7f2f08103af0 2024-04-25T11:25:32.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.503+0000 7f2f0da61640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f08104d30 0x7f2f0819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:32.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.503+0000 7f2f0da61640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f08104d30 0x7f2f0819a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54956/0 (socket says 172.21.15.69:54956) 2024-04-25T11:25:32.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.503+0000 7f2f0da61640 1 -- 172.21.15.69:0/3665831593 learned_addr learned my addr 172.21.15.69:0/3665831593 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:32.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.504+0000 7f2f0e262640 1 --2- 172.21.15.69:0/3665831593 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f08103af0 0x7f2f0819a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:32.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.504+0000 7f2f0ea63640 1 --2- 172.21.15.69:0/3665831593 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f08109a30 0x7f2f081a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:32.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.504+0000 7f2f0da61640 1 -- 172.21.15.69:0/3665831593 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f08103af0 msgr2=0x7f2f0819a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:32.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.504+0000 7f2f0da61640 1 --2- 172.21.15.69:0/3665831593 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f08103af0 0x7f2f0819a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:32.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.504+0000 7f2f0da61640 1 -- 172.21.15.69:0/3665831593 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f08109a30 msgr2=0x7f2f081a1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:32.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.504+0000 7f2f0da61640 1 --2- 172.21.15.69:0/3665831593 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f08109a30 0x7f2f081a1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:32.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.504+0000 7f2f0da61640 1 -- 172.21.15.69:0/3665831593 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f081a1ca0 con 0x7f2f08104d30 2024-04-25T11:25:32.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.504+0000 7f2f0ea63640 1 --2- 172.21.15.69:0/3665831593 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f08109a30 0x7f2f081a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:25:32.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.504+0000 7f2f0da61640 1 --2- 172.21.15.69:0/3665831593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f08104d30 0x7f2f0819a6a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2f04009230 tx=0x7f2f040040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:32.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.504+0000 7f2eff7fe640 1 -- 172.21.15.69:0/3665831593 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0403f030 con 0x7f2f08104d30 2024-04-25T11:25:32.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.504+0000 7f2f104c6640 1 -- 172.21.15.69:0/3665831593 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f081a1f30 con 0x7f2f08104d30 2024-04-25T11:25:32.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.504+0000 7f2eff7fe640 1 -- 172.21.15.69:0/3665831593 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f040044c0 con 0x7f2f08104d30 2024-04-25T11:25:32.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.504+0000 7f2eff7fe640 1 -- 172.21.15.69:0/3665831593 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f04035070 con 0x7f2f08104d30 2024-04-25T11:25:32.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.504+0000 7f2f104c6640 1 -- 172.21.15.69:0/3665831593 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f081a2390 con 0x7f2f08104d30 2024-04-25T11:25:32.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.505+0000 7f2f104c6640 1 -- 172.21.15.69:0/3665831593 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f08108840 con 0x7f2f08104d30 2024-04-25T11:25:32.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.507+0000 7f2eff7fe640 1 -- 172.21.15.69:0/3665831593 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f2f04042030 con 0x7f2f08104d30 2024-04-25T11:25:32.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.508+0000 7f2eff7fe640 1 --2- 172.21.15.69:0/3665831593 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f2ef40afaf0 0x7f2ef40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:32.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.508+0000 7f2f0e262640 1 --2- 172.21.15.69:0/3665831593 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f2ef40afaf0 0x7f2ef40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:32.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.508+0000 7f2eff7fe640 1 -- 172.21.15.69:0/3665831593 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7f2f040ff9c0 con 0x7f2f08104d30 2024-04-25T11:25:32.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.508+0000 7f2f0e262640 1 --2- 172.21.15.69:0/3665831593 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f2ef40afaf0 0x7f2ef40b2000 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2f081046f0 tx=0x7f2ef8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:32.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.511+0000 7f2eff7fe640 1 -- 172.21.15.69:0/3665831593 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2f040c8240 con 0x7f2f08104d30 2024-04-25T11:25:32.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.689+0000 7f2f104c6640 1 -- 172.21.15.69:0/3665831593 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2f080637a0 con 0x7f2f08104d30 2024-04-25T11:25:32.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.690+0000 7f2eff7fe640 1 -- 172.21.15.69:0/3665831593 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) ==== 104+0+11 (secure 0 0 0) 0x7f2f040d02c0 con 0x7f2f08104d30 2024-04-25T11:25:32.683 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-25T11:25:32.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.692+0000 7f2f104c6640 1 -- 172.21.15.69:0/3665831593 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f2ef40afaf0 msgr2=0x7f2ef40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:32.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.692+0000 7f2f104c6640 1 --2- 172.21.15.69:0/3665831593 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f2ef40afaf0 0x7f2ef40b2000 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2f081046f0 tx=0x7f2ef8005f90 comp rx=0 tx=0).stop 2024-04-25T11:25:32.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.692+0000 7f2f104c6640 1 -- 172.21.15.69:0/3665831593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f08104d30 msgr2=0x7f2f0819a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:32.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.692+0000 7f2f104c6640 1 --2- 172.21.15.69:0/3665831593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f08104d30 0x7f2f0819a6a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2f04009230 tx=0x7f2f040040c0 comp rx=0 tx=0).stop 2024-04-25T11:25:32.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.692+0000 7f2f104c6640 1 -- 172.21.15.69:0/3665831593 shutdown_connections 2024-04-25T11:25:32.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.692+0000 7f2f104c6640 1 --2- 172.21.15.69:0/3665831593 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f2ef40afaf0 0x7f2ef40b2000 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:32.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.692+0000 7f2f104c6640 1 --2- 172.21.15.69:0/3665831593 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f08109a30 0x7f2f081a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:32.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.692+0000 7f2f104c6640 1 --2- 172.21.15.69:0/3665831593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2f08104d30 0x7f2f0819a6a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:32.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.692+0000 7f2f104c6640 1 --2- 172.21.15.69:0/3665831593 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f08103af0 0x7f2f0819a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:32.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.692+0000 7f2f104c6640 1 -- 172.21.15.69:0/3665831593 >> 172.21.15.69:0/3665831593 conn(0x7f2f080fece0 msgr2=0x7f2f08105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:32.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.692+0000 7f2f104c6640 1 -- 172.21.15.69:0/3665831593 shutdown_connections 2024-04-25T11:25:32.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.692+0000 7f2f104c6640 1 -- 172.21.15.69:0/3665831593 wait complete. 2024-04-25T11:25:32.696 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:25:32.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.813+0000 7fe0d6896640 1 -- 172.21.15.69:0/3609568778 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0d0104480 msgr2=0x7fe0d010abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:32.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.813+0000 7fe0d6896640 1 --2- 172.21.15.69:0/3609568778 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0d0104480 0x7fe0d010abd0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8009ca0 tx=0x7fe0c802a8c0 comp rx=0 tx=0).stop 2024-04-25T11:25:32.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.813+0000 7fe0d6896640 1 -- 172.21.15.69:0/3609568778 shutdown_connections 2024-04-25T11:25:32.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.813+0000 7fe0d6896640 1 --2- 172.21.15.69:0/3609568778 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0d0104480 0x7fe0d010abd0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:32.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.813+0000 7fe0d6896640 1 --2- 172.21.15.69:0/3609568778 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0d0103960 0x7fe0d0103df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:32.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.813+0000 7fe0d6896640 1 --2- 172.21.15.69:0/3609568778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0d0102f80 0x7fe0d0103390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:32.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.813+0000 7fe0d6896640 1 -- 172.21.15.69:0/3609568778 >> 172.21.15.69:0/3609568778 conn(0x7fe0d00fd380 msgr2=0x7fe0d00ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:32.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.813+0000 7fe0d6896640 1 -- 172.21.15.69:0/3609568778 shutdown_connections 2024-04-25T11:25:32.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.813+0000 7fe0d6896640 1 -- 172.21.15.69:0/3609568778 wait complete. 2024-04-25T11:25:32.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.814+0000 7fe0d6896640 1 Processor -- start 2024-04-25T11:25:32.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.814+0000 7fe0d6896640 1 -- start start 2024-04-25T11:25:32.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.814+0000 7fe0d6896640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0d0102f80 0x7fe0d0199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:32.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.814+0000 7fe0d6896640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0d0103960 0x7fe0d0199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:32.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.814+0000 7fe0d6896640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0d0104480 0x7fe0d01a0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:32.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.814+0000 7fe0d6896640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d010e450 con 0x7fe0d0104480 2024-04-25T11:25:32.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.814+0000 7fe0d6896640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d010e2d0 con 0x7fe0d0102f80 2024-04-25T11:25:32.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.814+0000 7fe0d6896640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d010e5d0 con 0x7fe0d0103960 2024-04-25T11:25:32.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.814+0000 7fe0d4e33640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0d0104480 0x7fe0d01a0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:32.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.814+0000 7fe0d4e33640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0d0104480 0x7fe0d01a0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54978/0 (socket says 172.21.15.69:54978) 2024-04-25T11:25:32.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.814+0000 7fe0d4e33640 1 -- 172.21.15.69:0/850698178 learned_addr learned my addr 172.21.15.69:0/850698178 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:32.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.814+0000 7fe0cf7fe640 1 --2- 172.21.15.69:0/850698178 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0d0103960 0x7fe0d0199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:32.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.815+0000 7fe0cffff640 1 --2- 172.21.15.69:0/850698178 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0d0102f80 0x7fe0d0199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:32.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.815+0000 7fe0d4e33640 1 -- 172.21.15.69:0/850698178 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0d0103960 msgr2=0x7fe0d0199540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:32.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.815+0000 7fe0d4e33640 1 --2- 172.21.15.69:0/850698178 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0d0103960 0x7fe0d0199540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:32.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.815+0000 7fe0d4e33640 1 -- 172.21.15.69:0/850698178 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0d0102f80 msgr2=0x7fe0d0199000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:32.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.815+0000 7fe0d4e33640 1 --2- 172.21.15.69:0/850698178 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0d0102f80 0x7fe0d0199000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:32.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.815+0000 7fe0d4e33640 1 -- 172.21.15.69:0/850698178 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0d01a0af0 con 0x7fe0d0104480 2024-04-25T11:25:32.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.815+0000 7fe0cffff640 1 --2- 172.21.15.69:0/850698178 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0d0102f80 0x7fe0d0199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:25:32.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.815+0000 7fe0d4e33640 1 --2- 172.21.15.69:0/850698178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0d0104480 0x7fe0d01a0480 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8031040 tx=0x7fe0c8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:32.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.815+0000 7fe0cd7fa640 1 -- 172.21.15.69:0/850698178 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c8004040 con 0x7fe0d0104480 2024-04-25T11:25:32.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.815+0000 7fe0cd7fa640 1 -- 172.21.15.69:0/850698178 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe0c8004670 con 0x7fe0d0104480 2024-04-25T11:25:32.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.815+0000 7fe0d6896640 1 -- 172.21.15.69:0/850698178 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0d01a0d80 con 0x7fe0d0104480 2024-04-25T11:25:32.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.815+0000 7fe0cd7fa640 1 -- 172.21.15.69:0/850698178 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c8030d50 con 0x7fe0d0104480 2024-04-25T11:25:32.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.815+0000 7fe0d6896640 1 -- 172.21.15.69:0/850698178 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0d01a1210 con 0x7fe0d0104480 2024-04-25T11:25:32.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.816+0000 7fe0d6896640 1 -- 172.21.15.69:0/850698178 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0d0109930 con 0x7fe0d0104480 2024-04-25T11:25:32.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.817+0000 7fe0cd7fa640 1 -- 172.21.15.69:0/850698178 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fe0c80305c0 con 0x7fe0d0104480 2024-04-25T11:25:32.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.818+0000 7fe0cd7fa640 1 --2- 172.21.15.69:0/850698178 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fe0b80afaf0 0x7fe0b80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:32.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.818+0000 7fe0cffff640 1 --2- 172.21.15.69:0/850698178 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fe0b80afaf0 0x7fe0b80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:32.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.818+0000 7fe0cd7fa640 1 -- 172.21.15.69:0/850698178 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 9092+0+0 (secure 0 0 0) 0x7fe0c8100730 con 0x7fe0d0104480 2024-04-25T11:25:32.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.819+0000 7fe0cffff640 1 --2- 172.21.15.69:0/850698178 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fe0b80afaf0 0x7fe0b80b2000 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe0c00058c0 tx=0x7fe0c0005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:32.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:32.822+0000 7fe0cd7fa640 1 -- 172.21.15.69:0/850698178 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe0c80c8fb0 con 0x7fe0d0104480 2024-04-25T11:25:33.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.007+0000 7fe0d6896640 1 -- 172.21.15.69:0/850698178 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe0d00637a0 con 0x7fe0d0104480 2024-04-25T11:25:33.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.007+0000 7fe0cd7fa640 1 -- 172.21.15.69:0/850698178 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) ==== 100+0+11 (secure 0 0 0) 0x7fe0c80c88b0 con 0x7fe0d0104480 2024-04-25T11:25:33.000 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-04-25T11:25:33.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.010+0000 7fe0d6896640 1 -- 172.21.15.69:0/850698178 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fe0b80afaf0 msgr2=0x7fe0b80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:33.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.010+0000 7fe0d6896640 1 --2- 172.21.15.69:0/850698178 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fe0b80afaf0 0x7fe0b80b2000 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe0c00058c0 tx=0x7fe0c0005d30 comp rx=0 tx=0).stop 2024-04-25T11:25:33.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.010+0000 7fe0d6896640 1 -- 172.21.15.69:0/850698178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0d0104480 msgr2=0x7fe0d01a0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:33.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.010+0000 7fe0d6896640 1 --2- 172.21.15.69:0/850698178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0d0104480 0x7fe0d01a0480 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8031040 tx=0x7fe0c8003ba0 comp rx=0 tx=0).stop 2024-04-25T11:25:33.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.010+0000 7fe0d6896640 1 -- 172.21.15.69:0/850698178 shutdown_connections 2024-04-25T11:25:33.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.010+0000 7fe0d6896640 1 --2- 172.21.15.69:0/850698178 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fe0b80afaf0 0x7fe0b80b2000 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:33.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.010+0000 7fe0d6896640 1 --2- 172.21.15.69:0/850698178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0d0104480 0x7fe0d01a0480 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:33.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.010+0000 7fe0d6896640 1 --2- 172.21.15.69:0/850698178 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0d0103960 0x7fe0d0199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:33.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.010+0000 7fe0d6896640 1 --2- 172.21.15.69:0/850698178 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0d0102f80 0x7fe0d0199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:33.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.010+0000 7fe0d6896640 1 -- 172.21.15.69:0/850698178 >> 172.21.15.69:0/850698178 conn(0x7fe0d00fd380 msgr2=0x7fe0d0106b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:33.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.010+0000 7fe0d6896640 1 -- 172.21.15.69:0/850698178 shutdown_connections 2024-04-25T11:25:33.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.010+0000 7fe0d6896640 1 -- 172.21.15.69:0/850698178 wait complete. 2024-04-25T11:25:33.013 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:32 smithi069 ceph-mon[29188]: pgmap v342: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:25:33.013 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:32 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-25T11:25:33.013 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:32 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]: dispatch 2024-04-25T11:25:33.013 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:32 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8, 5, 9]}]: dispatch 2024-04-25T11:25:33.013 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:32 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-04-25T11:25:33.013 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:32 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]: dispatch 2024-04-25T11:25:33.013 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:32 smithi069 ceph-mon[29188]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-25T11:25:33.013 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:32 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/475634253' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:25:33.014 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:32 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/3542860219' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:25:33.014 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:32 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/3665831593' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:25:33.014 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:32 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-25T11:25:33.014 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:32 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]': finished 2024-04-25T11:25:33.014 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:32 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8, 5, 9]}]': finished 2024-04-25T11:25:33.014 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:32 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-04-25T11:25:33.014 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:32 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]': finished 2024-04-25T11:25:33.015 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T11:25:33.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.133+0000 7f097fd72640 1 -- 172.21.15.69:0/233633810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09781029e0 msgr2=0x7f09781036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:33.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.133+0000 7f097fd72640 1 --2- 172.21.15.69:0/233633810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09781029e0 0x7f09781036e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0974009ca0 tx=0x7f097402a8c0 comp rx=0 tx=0).stop 2024-04-25T11:25:33.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.133+0000 7f097fd72640 1 -- 172.21.15.69:0/233633810 shutdown_connections 2024-04-25T11:25:33.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.133+0000 7f097fd72640 1 --2- 172.21.15.69:0/233633810 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0978108920 0x7f097810ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:33.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.133+0000 7f097fd72640 1 --2- 172.21.15.69:0/233633810 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0978103c20 0x7f0978104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:33.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.133+0000 7f097fd72640 1 --2- 172.21.15.69:0/233633810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09781029e0 0x7f09781036e0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:33.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.134+0000 7f097fd72640 1 -- 172.21.15.69:0/233633810 >> 172.21.15.69:0/233633810 conn(0x7f09780fdbd0 msgr2=0x7f0978100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:33.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.134+0000 7f097fd72640 1 -- 172.21.15.69:0/233633810 shutdown_connections 2024-04-25T11:25:33.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.134+0000 7f097fd72640 1 -- 172.21.15.69:0/233633810 wait complete. 2024-04-25T11:25:33.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.134+0000 7f097fd72640 1 Processor -- start 2024-04-25T11:25:33.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.134+0000 7f097fd72640 1 -- start start 2024-04-25T11:25:33.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.134+0000 7f097fd72640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09781029e0 0x7f0978199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:33.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.134+0000 7f097fd72640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0978103c20 0x7f09781999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:33.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.134+0000 7f097fd72640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0978108920 0x7f09781a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:33.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.134+0000 7f097fd72640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f097810e4c0 con 0x7f09781029e0 2024-04-25T11:25:33.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.134+0000 7f097fd72640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f097810e340 con 0x7f0978108920 2024-04-25T11:25:33.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.134+0000 7f097fd72640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f097810e640 con 0x7f0978103c20 2024-04-25T11:25:33.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.135+0000 7f097db0e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09781029e0 0x7f0978199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:33.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.135+0000 7f097db0e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09781029e0 0x7f0978199470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54990/0 (socket says 172.21.15.69:54990) 2024-04-25T11:25:33.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.135+0000 7f097db0e640 1 -- 172.21.15.69:0/3422671365 learned_addr learned my addr 172.21.15.69:0/3422671365 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:33.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.135+0000 7f097e30f640 1 --2- 172.21.15.69:0/3422671365 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0978108920 0x7f09781a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:33.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.135+0000 7f097d30d640 1 --2- 172.21.15.69:0/3422671365 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0978103c20 0x7f09781999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:33.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.135+0000 7f097db0e640 1 -- 172.21.15.69:0/3422671365 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0978103c20 msgr2=0x7f09781999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:33.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.135+0000 7f097db0e640 1 --2- 172.21.15.69:0/3422671365 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0978103c20 0x7f09781999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:33.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.135+0000 7f097db0e640 1 -- 172.21.15.69:0/3422671365 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0978108920 msgr2=0x7f09781a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:33.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.135+0000 7f097db0e640 1 --2- 172.21.15.69:0/3422671365 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0978108920 0x7f09781a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:33.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.135+0000 7f097db0e640 1 -- 172.21.15.69:0/3422671365 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09781a0fb0 con 0x7f09781029e0 2024-04-25T11:25:33.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.135+0000 7f097db0e640 1 --2- 172.21.15.69:0/3422671365 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09781029e0 0x7f0978199470 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f09740098d0 tx=0x7f0974003ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:33.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.135+0000 7f096effd640 1 -- 172.21.15.69:0/3422671365 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0974032ed0 con 0x7f09781029e0 2024-04-25T11:25:33.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.135+0000 7f097fd72640 1 -- 172.21.15.69:0/3422671365 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09781a1240 con 0x7f09781029e0 2024-04-25T11:25:33.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.135+0000 7f096effd640 1 -- 172.21.15.69:0/3422671365 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f09740046d0 con 0x7f09781029e0 2024-04-25T11:25:33.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.135+0000 7f096effd640 1 -- 172.21.15.69:0/3422671365 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0974030dc0 con 0x7f09781029e0 2024-04-25T11:25:33.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.135+0000 7f097fd72640 1 -- 172.21.15.69:0/3422671365 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09781a1810 con 0x7f09781029e0 2024-04-25T11:25:33.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.136+0000 7f097fd72640 1 -- 172.21.15.69:0/3422671365 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0978107730 con 0x7f09781029e0 2024-04-25T11:25:33.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.138+0000 7f096effd640 1 -- 172.21.15.69:0/3422671365 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f09740305c0 con 0x7f09781029e0 2024-04-25T11:25:33.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.138+0000 7f096effd640 1 --2- 172.21.15.69:0/3422671365 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f09640afaf0 0x7f09640b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:33.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.138+0000 7f097d30d640 1 --2- 172.21.15.69:0/3422671365 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f09640afaf0 0x7f09640b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:33.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.138+0000 7f096effd640 1 -- 172.21.15.69:0/3422671365 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9216+0+0 (secure 0 0 0) 0x7f0974100a10 con 0x7f09781029e0 2024-04-25T11:25:33.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.139+0000 7f097d30d640 1 --2- 172.21.15.69:0/3422671365 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f09640afaf0 0x7f09640b2000 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f097819a510 tx=0x7f0968008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:33.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.142+0000 7f096effd640 1 -- 172.21.15.69:0/3422671365 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f09740c9220 con 0x7f09781029e0 2024-04-25T11:25:33.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:32 smithi152 ceph-mon[32701]: pgmap v342: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:25:33.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:32 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-25T11:25:33.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:32 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]: dispatch 2024-04-25T11:25:33.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:32 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8, 5, 9]}]: dispatch 2024-04-25T11:25:33.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:32 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-04-25T11:25:33.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:32 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]: dispatch 2024-04-25T11:25:33.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:32 smithi152 ceph-mon[32701]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-25T11:25:33.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:32 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/475634253' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:25:33.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:32 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/3542860219' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:25:33.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:32 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/3665831593' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:25:33.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:32 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-25T11:25:33.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:32 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]': finished 2024-04-25T11:25:33.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:32 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8, 5, 9]}]': finished 2024-04-25T11:25:33.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:32 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-04-25T11:25:33.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:32 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]': finished 2024-04-25T11:25:33.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.319+0000 7f097fd72640 1 -- 172.21.15.69:0/3422671365 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f097802dd10 con 0x7f09781029e0 2024-04-25T11:25:33.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.320+0000 7f096effd640 1 -- 172.21.15.69:0/3422671365 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v83) ==== 103+0+11 (secure 0 0 0) 0x7f09740c8b20 con 0x7f09781029e0 2024-04-25T11:25:33.313 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-25T11:25:33.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.322+0000 7f097fd72640 1 -- 172.21.15.69:0/3422671365 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f09640afaf0 msgr2=0x7f09640b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:33.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.322+0000 7f097fd72640 1 --2- 172.21.15.69:0/3422671365 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f09640afaf0 0x7f09640b2000 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f097819a510 tx=0x7f0968008040 comp rx=0 tx=0).stop 2024-04-25T11:25:33.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.322+0000 7f097fd72640 1 -- 172.21.15.69:0/3422671365 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09781029e0 msgr2=0x7f0978199470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:33.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.322+0000 7f097fd72640 1 --2- 172.21.15.69:0/3422671365 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09781029e0 0x7f0978199470 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f09740098d0 tx=0x7f0974003ca0 comp rx=0 tx=0).stop 2024-04-25T11:25:33.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.322+0000 7f097fd72640 1 -- 172.21.15.69:0/3422671365 shutdown_connections 2024-04-25T11:25:33.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.322+0000 7f097fd72640 1 --2- 172.21.15.69:0/3422671365 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f09640afaf0 0x7f09640b2000 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:33.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.322+0000 7f097fd72640 1 --2- 172.21.15.69:0/3422671365 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0978108920 0x7f09781a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:33.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.322+0000 7f097fd72640 1 --2- 172.21.15.69:0/3422671365 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0978103c20 0x7f09781999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:33.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.322+0000 7f097fd72640 1 --2- 172.21.15.69:0/3422671365 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09781029e0 0x7f0978199470 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:33.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.322+0000 7f097fd72640 1 -- 172.21.15.69:0/3422671365 >> 172.21.15.69:0/3422671365 conn(0x7f09780fdbd0 msgr2=0x7f0978104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:33.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.322+0000 7f097fd72640 1 -- 172.21.15.69:0/3422671365 shutdown_connections 2024-04-25T11:25:33.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.322+0000 7f097fd72640 1 -- 172.21.15.69:0/3422671365 wait complete. 2024-04-25T11:25:33.326 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:25:33.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:32 smithi146 ceph-mon[32821]: pgmap v342: 129 pgs: 129 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:25:33.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:32 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-25T11:25:33.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:32 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]: dispatch 2024-04-25T11:25:33.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:32 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8, 5, 9]}]: dispatch 2024-04-25T11:25:33.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:32 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-04-25T11:25:33.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:32 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]: dispatch 2024-04-25T11:25:33.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:32 smithi146 ceph-mon[32821]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-25T11:25:33.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:32 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/475634253' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:25:33.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:32 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/3542860219' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:25:33.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:32 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/3665831593' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:25:33.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:32 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-25T11:25:33.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:32 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]': finished 2024-04-25T11:25:33.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:32 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8, 5, 9]}]': finished 2024-04-25T11:25:33.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:32 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-04-25T11:25:33.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:32 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]': finished 2024-04-25T11:25:33.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.441+0000 7f1eb1053640 1 -- 172.21.15.69:0/2680937042 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1eac106f50 msgr2=0x7f1eac109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:33.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.441+0000 7f1eb1053640 1 --2- 172.21.15.69:0/2680937042 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1eac106f50 0x7f1eac109370 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0009900 tx=0x7f1ea002a410 comp rx=0 tx=0).stop 2024-04-25T11:25:33.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.441+0000 7f1eb1053640 1 -- 172.21.15.69:0/2680937042 shutdown_connections 2024-04-25T11:25:33.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.441+0000 7f1eb1053640 1 --2- 172.21.15.69:0/2680937042 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1eac1098b0 0x7f1eac10bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:33.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.441+0000 7f1eb1053640 1 --2- 172.21.15.69:0/2680937042 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1eac106f50 0x7f1eac109370 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:33.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.441+0000 7f1eb1053640 1 --2- 172.21.15.69:0/2680937042 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1eac1045f0 0x7f1eac106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:33.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.441+0000 7f1eb1053640 1 -- 172.21.15.69:0/2680937042 >> 172.21.15.69:0/2680937042 conn(0x7f1eac0fdbf0 msgr2=0x7f1eac1000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:33.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.441+0000 7f1eb1053640 1 -- 172.21.15.69:0/2680937042 shutdown_connections 2024-04-25T11:25:33.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.441+0000 7f1eb1053640 1 -- 172.21.15.69:0/2680937042 wait complete. 2024-04-25T11:25:33.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.442+0000 7f1eb1053640 1 Processor -- start 2024-04-25T11:25:33.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.442+0000 7f1eb1053640 1 -- start start 2024-04-25T11:25:33.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.442+0000 7f1eb1053640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1eac1045f0 0x7f1eac19a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:33.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.442+0000 7f1eb1053640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1eac106f50 0x7f1eac19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:33.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.442+0000 7f1eb1053640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1eac1098b0 0x7f1eac1a1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:33.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.442+0000 7f1eb1053640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac10f4c0 con 0x7f1eac1045f0 2024-04-25T11:25:33.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.442+0000 7f1eb1053640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac10f340 con 0x7f1eac1098b0 2024-04-25T11:25:33.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.442+0000 7f1eb1053640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac10f640 con 0x7f1eac106f50 2024-04-25T11:25:33.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.442+0000 7f1eaad9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1eac1045f0 0x7f1eac19a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:33.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.442+0000 7f1eaad9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1eac1045f0 0x7f1eac19a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55016/0 (socket says 172.21.15.69:55016) 2024-04-25T11:25:33.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.442+0000 7f1eaad9d640 1 -- 172.21.15.69:0/1823119367 learned_addr learned my addr 172.21.15.69:0/1823119367 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:33.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.442+0000 7f1eaa59c640 1 --2- 172.21.15.69:0/1823119367 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1eac106f50 0x7f1eac19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:33.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.442+0000 7f1eab59e640 1 --2- 172.21.15.69:0/1823119367 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1eac1098b0 0x7f1eac1a1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:33.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.442+0000 7f1eaad9d640 1 -- 172.21.15.69:0/1823119367 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1eac106f50 msgr2=0x7f1eac19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:33.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.442+0000 7f1eaad9d640 1 --2- 172.21.15.69:0/1823119367 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1eac106f50 0x7f1eac19a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:33.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.442+0000 7f1eaad9d640 1 -- 172.21.15.69:0/1823119367 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1eac1098b0 msgr2=0x7f1eac1a1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:33.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.442+0000 7f1eaad9d640 1 --2- 172.21.15.69:0/1823119367 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1eac1098b0 0x7f1eac1a1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:33.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.442+0000 7f1eaad9d640 1 -- 172.21.15.69:0/1823119367 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eac1a1c70 con 0x7f1eac1045f0 2024-04-25T11:25:33.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.442+0000 7f1eaa59c640 1 --2- 172.21.15.69:0/1823119367 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1eac106f50 0x7f1eac19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:25:33.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.442+0000 7f1eab59e640 1 --2- 172.21.15.69:0/1823119367 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1eac1098b0 0x7f1eac1a1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:25:33.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.443+0000 7f1eaad9d640 1 --2- 172.21.15.69:0/1823119367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1eac1045f0 0x7f1eac19a130 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c00d770 tx=0x7f1e9c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:33.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.443+0000 7f1e93fff640 1 -- 172.21.15.69:0/1823119367 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c017070 con 0x7f1eac1045f0 2024-04-25T11:25:33.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.443+0000 7f1e93fff640 1 -- 172.21.15.69:0/1823119367 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e9c009c30 con 0x7f1eac1045f0 2024-04-25T11:25:33.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.443+0000 7f1eb1053640 1 -- 172.21.15.69:0/1823119367 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eac1a1f60 con 0x7f1eac1045f0 2024-04-25T11:25:33.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.443+0000 7f1e93fff640 1 -- 172.21.15.69:0/1823119367 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c00f620 con 0x7f1eac1045f0 2024-04-25T11:25:33.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.443+0000 7f1eb1053640 1 -- 172.21.15.69:0/1823119367 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eac1a23c0 con 0x7f1eac1045f0 2024-04-25T11:25:33.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.444+0000 7f1eb1053640 1 -- 172.21.15.69:0/1823119367 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1eac06a190 con 0x7f1eac1045f0 2024-04-25T11:25:33.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.445+0000 7f1e93fff640 1 -- 172.21.15.69:0/1823119367 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f1e9c01a070 con 0x7f1eac1045f0 2024-04-25T11:25:33.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.446+0000 7f1e93fff640 1 --2- 172.21.15.69:0/1823119367 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f1e940afaf0 0x7f1e940b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:33.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.446+0000 7f1eaa59c640 1 --2- 172.21.15.69:0/1823119367 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f1e940afaf0 0x7f1e940b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:33.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.446+0000 7f1e93fff640 1 -- 172.21.15.69:0/1823119367 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9216+0+0 (secure 0 0 0) 0x7f1e9c0deba0 con 0x7f1eac1045f0 2024-04-25T11:25:33.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.446+0000 7f1eaa59c640 1 --2- 172.21.15.69:0/1823119367 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f1e940afaf0 0x7f1e940b2000 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0006010 tx=0x7f1ea0031420 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:33.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.450+0000 7f1e93fff640 1 -- 172.21.15.69:0/1823119367 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1e9c009de0 con 0x7f1eac1045f0 2024-04-25T11:25:33.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.667+0000 7f1eb1053640 1 -- 172.21.15.69:0/1823119367 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1eac0637a0 con 0x7f1eac1045f0 2024-04-25T11:25:33.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.667+0000 7f1e93fff640 1 -- 172.21.15.69:0/1823119367 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11944 (secure 0 0 0) 0x7f1e9c0a73b0 con 0x7f1eac1045f0 2024-04-25T11:25:33.660 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:25:33.660 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3399424094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3399424094},{"type":"v1","addr":"172.21.15.69:6835","nonce":3399424094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1463217771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1463217771},{"type":"v1","addr":"172.21.15.69:6837","nonce":1463217771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T11:23:40.984930+0000","modified":"2024-04-25T11:24:09.061188+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24485,"mds_2":24475,"mds_3":24458,"mds_4":14625},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6839/966316001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":966316001},{"type":"v1","addr":"172.21.15.69:6839","nonce":966316001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.69:6841/2060225181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":2060225181},{"type":"v1","addr":"172.21.15.69:6841","nonce":2060225181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6833/4197482473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":4197482473},{"type":"v1","addr":"172.21.15.152:6833","nonce":4197482473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/4267885323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4267885323},{"type":"v1","addr":"172.21.15.146:6833","nonce":4267885323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6835/3188843362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3188843362},{"type":"v1","addr":"172.21.15.152:6835","nonce":3188843362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6837/563274982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":563274982},{"type":"v1","addr":"172.21.15.152:6837","nonce":563274982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.146:6835/1876533699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1876533699},{"type":"v1","addr":"172.21.15.146:6835","nonce":1876533699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.152:6839/3008615657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":3008615657},{"type":"v1","addr":"172.21.15.152:6839","nonce":3008615657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.146:6837/2730317134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2730317134},{"type":"v1","addr":"172.21.15.146:6837","nonce":2730317134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6839/2013283448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":2013283448},{"type":"v1","addr":"172.21.15.146:6839","nonce":2013283448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24485,24515,24475,24458,14625]},"id":1}]} 2024-04-25T11:25:33.661 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 28 2024-04-25T11:25:33.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.669+0000 7f1eb1053640 1 -- 172.21.15.69:0/1823119367 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f1e940afaf0 msgr2=0x7f1e940b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:33.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.669+0000 7f1eb1053640 1 --2- 172.21.15.69:0/1823119367 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f1e940afaf0 0x7f1e940b2000 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0006010 tx=0x7f1ea0031420 comp rx=0 tx=0).stop 2024-04-25T11:25:33.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.669+0000 7f1eb1053640 1 -- 172.21.15.69:0/1823119367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1eac1045f0 msgr2=0x7f1eac19a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:33.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.669+0000 7f1eb1053640 1 --2- 172.21.15.69:0/1823119367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1eac1045f0 0x7f1eac19a130 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c00d770 tx=0x7f1e9c00dc30 comp rx=0 tx=0).stop 2024-04-25T11:25:33.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.669+0000 7f1eb1053640 1 -- 172.21.15.69:0/1823119367 shutdown_connections 2024-04-25T11:25:33.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.669+0000 7f1eb1053640 1 --2- 172.21.15.69:0/1823119367 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f1e940afaf0 0x7f1e940b2000 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:33.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.669+0000 7f1eb1053640 1 --2- 172.21.15.69:0/1823119367 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1eac1098b0 0x7f1eac1a1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:33.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.669+0000 7f1eb1053640 1 --2- 172.21.15.69:0/1823119367 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1eac106f50 0x7f1eac19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:33.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.669+0000 7f1eb1053640 1 --2- 172.21.15.69:0/1823119367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1eac1045f0 0x7f1eac19a130 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:33.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.669+0000 7f1eb1053640 1 -- 172.21.15.69:0/1823119367 >> 172.21.15.69:0/1823119367 conn(0x7f1eac0fdbf0 msgr2=0x7f1eac105280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:33.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.670+0000 7f1eb1053640 1 -- 172.21.15.69:0/1823119367 shutdown_connections 2024-04-25T11:25:33.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.670+0000 7f1eb1053640 1 -- 172.21.15.69:0/1823119367 wait complete. 2024-04-25T11:25:33.674 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:25:33.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.789+0000 7faaa05b7640 1 -- 172.21.15.69:0/2409571429 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa98106f10 msgr2=0x7faa98109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:33.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.789+0000 7faaa05b7640 1 --2- 172.21.15.69:0/2409571429 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa98106f10 0x7faa98109330 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7faa88009900 tx=0x7faa8802a410 comp rx=0 tx=0).stop 2024-04-25T11:25:33.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.789+0000 7faaa05b7640 1 -- 172.21.15.69:0/2409571429 shutdown_connections 2024-04-25T11:25:33.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.789+0000 7faaa05b7640 1 --2- 172.21.15.69:0/2409571429 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa98109870 0x7faa9810bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:33.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.789+0000 7faaa05b7640 1 --2- 172.21.15.69:0/2409571429 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa98106f10 0x7faa98109330 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:33.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.789+0000 7faaa05b7640 1 --2- 172.21.15.69:0/2409571429 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faa981045b0 0x7faa981069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:33.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.789+0000 7faaa05b7640 1 -- 172.21.15.69:0/2409571429 >> 172.21.15.69:0/2409571429 conn(0x7faa980fdbd0 msgr2=0x7faa98100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:33.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.789+0000 7faaa05b7640 1 -- 172.21.15.69:0/2409571429 shutdown_connections 2024-04-25T11:25:33.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.789+0000 7faaa05b7640 1 -- 172.21.15.69:0/2409571429 wait complete. 2024-04-25T11:25:33.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.789+0000 7faaa05b7640 1 Processor -- start 2024-04-25T11:25:33.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.789+0000 7faaa05b7640 1 -- start start 2024-04-25T11:25:33.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.790+0000 7faaa05b7640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faa981045b0 0x7faa9819a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:33.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.790+0000 7faaa05b7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa98106f10 0x7faa9819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:33.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.790+0000 7faaa05b7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa98109870 0x7faa981a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:33.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.790+0000 7faaa05b7640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7faa9810f4f0 con 0x7faa98106f10 2024-04-25T11:25:33.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.790+0000 7faaa05b7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7faa9810f370 con 0x7faa98109870 2024-04-25T11:25:33.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.790+0000 7faaa05b7640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7faa9810f670 con 0x7faa981045b0 2024-04-25T11:25:33.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.790+0000 7faa9db52640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa98106f10 0x7faa9819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:33.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.790+0000 7faa9db52640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa98106f10 0x7faa9819a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55028/0 (socket says 172.21.15.69:55028) 2024-04-25T11:25:33.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.790+0000 7faa9db52640 1 -- 172.21.15.69:0/3494840236 learned_addr learned my addr 172.21.15.69:0/3494840236 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:33.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.790+0000 7faa9eb54640 1 --2- 172.21.15.69:0/3494840236 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa98109870 0x7faa981a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:33.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.790+0000 7faa9e353640 1 --2- 172.21.15.69:0/3494840236 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faa981045b0 0x7faa9819a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:33.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.790+0000 7faa9db52640 1 -- 172.21.15.69:0/3494840236 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faa981045b0 msgr2=0x7faa9819a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:33.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.790+0000 7faa9db52640 1 --2- 172.21.15.69:0/3494840236 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faa981045b0 0x7faa9819a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:33.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.790+0000 7faa9db52640 1 -- 172.21.15.69:0/3494840236 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa98109870 msgr2=0x7faa981a1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:33.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.790+0000 7faa9db52640 1 --2- 172.21.15.69:0/3494840236 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa98109870 0x7faa981a1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:33.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.790+0000 7faa9db52640 1 -- 172.21.15.69:0/3494840236 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa981a1ca0 con 0x7faa98106f10 2024-04-25T11:25:33.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.790+0000 7faa9eb54640 1 --2- 172.21.15.69:0/3494840236 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa98109870 0x7faa981a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:25:33.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.790+0000 7faa9db52640 1 --2- 172.21.15.69:0/3494840236 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa98106f10 0x7faa9819a6a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7faa88005760 tx=0x7faa88003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:33.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.790+0000 7faa8f7fe640 1 -- 172.21.15.69:0/3494840236 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8803f070 con 0x7faa98106f10 2024-04-25T11:25:33.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.790+0000 7faa8f7fe640 1 -- 172.21.15.69:0/3494840236 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faa88031da0 con 0x7faa98106f10 2024-04-25T11:25:33.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.790+0000 7faaa05b7640 1 -- 172.21.15.69:0/3494840236 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa981a1f30 con 0x7faa98106f10 2024-04-25T11:25:33.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.790+0000 7faa8f7fe640 1 -- 172.21.15.69:0/3494840236 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa88035030 con 0x7faa98106f10 2024-04-25T11:25:33.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.790+0000 7faaa05b7640 1 -- 172.21.15.69:0/3494840236 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa981a2340 con 0x7faa98106f10 2024-04-25T11:25:33.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.792+0000 7faaa05b7640 1 -- 172.21.15.69:0/3494840236 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa980ff6f0 con 0x7faa98106f10 2024-04-25T11:25:33.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.793+0000 7faa8f7fe640 1 -- 172.21.15.69:0/3494840236 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7faa88042070 con 0x7faa98106f10 2024-04-25T11:25:33.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.794+0000 7faa8f7fe640 1 --2- 172.21.15.69:0/3494840236 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7faa840afaf0 0x7faa840b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:33.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.794+0000 7faa9e353640 1 --2- 172.21.15.69:0/3494840236 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7faa840afaf0 0x7faa840b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:33.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.794+0000 7faa8f7fe640 1 -- 172.21.15.69:0/3494840236 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9216+0+0 (secure 0 0 0) 0x7faa88100180 con 0x7faa98106f10 2024-04-25T11:25:33.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.794+0000 7faa9e353640 1 --2- 172.21.15.69:0/3494840236 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7faa840afaf0 0x7faa840b2000 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7faa94005fd0 tx=0x7faa94005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:33.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.797+0000 7faa8f7fe640 1 -- 172.21.15.69:0/3494840236 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faa880c8990 con 0x7faa98106f10 2024-04-25T11:25:33.942 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:33 smithi069 ceph-mon[29188]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T11:25:33.942 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:33 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/850698178' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:25:33.942 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:33 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/3422671365' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T11:25:33.942 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:33 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/1823119367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:25:33.942 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:33 smithi069 ceph-mon[29188]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T11:25:33.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.988+0000 7faaa05b7640 1 -- 172.21.15.69:0/3494840236 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faa980637d0 con 0x7faa98106f10 2024-04-25T11:25:33.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.989+0000 7faa8f7fe640 1 -- 172.21.15.69:0/3494840236 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+24862 (secure 0 0 0) 0x7faa880c8290 con 0x7faa98106f10 2024-04-25T11:25:33.982 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:25:33.983 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":84,"fsid":"f7ed73dc-02f4-11ef-bc93-c7b262605968","created":"2024-04-25T11:15:37.965281+0000","modified":"2024-04-25T11:25:33.888184+0000","last_up_change":"2024-04-25T11:22:50.691869+0000","last_in_change":"2024-04-25T11:22:30.271959+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:18:57.781263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:23:38.685110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:23:39.943803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T11:23:43.650521+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6a0d179-4998-4754-bf82-2ea3dac54e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6803","nonce":3756025235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6805","nonce":3756025235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6809","nonce":3756025235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6807","nonce":3756025235}]},"public_addr":"172.21.15.69:6803/3756025235","cluster_addr":"172.21.15.69:6805/3756025235","heartbeat_back_addr":"172.21.15.69:6809/3756025235","heartbeat_front_addr":"172.21.15.69:6807/3756025235","state":["exists","up"]},{"osd":1,"uuid":"c0c66930-4233-493f-b2bb-99c473994b59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6801","nonce":3166305213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6803","nonce":3166305213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6807","nonce":3166305213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6805","nonce":3166305213}]},"public_addr":"172.21.15.146:6801/3166305213","cluster_addr":"172.21.15.146:6803/3166305213","heartbeat_back_addr":"172.21.15.146:6807/3166305213","heartbeat_front_addr":"172.21.15.146:6805/3166305213","state":["exists","up"]},{"osd":2,"uuid":"d4ef735b-9786-4cb9-83ef-7c3f0691a729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6801","nonce":1474319175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6803","nonce":1474319175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6807","nonce":1474319175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6805","nonce":1474319175}]},"public_addr":"172.21.15.152:6801/1474319175","cluster_addr":"172.21.15.152:6803/1474319175","heartbeat_back_addr":"172.21.15.152:6807/1474319175","heartbeat_front_addr":"172.21.15.152:6805/1474319175","state":["exists","up"]},{"osd":3,"uuid":"0ab85fb8-5bbb-4ffc-bddf-cae7644a64ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6811","nonce":1809071296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6813","nonce":1809071296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6817","nonce":1809071296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6815","nonce":1809071296}]},"public_addr":"172.21.15.69:6811/1809071296","cluster_addr":"172.21.15.69:6813/1809071296","heartbeat_back_addr":"172.21.15.69:6817/1809071296","heartbeat_front_addr":"172.21.15.69:6815/1809071296","state":["exists","up"]},{"osd":4,"uuid":"273e0592-bbbc-4188-8eeb-46a3879cf632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6809","nonce":1898520972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6811","nonce":1898520972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6815","nonce":1898520972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6813","nonce":1898520972}]},"public_addr":"172.21.15.146:6809/1898520972","cluster_addr":"172.21.15.146:6811/1898520972","heartbeat_back_addr":"172.21.15.146:6815/1898520972","heartbeat_front_addr":"172.21.15.146:6813/1898520972","state":["exists","up"]},{"osd":5,"uuid":"bbeae945-9fa2-48f9-a1d6-8bf18dcaef19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6809","nonce":1996555108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6811","nonce":1996555108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6815","nonce":1996555108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6813","nonce":1996555108}]},"public_addr":"172.21.15.152:6809/1996555108","cluster_addr":"172.21.15.152:6811/1996555108","heartbeat_back_addr":"172.21.15.152:6815/1996555108","heartbeat_front_addr":"172.21.15.152:6813/1996555108","state":["exists","up"]},{"osd":6,"uuid":"dbb61253-30bc-418e-bd75-118c80bbf931","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6819","nonce":491215762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6821","nonce":491215762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6825","nonce":491215762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6823","nonce":491215762}]},"public_addr":"172.21.15.69:6819/491215762","cluster_addr":"172.21.15.69:6821/491215762","heartbeat_back_addr":"172.21.15.69:6825/491215762","heartbeat_front_addr":"172.21.15.69:6823/491215762","state":["exists","up"]},{"osd":7,"uuid":"75f23854-e283-4e3e-94cf-7b4079d83876","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6817","nonce":2330294690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6819","nonce":2330294690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6823","nonce":2330294690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6821","nonce":2330294690}]},"public_addr":"172.21.15.146:6817/2330294690","cluster_addr":"172.21.15.146:6819/2330294690","heartbeat_back_addr":"172.21.15.146:6823/2330294690","heartbeat_front_addr":"172.21.15.146:6821/2330294690","state":["exists","up"]},{"osd":8,"uuid":"1aa1060c-0c14-4abd-be56-5ebfeb78759c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6817","nonce":950669381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6819","nonce":950669381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6823","nonce":950669381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6821","nonce":950669381}]},"public_addr":"172.21.15.152:6817/950669381","cluster_addr":"172.21.15.152:6819/950669381","heartbeat_back_addr":"172.21.15.152:6823/950669381","heartbeat_front_addr":"172.21.15.152:6821/950669381","state":["exists","up"]},{"osd":9,"uuid":"3f4bf800-4090-4138-9bb3-d001f13f450c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6827","nonce":2414186758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6829","nonce":2414186758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6833","nonce":2414186758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6831","nonce":2414186758}]},"public_addr":"172.21.15.69:6827/2414186758","cluster_addr":"172.21.15.69:6829/2414186758","heartbeat_back_addr":"172.21.15.69:6833/2414186758","heartbeat_front_addr":"172.21.15.69:6831/2414186758","state":["exists","up"]},{"osd":10,"uuid":"19e967e5-51ce-4506-80c9-0a95441baf99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6825","nonce":707256400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6827","nonce":707256400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6831","nonce":707256400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6829","nonce":707256400}]},"public_addr":"172.21.15.146:6825/707256400","cluster_addr":"172.21.15.146:6827/707256400","heartbeat_back_addr":"172.21.15.146:6831/707256400","heartbeat_front_addr":"172.21.15.146:6829/707256400","state":["exists","up"]},{"osd":11,"uuid":"7788c2de-6930-4808-be0f-f1fd1ae0cb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6825","nonce":2934336833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6827","nonce":2934336833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6831","nonce":2934336833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6829","nonce":2934336833}]},"public_addr":"172.21.15.152:6825/2934336833","cluster_addr":"172.21.15.152:6827/2934336833","heartbeat_back_addr":"172.21.15.152:6831/2934336833","heartbeat_front_addr":"172.21.15.152:6829/2934336833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:07.497448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:31.227494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:55.027978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:20.713437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:46.214704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:20:11.199983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:20:37.177070+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:02.951240+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:28.963680+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:56.095807+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:22:22.221744+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:22:48.929334+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1b","mappings":[{"from":5,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/1372727546":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6801/1097124842":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6800/1508198215":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3103465990":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3540132960":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3899912190":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6800/1097124842":"2024-04-26T11:16:31.688159+0000","172.21.15.69:0/1199907202":"2024-04-26T11:16:04.860792+0000","172.21.15.69:6801/1508198215":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/2879102529":"2024-04-26T11:16:31.688159+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:25:33.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.992+0000 7faaa05b7640 1 -- 172.21.15.69:0/3494840236 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7faa840afaf0 msgr2=0x7faa840b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:33.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.992+0000 7faaa05b7640 1 --2- 172.21.15.69:0/3494840236 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7faa840afaf0 0x7faa840b2000 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7faa94005fd0 tx=0x7faa94005850 comp rx=0 tx=0).stop 2024-04-25T11:25:33.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.992+0000 7faaa05b7640 1 -- 172.21.15.69:0/3494840236 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa98106f10 msgr2=0x7faa9819a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:33.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.992+0000 7faaa05b7640 1 --2- 172.21.15.69:0/3494840236 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa98106f10 0x7faa9819a6a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7faa88005760 tx=0x7faa88003d50 comp rx=0 tx=0).stop 2024-04-25T11:25:33.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.992+0000 7faaa05b7640 1 -- 172.21.15.69:0/3494840236 shutdown_connections 2024-04-25T11:25:33.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.992+0000 7faaa05b7640 1 --2- 172.21.15.69:0/3494840236 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7faa840afaf0 0x7faa840b2000 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:33.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.992+0000 7faaa05b7640 1 --2- 172.21.15.69:0/3494840236 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faa98109870 0x7faa981a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:33.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.992+0000 7faaa05b7640 1 --2- 172.21.15.69:0/3494840236 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa98106f10 0x7faa9819a6a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:33.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.992+0000 7faaa05b7640 1 --2- 172.21.15.69:0/3494840236 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faa981045b0 0x7faa9819a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:33.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.992+0000 7faaa05b7640 1 -- 172.21.15.69:0/3494840236 >> 172.21.15.69:0/3494840236 conn(0x7faa980fdbd0 msgr2=0x7faa98105240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:33.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.992+0000 7faaa05b7640 1 -- 172.21.15.69:0/3494840236 shutdown_connections 2024-04-25T11:25:33.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:33.992+0000 7faaa05b7640 1 -- 172.21.15.69:0/3494840236 wait complete. 2024-04-25T11:25:33.996 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:25:34.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.112+0000 7f849520c640 1 -- 172.21.15.69:0/1694886673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8490104d40 msgr2=0x7f8490105150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:34.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.112+0000 7f849520c640 1 --2- 172.21.15.69:0/1694886673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8490104d40 0x7f8490105150 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8478009900 tx=0x7f847802a410 comp rx=0 tx=0).stop 2024-04-25T11:25:34.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.112+0000 7f849520c640 1 -- 172.21.15.69:0/1694886673 shutdown_connections 2024-04-25T11:25:34.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.112+0000 7f849520c640 1 --2- 172.21.15.69:0/1694886673 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8490109a40 0x7f849010be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.112+0000 7f849520c640 1 --2- 172.21.15.69:0/1694886673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8490104d40 0x7f8490105150 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.112+0000 7f849520c640 1 --2- 172.21.15.69:0/1694886673 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8490103b00 0x7f8490104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.112+0000 7f849520c640 1 -- 172.21.15.69:0/1694886673 >> 172.21.15.69:0/1694886673 conn(0x7f84900fed10 msgr2=0x7f84901011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:34.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.112+0000 7f849520c640 1 -- 172.21.15.69:0/1694886673 shutdown_connections 2024-04-25T11:25:34.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.112+0000 7f849520c640 1 -- 172.21.15.69:0/1694886673 wait complete. 2024-04-25T11:25:34.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.113+0000 7f849520c640 1 Processor -- start 2024-04-25T11:25:34.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.113+0000 7f849520c640 1 -- start start 2024-04-25T11:25:34.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.113+0000 7f849520c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8490103b00 0x7f849019a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.113+0000 7f849520c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8490104d40 0x7f849019a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.113+0000 7f849520c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8490109a40 0x7f84901a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.113+0000 7f849520c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f849010f4d0 con 0x7f8490103b00 2024-04-25T11:25:34.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.113+0000 7f849520c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f849010f350 con 0x7f8490104d40 2024-04-25T11:25:34.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.113+0000 7f849520c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f849010f650 con 0x7f8490109a40 2024-04-25T11:25:34.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.113+0000 7f848ed9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8490103b00 0x7f849019a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.113+0000 7f848ed9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8490103b00 0x7f849019a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55040/0 (socket says 172.21.15.69:55040) 2024-04-25T11:25:34.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.113+0000 7f848ed9d640 1 -- 172.21.15.69:0/793316877 learned_addr learned my addr 172.21.15.69:0/793316877 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:34.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.113+0000 7f848f59e640 1 --2- 172.21.15.69:0/793316877 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8490109a40 0x7f84901a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.113+0000 7f8487fff640 1 --2- 172.21.15.69:0/793316877 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8490104d40 0x7f849019a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.114+0000 7f848ed9d640 1 -- 172.21.15.69:0/793316877 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8490109a40 msgr2=0x7f84901a1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:34.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.114+0000 7f848ed9d640 1 --2- 172.21.15.69:0/793316877 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8490109a40 0x7f84901a1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.114+0000 7f848ed9d640 1 -- 172.21.15.69:0/793316877 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8490104d40 msgr2=0x7f849019a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:34.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.114+0000 7f848ed9d640 1 --2- 172.21.15.69:0/793316877 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8490104d40 0x7f849019a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.114+0000 7f848ed9d640 1 -- 172.21.15.69:0/793316877 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84901a1bd0 con 0x7f8490103b00 2024-04-25T11:25:34.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.114+0000 7f848f59e640 1 --2- 172.21.15.69:0/793316877 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8490109a40 0x7f84901a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:25:34.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.114+0000 7f848ed9d640 1 --2- 172.21.15.69:0/793316877 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8490103b00 0x7f849019a190 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8480005d00 tx=0x7f848000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:34.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.114+0000 7f848cd99640 1 -- 172.21.15.69:0/793316877 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8480017070 con 0x7f8490103b00 2024-04-25T11:25:34.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.114+0000 7f848cd99640 1 -- 172.21.15.69:0/793316877 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8480009c30 con 0x7f8490103b00 2024-04-25T11:25:34.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.114+0000 7f849520c640 1 -- 172.21.15.69:0/793316877 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84901a1ec0 con 0x7f8490103b00 2024-04-25T11:25:34.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.114+0000 7f848cd99640 1 -- 172.21.15.69:0/793316877 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848000f620 con 0x7f8490103b00 2024-04-25T11:25:34.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.114+0000 7f849520c640 1 -- 172.21.15.69:0/793316877 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84901a2320 con 0x7f8490103b00 2024-04-25T11:25:34.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.115+0000 7f849520c640 1 -- 172.21.15.69:0/793316877 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8490108850 con 0x7f8490103b00 2024-04-25T11:25:34.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.116+0000 7f848cd99640 1 -- 172.21.15.69:0/793316877 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f848001a070 con 0x7f8490103b00 2024-04-25T11:25:34.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.117+0000 7f848cd99640 1 --2- 172.21.15.69:0/793316877 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f847c0afaf0 0x7f847c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.117+0000 7f848cd99640 1 -- 172.21.15.69:0/793316877 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f84800dead0 con 0x7f8490103b00 2024-04-25T11:25:34.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.117+0000 7f8487fff640 1 --2- 172.21.15.69:0/793316877 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f847c0afaf0 0x7f847c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.117+0000 7f8487fff640 1 --2- 172.21.15.69:0/793316877 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f847c0afaf0 0x7f847c0b2000 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8478006010 tx=0x7f8478000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:34.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.121+0000 7f848cd99640 1 -- 172.21.15.69:0/793316877 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8480009de0 con 0x7f8490103b00 2024-04-25T11:25:34.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:33 smithi152 ceph-mon[32701]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T11:25:34.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:33 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/850698178' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:25:34.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:33 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/3422671365' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T11:25:34.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:33 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/1823119367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:25:34.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:33 smithi152 ceph-mon[32701]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T11:25:34.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.335+0000 7f849520c640 1 -- 172.21.15.69:0/793316877 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f849002dc70 con 0x7f8490103b00 2024-04-25T11:25:34.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.336+0000 7f848cd99640 1 -- 172.21.15.69:0/793316877 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11944 (secure 0 0 0) 0x7f84800a72e0 con 0x7f8490103b00 2024-04-25T11:25:34.328 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:25:34.329 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3399424094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3399424094},{"type":"v1","addr":"172.21.15.69:6835","nonce":3399424094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1463217771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1463217771},{"type":"v1","addr":"172.21.15.69:6837","nonce":1463217771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T11:23:40.984930+0000","modified":"2024-04-25T11:24:09.061188+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24485,"mds_2":24475,"mds_3":24458,"mds_4":14625},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6839/966316001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":966316001},{"type":"v1","addr":"172.21.15.69:6839","nonce":966316001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.69:6841/2060225181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":2060225181},{"type":"v1","addr":"172.21.15.69:6841","nonce":2060225181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6833/4197482473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":4197482473},{"type":"v1","addr":"172.21.15.152:6833","nonce":4197482473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/4267885323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4267885323},{"type":"v1","addr":"172.21.15.146:6833","nonce":4267885323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6835/3188843362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3188843362},{"type":"v1","addr":"172.21.15.152:6835","nonce":3188843362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6837/563274982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":563274982},{"type":"v1","addr":"172.21.15.152:6837","nonce":563274982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.146:6835/1876533699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1876533699},{"type":"v1","addr":"172.21.15.146:6835","nonce":1876533699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.152:6839/3008615657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":3008615657},{"type":"v1","addr":"172.21.15.152:6839","nonce":3008615657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.146:6837/2730317134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2730317134},{"type":"v1","addr":"172.21.15.146:6837","nonce":2730317134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6839/2013283448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":2013283448},{"type":"v1","addr":"172.21.15.146:6839","nonce":2013283448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24485,24515,24475,24458,14625]},"id":1}]} 2024-04-25T11:25:34.329 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 28 2024-04-25T11:25:34.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.338+0000 7f849520c640 1 -- 172.21.15.69:0/793316877 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f847c0afaf0 msgr2=0x7f847c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:34.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.338+0000 7f849520c640 1 --2- 172.21.15.69:0/793316877 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f847c0afaf0 0x7f847c0b2000 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8478006010 tx=0x7f8478000bc0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.338+0000 7f849520c640 1 -- 172.21.15.69:0/793316877 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8490103b00 msgr2=0x7f849019a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:34.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.338+0000 7f849520c640 1 --2- 172.21.15.69:0/793316877 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8490103b00 0x7f849019a190 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8480005d00 tx=0x7f848000dde0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.338+0000 7f849520c640 1 -- 172.21.15.69:0/793316877 shutdown_connections 2024-04-25T11:25:34.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.338+0000 7f849520c640 1 --2- 172.21.15.69:0/793316877 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f847c0afaf0 0x7f847c0b2000 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.338+0000 7f849520c640 1 --2- 172.21.15.69:0/793316877 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8490109a40 0x7f84901a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.338+0000 7f849520c640 1 --2- 172.21.15.69:0/793316877 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8490104d40 0x7f849019a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.338+0000 7f849520c640 1 --2- 172.21.15.69:0/793316877 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8490103b00 0x7f849019a190 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.338+0000 7f849520c640 1 -- 172.21.15.69:0/793316877 >> 172.21.15.69:0/793316877 conn(0x7f84900fed10 msgr2=0x7f8490105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:34.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.338+0000 7f849520c640 1 -- 172.21.15.69:0/793316877 shutdown_connections 2024-04-25T11:25:34.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.338+0000 7f849520c640 1 -- 172.21.15.69:0/793316877 wait complete. 2024-04-25T11:25:34.342 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T11:23:40.984930+0000', 'modified': '2024-04-25T11:24:09.061188+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24515, 'mds_1': 24485, 'mds_2': 24475, 'mds_3': 24458, 'mds_4': 14625}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14613': {'gid': 14613, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.69:6839/966316001', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6838', 'nonce': 966316001}, {'type': 'v1', 'addr': '172.21.15.69:6839', 'nonce': 966316001}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14625': {'gid': 14625, 'name': 'a', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.69:6841/2060225181', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6840', 'nonce': 2060225181}, {'type': 'v1', 'addr': '172.21.15.69:6841', 'nonce': 2060225181}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24391': {'gid': 24391, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.152:6833/4197482473', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6832', 'nonce': 4197482473}, {'type': 'v1', 'addr': '172.21.15.152:6833', 'nonce': 4197482473}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24425': {'gid': 24425, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.146:6833/4267885323', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6832', 'nonce': 4267885323}, {'type': 'v1', 'addr': '172.21.15.146:6833', 'nonce': 4267885323}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24439': {'gid': 24439, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.152:6835/3188843362', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6834', 'nonce': 3188843362}, {'type': 'v1', 'addr': '172.21.15.152:6835', 'nonce': 3188843362}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.152:6837/563274982', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6836', 'nonce': 563274982}, {'type': 'v1', 'addr': '172.21.15.152:6837', 'nonce': 563274982}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24458': {'gid': 24458, 'name': 'e', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.146:6835/1876533699', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6834', 'nonce': 1876533699}, {'type': 'v1', 'addr': '172.21.15.146:6835', 'nonce': 1876533699}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24475': {'gid': 24475, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.152:6839/3008615657', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6838', 'nonce': 3008615657}, {'type': 'v1', 'addr': '172.21.15.152:6839', 'nonce': 3008615657}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24485': {'gid': 24485, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.146:6837/2730317134', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6836', 'nonce': 2730317134}, {'type': 'v1', 'addr': '172.21.15.146:6837', 'nonce': 2730317134}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24515': {'gid': 24515, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.146:6839/2013283448', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6838', 'nonce': 2013283448}, {'type': 'v1', 'addr': '172.21.15.146:6839', 'nonce': 2013283448}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24515, 'qdb_cluster': [24485, 24515, 24475, 24458, 14625]} 2024-04-25T11:25:34.342 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-25T11:25:34.343 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T11:25:34.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:33 smithi146 ceph-mon[32821]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T11:25:34.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:33 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/850698178' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:25:34.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:33 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/3422671365' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T11:25:34.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:33 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/1823119367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:25:34.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:33 smithi146 ceph-mon[32821]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T11:25:34.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.458+0000 7f90d9042640 1 -- 172.21.15.69:0/4253093858 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90d4104d30 msgr2=0x7f90d4105140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:34.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.458+0000 7f90d9042640 1 --2- 172.21.15.69:0/4253093858 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90d4104d30 0x7f90d4105140 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f90bc009900 tx=0x7f90bc02a410 comp rx=0 tx=0).stop 2024-04-25T11:25:34.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.458+0000 7f90d9042640 1 -- 172.21.15.69:0/4253093858 shutdown_connections 2024-04-25T11:25:34.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.458+0000 7f90d9042640 1 --2- 172.21.15.69:0/4253093858 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90d4109a30 0x7f90d410be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.458+0000 7f90d9042640 1 --2- 172.21.15.69:0/4253093858 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90d4104d30 0x7f90d4105140 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.458+0000 7f90d9042640 1 --2- 172.21.15.69:0/4253093858 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90d4103af0 0x7f90d41047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.458+0000 7f90d9042640 1 -- 172.21.15.69:0/4253093858 >> 172.21.15.69:0/4253093858 conn(0x7f90d40fece0 msgr2=0x7f90d4101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:34.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.458+0000 7f90d9042640 1 -- 172.21.15.69:0/4253093858 shutdown_connections 2024-04-25T11:25:34.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.458+0000 7f90d9042640 1 -- 172.21.15.69:0/4253093858 wait complete. 2024-04-25T11:25:34.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.459+0000 7f90d9042640 1 Processor -- start 2024-04-25T11:25:34.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.459+0000 7f90d9042640 1 -- start start 2024-04-25T11:25:34.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.459+0000 7f90d9042640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90d4103af0 0x7f90d419a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.459+0000 7f90d9042640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90d4104d30 0x7f90d419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.459+0000 7f90d9042640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90d4109a30 0x7f90d41a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.460+0000 7f90d9042640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f90d410f520 con 0x7f90d4104d30 2024-04-25T11:25:34.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.460+0000 7f90d9042640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f90d410f3a0 con 0x7f90d4109a30 2024-04-25T11:25:34.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.460+0000 7f90d9042640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f90d410f6a0 con 0x7f90d4103af0 2024-04-25T11:25:34.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.460+0000 7f90ca59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90d4104d30 0x7f90d419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.460+0000 7f90ca59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90d4104d30 0x7f90d419a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55056/0 (socket says 172.21.15.69:55056) 2024-04-25T11:25:34.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.460+0000 7f90ca59c640 1 -- 172.21.15.69:0/1214889620 learned_addr learned my addr 172.21.15.69:0/1214889620 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:34.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.460+0000 7f90d2d9d640 1 --2- 172.21.15.69:0/1214889620 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90d4103af0 0x7f90d419a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.460+0000 7f90d359e640 1 --2- 172.21.15.69:0/1214889620 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90d4109a30 0x7f90d41a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.460+0000 7f90ca59c640 1 -- 172.21.15.69:0/1214889620 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90d4103af0 msgr2=0x7f90d419a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:34.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.460+0000 7f90ca59c640 1 --2- 172.21.15.69:0/1214889620 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90d4103af0 0x7f90d419a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.460+0000 7f90ca59c640 1 -- 172.21.15.69:0/1214889620 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90d4109a30 msgr2=0x7f90d41a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:34.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.460+0000 7f90ca59c640 1 --2- 172.21.15.69:0/1214889620 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90d4109a30 0x7f90d41a1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.460+0000 7f90ca59c640 1 -- 172.21.15.69:0/1214889620 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90d41a1be0 con 0x7f90d4104d30 2024-04-25T11:25:34.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.460+0000 7f90ca59c640 1 --2- 172.21.15.69:0/1214889620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90d4104d30 0x7f90d419a690 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f90bc009230 tx=0x7f90bc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:34.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.460+0000 7f90d0d99640 1 -- 172.21.15.69:0/1214889620 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90bc03f030 con 0x7f90d4104d30 2024-04-25T11:25:34.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.460+0000 7f90d0d99640 1 -- 172.21.15.69:0/1214889620 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90bc003c90 con 0x7f90d4104d30 2024-04-25T11:25:34.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.460+0000 7f90d9042640 1 -- 172.21.15.69:0/1214889620 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90d41a1e70 con 0x7f90d4104d30 2024-04-25T11:25:34.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.460+0000 7f90d0d99640 1 -- 172.21.15.69:0/1214889620 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90bc035070 con 0x7f90d4104d30 2024-04-25T11:25:34.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.460+0000 7f90d9042640 1 -- 172.21.15.69:0/1214889620 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90d41a2300 con 0x7f90d4104d30 2024-04-25T11:25:34.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.463+0000 7f90d0d99640 1 -- 172.21.15.69:0/1214889620 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f90bc042030 con 0x7f90d4104d30 2024-04-25T11:25:34.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.463+0000 7f90dffce740 1 Processor -- start 2024-04-25T11:25:34.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.463+0000 7f90dffce740 1 -- start start 2024-04-25T11:25:34.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.463+0000 7f90dffce740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d935271270 0x55d935612370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.463+0000 7f90dffce740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d9352483b0 0x55d9356128f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.463+0000 7f90dffce740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d935250590 0x55d9356197f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.463+0000 7f90dffce740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d935577740 con 0x55d9352483b0 2024-04-25T11:25:34.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.463+0000 7f90dffce740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55d9355133a0 con 0x55d935250590 2024-04-25T11:25:34.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.463+0000 7f90dffce740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55d935573700 con 0x55d935271270 2024-04-25T11:25:34.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.463+0000 7f90d2d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d9352483b0 0x55d9356128f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.463+0000 7f90d2d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d9352483b0 0x55d9356128f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55068/0 (socket says 172.21.15.69:55068) 2024-04-25T11:25:34.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.463+0000 7f90d2d9d640 1 -- 172.21.15.69:0/2498661626 learned_addr learned my addr 172.21.15.69:0/2498661626 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:34.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.464+0000 7f90d359e640 1 --2- 172.21.15.69:0/2498661626 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d935271270 0x55d935612370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.464+0000 7f90d0d99640 1 --2- 172.21.15.69:0/1214889620 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f90c00afaf0 0x7f90c00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.464+0000 7f90ca59c640 1 --2- 172.21.15.69:0/2498661626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d935250590 0x55d9356197f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.464+0000 7f90d2d9d640 1 -- 172.21.15.69:0/2498661626 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d935271270 msgr2=0x55d935612370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:25:34.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.464+0000 7f90d2d9d640 1 --2- 172.21.15.69:0/2498661626 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d935271270 0x55d935612370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.464+0000 7f90d2d9d640 1 -- 172.21.15.69:0/2498661626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d935250590 msgr2=0x55d9356197f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:25:34.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.464+0000 7f90d2d9d640 1 --2- 172.21.15.69:0/2498661626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d935250590 0x55d9356197f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.464+0000 7f90d2d9d640 1 -- 172.21.15.69:0/2498661626 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d93552f750 con 0x55d9352483b0 2024-04-25T11:25:34.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.464+0000 7f90d359e640 1 --2- 172.21.15.69:0/2498661626 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d935271270 0x55d935612370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:25:34.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.464+0000 7f90d359e640 1 --2- 172.21.15.69:0/1214889620 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f90c00afaf0 0x7f90c00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.464+0000 7f90d0d99640 1 -- 172.21.15.69:0/1214889620 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f90bc0ff6a0 con 0x7f90d4104d30 2024-04-25T11:25:34.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.464+0000 7f90d2d9d640 1 --2- 172.21.15.69:0/2498661626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d9352483b0 0x55d9356128f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f90c400ac40 tx=0x7f90c400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6778f2d64d3b663d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:34.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.464+0000 7f90c959a640 1 -- 172.21.15.69:0/2498661626 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c4005220 con 0x55d9352483b0 2024-04-25T11:25:34.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.464+0000 7f90c959a640 1 -- 172.21.15.69:0/2498661626 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90c40047a0 con 0x55d9352483b0 2024-04-25T11:25:34.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.464+0000 7f90c959a640 1 -- 172.21.15.69:0/2498661626 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c40186a0 con 0x55d9352483b0 2024-04-25T11:25:34.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.465+0000 7f90dffce740 1 -- 172.21.15.69:0/2498661626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d9352483b0 msgr2=0x55d9356128f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:34.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.465+0000 7f90d359e640 1 --2- 172.21.15.69:0/1214889620 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f90c00afaf0 0x7f90c00b2000 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x55d935613540 tx=0x7f90cc004b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:34.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.465+0000 7f90dffce740 1 --2- 172.21.15.69:0/2498661626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d9352483b0 0x55d9356128f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f90c400ac40 tx=0x7f90c400f790 comp rx=0 tx=0).stop 2024-04-25T11:25:34.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.465+0000 7f90dffce740 1 -- 172.21.15.69:0/2498661626 shutdown_connections 2024-04-25T11:25:34.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.465+0000 7f90dffce740 1 --2- 172.21.15.69:0/2498661626 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d935250590 0x55d9356197f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.465+0000 7f90dffce740 1 --2- 172.21.15.69:0/2498661626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d9352483b0 0x55d9356128f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.465+0000 7f90dffce740 1 --2- 172.21.15.69:0/2498661626 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d935271270 0x55d935612370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.465+0000 7f90dffce740 1 -- 172.21.15.69:0/2498661626 >> 172.21.15.69:0/2498661626 conn(0x55d9352653c0 msgr2=0x55d93560dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:34.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.465+0000 7f90dffce740 1 -- 172.21.15.69:0/2498661626 shutdown_connections 2024-04-25T11:25:34.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.465+0000 7f90dffce740 1 -- 172.21.15.69:0/2498661626 wait complete. 2024-04-25T11:25:34.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.466+0000 7f90dffce740 1 -- start start 2024-04-25T11:25:34.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.466+0000 7f90dffce740 1 Processor -- start 2024-04-25T11:25:34.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.466+0000 7f90dffce740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9352483b0 0x55d93561c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.466+0000 7f90dffce740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d935250590 0x55d93561c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.466+0000 7f90dffce740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d935271270 0x55d9356236f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.466+0000 7f90dffce740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d93561a240 con 0x55d935271270 2024-04-25T11:25:34.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.466+0000 7f90dffce740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55d9354fefe0 con 0x55d935250590 2024-04-25T11:25:34.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.466+0000 7f90dffce740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55d93561a3c0 con 0x55d9352483b0 2024-04-25T11:25:34.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.466+0000 7f90dffce740 10 client.0 _ll_register_callbacks cb 0x55d935515150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:25:34.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.466+0000 7f90d359e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d935271270 0x55d9356236f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.466+0000 7f90d359e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d935271270 0x55d9356236f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55074/0 (socket says 172.21.15.69:55074) 2024-04-25T11:25:34.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.466+0000 7f90d359e640 1 -- 172.21.15.69:0/3657765201 learned_addr learned my addr 172.21.15.69:0/3657765201 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:34.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.466+0000 7f90d2d9d640 1 --2- 172.21.15.69:0/3657765201 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9352483b0 0x55d93561c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.466+0000 7f90d359e640 1 -- 172.21.15.69:0/3657765201 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9352483b0 msgr2=0x55d93561c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:25:34.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.466+0000 7f90d359e640 1 --2- 172.21.15.69:0/3657765201 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9352483b0 0x55d93561c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.466+0000 7f90ca59c640 1 --2- 172.21.15.69:0/3657765201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d935250590 0x55d93561c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.466+0000 7f90d359e640 1 -- 172.21.15.69:0/3657765201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d935250590 msgr2=0x55d93561c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:25:34.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.466+0000 7f90d359e640 1 --2- 172.21.15.69:0/3657765201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d935250590 0x55d93561c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.466+0000 7f90d359e640 1 -- 172.21.15.69:0/3657765201 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9352892d0 con 0x55d935271270 2024-04-25T11:25:34.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.467+0000 7f90d359e640 1 --2- 172.21.15.69:0/3657765201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d935271270 0x55d9356236f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f90cc007e30 tx=0x7f90cc00d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dc7f6140ebb16a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:34.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.467+0000 7f90ba7fc640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-25T11:25:34.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.467+0000 7f90ba7fc640 1 -- 172.21.15.69:0/3657765201 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90cc01c050 con 0x55d935271270 2024-04-25T11:25:34.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.467+0000 7f90ba7fc640 1 -- 172.21.15.69:0/3657765201 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f90cc0205b0 con 0x55d935271270 2024-04-25T11:25:34.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.467+0000 7f90dffce740 1 -- 172.21.15.69:0/3657765201 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90c40047a0 con 0x55d935271270 2024-04-25T11:25:34.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.467+0000 7f90ba7fc640 1 -- 172.21.15.69:0/3657765201 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f90cc02f050 con 0x55d935271270 2024-04-25T11:25:34.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.467+0000 7f90dffce740 10 client.15216 fetch_fsmap learned FSMap version 28 2024-04-25T11:25:34.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.467+0000 7f90dffce740 1 -- 172.21.15.69:0/3657765201 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d9353536b0 con 0x55d935271270 2024-04-25T11:25:34.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.467+0000 7f90ba7fc640 1 -- 172.21.15.69:0/3657765201 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90c40047a0 con 0x55d935271270 2024-04-25T11:25:34.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.467+0000 7f90ba7fc640 1 -- 172.21.15.69:0/3657765201 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f90cc021970 con 0x55d935271270 2024-04-25T11:25:34.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.467+0000 7f90dffce740 10 client.15216 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T11:25:34.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.467+0000 7f90dffce740 10 client.15216 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T11:25:34.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.467+0000 7f90dffce740 20 client.15216 populate_metadata read hostname 'smithi069' 2024-04-25T11:25:34.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.467+0000 7f90dffce740 1 --2- 172.21.15.69:0/3657765201 >> [v2:172.21.15.69:6840/2060225181,v1:172.21.15.69:6841/2060225181] conn(0x55d93522fe40 0x55d93562a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.467+0000 7f90dffce740 4 client.15216 mds_command: new command op to 14625 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:25:34.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.467+0000 7f90dffce740 1 -- 172.21.15.69:0/3657765201 --> [v2:172.21.15.69:6840/2060225181,v1:172.21.15.69:6841/2060225181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d935585350 con 0x55d93522fe40 2024-04-25T11:25:34.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.467+0000 7f90d2d9d640 1 --2- 172.21.15.69:0/3657765201 >> [v2:172.21.15.69:6840/2060225181,v1:172.21.15.69:6841/2060225181] conn(0x55d93522fe40 0x55d93562a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.468+0000 7f90ba7fc640 10 client.15216 ms_handle_connect on v2:172.21.15.69:6840/2060225181 2024-04-25T11:25:34.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.468+0000 7f90d2d9d640 1 --2- 172.21.15.69:0/3657765201 >> [v2:172.21.15.69:6840/2060225181,v1:172.21.15.69:6841/2060225181] conn(0x55d93522fe40 0x55d93562a010 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cfa20045eafb4bb4 server_cookie=cb6e20e328ad0bc6 in_seq=0 out_seq=0 2024-04-25T11:25:34.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.468+0000 7f90ba7fc640 1 -- 172.21.15.69:0/3657765201 <== mds.4 v2:172.21.15.69:6840/2060225181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d935585350 con 0x55d93522fe40 2024-04-25T11:25:34.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.468+0000 7f90ba7fc640 10 client.15216 handle_command_reply: tid=0 2024-04-25T11:25:34.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.469+0000 7f90dffce740 1 client.15216 shutdown 2024-04-25T11:25:34.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.469+0000 7f90dffce740 1 -- 172.21.15.69:0/3657765201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d935271270 msgr2=0x55d9356236f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:34.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.469+0000 7f90dffce740 1 --2- 172.21.15.69:0/3657765201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d935271270 0x55d9356236f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f90cc007e30 tx=0x7f90cc00d040 comp rx=0 tx=0).stop 2024-04-25T11:25:34.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.469+0000 7f90dffce740 1 -- 172.21.15.69:0/3657765201 shutdown_connections 2024-04-25T11:25:34.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.469+0000 7f90dffce740 1 --2- 172.21.15.69:0/3657765201 >> [v2:172.21.15.69:6840/2060225181,v1:172.21.15.69:6841/2060225181] conn(0x55d93522fe40 0x55d93562a010 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.469+0000 7f90dffce740 1 --2- 172.21.15.69:0/3657765201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d935271270 0x55d9356236f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.469+0000 7f90dffce740 1 --2- 172.21.15.69:0/3657765201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d935250590 0x55d93561c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.469+0000 7f90dffce740 1 --2- 172.21.15.69:0/3657765201 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9352483b0 0x55d93561c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.469+0000 7f90ba7fc640 0 client.15216 ms_handle_reset on v2:172.21.15.69:6840/2060225181 2024-04-25T11:25:34.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.469+0000 7f90dffce740 1 -- 172.21.15.69:0/3657765201 >> 172.21.15.69:0/3657765201 conn(0x55d9352653c0 msgr2=0x55d93560cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:34.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.469+0000 7f90dffce740 1 -- 172.21.15.69:0/3657765201 shutdown_connections 2024-04-25T11:25:34.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.469+0000 7f90dffce740 1 -- 172.21.15.69:0/3657765201 wait complete. 2024-04-25T11:25:34.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.469+0000 7f90dffce740 20 client.15216 trim_cache size 0 max 16384 2024-04-25T11:25:34.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.487+0000 7f90dffce740 1 Processor -- start 2024-04-25T11:25:34.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.487+0000 7f90dffce740 1 -- start start 2024-04-25T11:25:34.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.488+0000 7f90dffce740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9352483b0 0x55d93561a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.488+0000 7f90dffce740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d935250590 0x55d935623610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.488+0000 7f90dffce740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d935271270 0x55d93561acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.488+0000 7f90dffce740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d935628350 con 0x55d935250590 2024-04-25T11:25:34.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.488+0000 7f90dffce740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55d935628a20 con 0x55d935271270 2024-04-25T11:25:34.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.488+0000 7f90dffce740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55d9356290f0 con 0x55d9352483b0 2024-04-25T11:25:34.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.488+0000 7f90ca59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d935250590 0x55d935623610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.488+0000 7f90ca59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d935250590 0x55d935623610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55090/0 (socket says 172.21.15.69:55090) 2024-04-25T11:25:34.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.488+0000 7f90ca59c640 1 -- 172.21.15.69:0/1892589457 learned_addr learned my addr 172.21.15.69:0/1892589457 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:34.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.488+0000 7f90d359e640 1 --2- 172.21.15.69:0/1892589457 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d935271270 0x55d93561acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.488+0000 7f90d2d9d640 1 --2- 172.21.15.69:0/1892589457 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9352483b0 0x55d93561a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.488+0000 7f90ca59c640 1 -- 172.21.15.69:0/1892589457 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9352483b0 msgr2=0x55d93561a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:25:34.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.488+0000 7f90ca59c640 1 --2- 172.21.15.69:0/1892589457 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9352483b0 0x55d93561a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.488+0000 7f90ca59c640 1 -- 172.21.15.69:0/1892589457 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d935271270 msgr2=0x55d93561acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:25:34.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.488+0000 7f90ca59c640 1 --2- 172.21.15.69:0/1892589457 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d935271270 0x55d93561acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.488+0000 7f90ca59c640 1 -- 172.21.15.69:0/1892589457 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d935611f20 con 0x55d935250590 2024-04-25T11:25:34.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.488+0000 7f90ca59c640 1 --2- 172.21.15.69:0/1892589457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d935250590 0x55d935623610 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f90bc0c38a0 tx=0x7f90bc0fcc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc7891d8cd7f544e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:34.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.489+0000 7f90c8d99640 1 -- 172.21.15.69:0/1892589457 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90bc049070 con 0x55d935250590 2024-04-25T11:25:34.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.489+0000 7f90c8d99640 1 -- 172.21.15.69:0/1892589457 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90bc003e40 con 0x55d935250590 2024-04-25T11:25:34.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.489+0000 7f90c8d99640 1 -- 172.21.15.69:0/1892589457 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90bc0c7330 con 0x55d935250590 2024-04-25T11:25:34.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.489+0000 7f90dffce740 1 -- 172.21.15.69:0/1892589457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d935250590 msgr2=0x55d935623610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:34.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.489+0000 7f90dffce740 1 --2- 172.21.15.69:0/1892589457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d935250590 0x55d935623610 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f90bc0c38a0 tx=0x7f90bc0fcc60 comp rx=0 tx=0).stop 2024-04-25T11:25:34.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.489+0000 7f90dffce740 1 -- 172.21.15.69:0/1892589457 shutdown_connections 2024-04-25T11:25:34.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.489+0000 7f90dffce740 1 --2- 172.21.15.69:0/1892589457 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d935271270 0x55d93561acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.490+0000 7f90dffce740 1 --2- 172.21.15.69:0/1892589457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d935250590 0x55d935623610 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.490+0000 7f90dffce740 1 --2- 172.21.15.69:0/1892589457 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9352483b0 0x55d93561a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.490+0000 7f90dffce740 1 -- 172.21.15.69:0/1892589457 >> 172.21.15.69:0/1892589457 conn(0x55d9352653c0 msgr2=0x55d935616690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:34.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.490+0000 7f90dffce740 1 -- 172.21.15.69:0/1892589457 shutdown_connections 2024-04-25T11:25:34.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.490+0000 7f90dffce740 1 -- 172.21.15.69:0/1892589457 wait complete. 2024-04-25T11:25:34.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.490+0000 7f90dffce740 1 -- start start 2024-04-25T11:25:34.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.490+0000 7f90dffce740 1 Processor -- start 2024-04-25T11:25:34.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.490+0000 7f90dffce740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d9352483b0 0x55d93561ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.490+0000 7f90dffce740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d935250590 0x55d93561f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.490+0000 7f90dffce740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d935271270 0x55d935622040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.490+0000 7f90dffce740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d935629e90 con 0x55d935271270 2024-04-25T11:25:34.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.490+0000 7f90dffce740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55d9356297c0 con 0x55d9352483b0 2024-04-25T11:25:34.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.490+0000 7f90dffce740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55d93562a560 con 0x55d935250590 2024-04-25T11:25:34.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.490+0000 7f90dffce740 10 client.0 _ll_register_callbacks cb 0x7f90cc007ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:25:34.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.490+0000 7f90d359e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d935271270 0x55d935622040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.490+0000 7f90d359e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d935271270 0x55d935622040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55096/0 (socket says 172.21.15.69:55096) 2024-04-25T11:25:34.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.490+0000 7f90d359e640 1 -- 172.21.15.69:0/4092622906 learned_addr learned my addr 172.21.15.69:0/4092622906 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:34.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.490+0000 7f90d2d9d640 1 --2- 172.21.15.69:0/4092622906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d9352483b0 0x55d93561ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.490+0000 7f90ca59c640 1 --2- 172.21.15.69:0/4092622906 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d935250590 0x55d93561f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.491+0000 7f90d359e640 1 -- 172.21.15.69:0/4092622906 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d935250590 msgr2=0x55d93561f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:25:34.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.491+0000 7f90d359e640 1 --2- 172.21.15.69:0/4092622906 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d935250590 0x55d93561f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.491+0000 7f90d359e640 1 -- 172.21.15.69:0/4092622906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d9352483b0 msgr2=0x55d93561ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:25:34.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.491+0000 7f90d359e640 1 --2- 172.21.15.69:0/4092622906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d9352483b0 0x55d93561ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.491+0000 7f90d359e640 1 -- 172.21.15.69:0/4092622906 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d935622600 con 0x55d935271270 2024-04-25T11:25:34.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.491+0000 7f90d359e640 1 --2- 172.21.15.69:0/4092622906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d935271270 0x55d935622040 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f90cc021e10 tx=0x7f90cc021670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c76066c87679ac75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:34.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.491+0000 7f90ba7fc640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-25T11:25:34.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.491+0000 7f90ba7fc640 1 -- 172.21.15.69:0/4092622906 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90cc01c070 con 0x55d935271270 2024-04-25T11:25:34.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.491+0000 7f90ba7fc640 1 -- 172.21.15.69:0/4092622906 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f90cc0205b0 con 0x55d935271270 2024-04-25T11:25:34.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.491+0000 7f90dffce740 1 -- 172.21.15.69:0/4092622906 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d93557f1c0 con 0x55d935271270 2024-04-25T11:25:34.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.491+0000 7f90ba7fc640 1 -- 172.21.15.69:0/4092622906 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f90cc045050 con 0x55d935271270 2024-04-25T11:25:34.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.491+0000 7f90dffce740 10 client.15222 fetch_fsmap learned FSMap version 28 2024-04-25T11:25:34.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.491+0000 7f90dffce740 1 -- 172.21.15.69:0/4092622906 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d9356238f0 con 0x55d935271270 2024-04-25T11:25:34.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.491+0000 7f90ba7fc640 1 -- 172.21.15.69:0/4092622906 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55d93557f1c0 con 0x55d935271270 2024-04-25T11:25:34.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.491+0000 7f90ba7fc640 1 -- 172.21.15.69:0/4092622906 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f90cc03c890 con 0x55d935271270 2024-04-25T11:25:34.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.491+0000 7f90dffce740 10 client.15222 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T11:25:34.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.491+0000 7f90dffce740 10 client.15222 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T11:25:34.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.491+0000 7f90dffce740 20 client.15222 populate_metadata read hostname 'smithi069' 2024-04-25T11:25:34.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.492+0000 7f90dffce740 1 --2- 172.21.15.69:0/4092622906 >> [v2:172.21.15.69:6840/2060225181,v1:172.21.15.69:6841/2060225181] conn(0x55d935205280 0x55d935696470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.492+0000 7f90dffce740 4 client.15222 mds_command: new command op to 14625 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:25:34.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.492+0000 7f90dffce740 1 -- 172.21.15.69:0/4092622906 --> [v2:172.21.15.69:6840/2060225181,v1:172.21.15.69:6841/2060225181] -- command(tid 0: {"prefix": "status"}) -- 0x55d9354fa9a0 con 0x55d935205280 2024-04-25T11:25:34.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.492+0000 7f90d2d9d640 1 --2- 172.21.15.69:0/4092622906 >> [v2:172.21.15.69:6840/2060225181,v1:172.21.15.69:6841/2060225181] conn(0x55d935205280 0x55d935696470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.492+0000 7f90ba7fc640 10 client.15222 ms_handle_connect on v2:172.21.15.69:6840/2060225181 2024-04-25T11:25:34.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.492+0000 7f90d2d9d640 1 --2- 172.21.15.69:0/4092622906 >> [v2:172.21.15.69:6840/2060225181,v1:172.21.15.69:6841/2060225181] conn(0x55d935205280 0x55d935696470 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8ad9ed5c27770a90 server_cookie=4e04477cbdd2b61c in_seq=0 out_seq=0 2024-04-25T11:25:34.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.492+0000 7f90ba7fc640 1 -- 172.21.15.69:0/4092622906 <== mds.4 v2:172.21.15.69:6840/2060225181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55d9354fa9a0 con 0x55d935205280 2024-04-25T11:25:34.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.492+0000 7f90ba7fc640 10 client.15222 handle_command_reply: tid=0 2024-04-25T11:25:34.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.492+0000 7f90dffce740 1 client.15222 shutdown 2024-04-25T11:25:34.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.492+0000 7f90dffce740 1 -- 172.21.15.69:0/4092622906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d935271270 msgr2=0x55d935622040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:34.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.492+0000 7f90dffce740 1 --2- 172.21.15.69:0/4092622906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d935271270 0x55d935622040 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f90cc021e10 tx=0x7f90cc021670 comp rx=0 tx=0).stop 2024-04-25T11:25:34.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.492+0000 7f90dffce740 1 -- 172.21.15.69:0/4092622906 shutdown_connections 2024-04-25T11:25:34.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.492+0000 7f90dffce740 1 --2- 172.21.15.69:0/4092622906 >> [v2:172.21.15.69:6840/2060225181,v1:172.21.15.69:6841/2060225181] conn(0x55d935205280 0x55d935696470 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.492+0000 7f90dffce740 1 --2- 172.21.15.69:0/4092622906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d935271270 0x55d935622040 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.492+0000 7f90ba7fc640 0 client.15222 ms_handle_reset on v2:172.21.15.69:6840/2060225181 2024-04-25T11:25:34.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.492+0000 7f90dffce740 1 --2- 172.21.15.69:0/4092622906 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d935250590 0x55d93561f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.492+0000 7f90dffce740 1 --2- 172.21.15.69:0/4092622906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d9352483b0 0x55d93561ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.492+0000 7f90dffce740 1 -- 172.21.15.69:0/4092622906 >> 172.21.15.69:0/4092622906 conn(0x55d9352653c0 msgr2=0x55d93561a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:34.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.492+0000 7f90dffce740 1 -- 172.21.15.69:0/4092622906 shutdown_connections 2024-04-25T11:25:34.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.492+0000 7f90dffce740 1 -- 172.21.15.69:0/4092622906 wait complete. 2024-04-25T11:25:34.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.493+0000 7f90dffce740 20 client.15222 trim_cache size 0 max 16384 2024-04-25T11:25:34.493 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-25T11:25:34.493 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "f7ed73dc-02f4-11ef-bc93-c7b262605968", 2024-04-25T11:25:34.493 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 4, 2024-04-25T11:25:34.493 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 14625, 2024-04-25T11:25:34.493 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-04-25T11:25:34.494 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-04-25T11:25:34.494 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-04-25T11:25:34.494 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 86.437684254999994, 2024-04-25T11:25:34.494 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 28, 2024-04-25T11:25:34.494 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 84, 2024-04-25T11:25:34.494 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 80, 2024-04-25T11:25:34.494 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 122.70839503000001 2024-04-25T11:25:34.494 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-25T11:25:34.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.493+0000 7f90d9042640 1 -- 172.21.15.69:0/1214889620 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f90c00afaf0 msgr2=0x7f90c00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:34.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.493+0000 7f90d9042640 1 --2- 172.21.15.69:0/1214889620 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f90c00afaf0 0x7f90c00b2000 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x55d935613540 tx=0x7f90cc004b50 comp rx=0 tx=0).stop 2024-04-25T11:25:34.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.493+0000 7f90d9042640 1 -- 172.21.15.69:0/1214889620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90d4104d30 msgr2=0x7f90d419a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:34.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.493+0000 7f90d9042640 1 --2- 172.21.15.69:0/1214889620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90d4104d30 0x7f90d419a690 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f90bc009230 tx=0x7f90bc0042f0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.493+0000 7f90d9042640 1 -- 172.21.15.69:0/1214889620 shutdown_connections 2024-04-25T11:25:34.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.493+0000 7f90d9042640 1 --2- 172.21.15.69:0/1214889620 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f90c00afaf0 0x7f90c00b2000 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.493+0000 7f90d9042640 1 --2- 172.21.15.69:0/1214889620 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90d4109a30 0x7f90d41a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.493+0000 7f90d9042640 1 --2- 172.21.15.69:0/1214889620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90d4104d30 0x7f90d419a690 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.493+0000 7f90d9042640 1 --2- 172.21.15.69:0/1214889620 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90d4103af0 0x7f90d419a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.493+0000 7f90d9042640 1 -- 172.21.15.69:0/1214889620 >> 172.21.15.69:0/1214889620 conn(0x7f90d40fece0 msgr2=0x7f90d4105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:34.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.493+0000 7f90d9042640 1 -- 172.21.15.69:0/1214889620 shutdown_connections 2024-04-25T11:25:34.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.493+0000 7f90d9042640 1 -- 172.21.15.69:0/1214889620 wait complete. 2024-04-25T11:25:34.497 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-04-25T11:25:34.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.612+0000 7f470e508640 1 -- 172.21.15.69:0/3896357528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4708108930 msgr2=0x7f470810ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:34.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.612+0000 7f470e508640 1 --2- 172.21.15.69:0/3896357528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4708108930 0x7f470810ad50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4700009ca0 tx=0x7f470002a8c0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.613+0000 7f470e508640 1 -- 172.21.15.69:0/3896357528 shutdown_connections 2024-04-25T11:25:34.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.613+0000 7f470e508640 1 --2- 172.21.15.69:0/3896357528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4708108930 0x7f470810ad50 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.613+0000 7f470e508640 1 --2- 172.21.15.69:0/3896357528 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4708103c30 0x7f4708104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.613+0000 7f470e508640 1 --2- 172.21.15.69:0/3896357528 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47081029f0 0x7f47081036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.613+0000 7f470e508640 1 -- 172.21.15.69:0/3896357528 >> 172.21.15.69:0/3896357528 conn(0x7f47080fdbe0 msgr2=0x7f4708100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:34.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.613+0000 7f470e508640 1 -- 172.21.15.69:0/3896357528 shutdown_connections 2024-04-25T11:25:34.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.613+0000 7f470e508640 1 -- 172.21.15.69:0/3896357528 wait complete. 2024-04-25T11:25:34.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.613+0000 7f470e508640 1 Processor -- start 2024-04-25T11:25:34.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.613+0000 7f470e508640 1 -- start start 2024-04-25T11:25:34.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.614+0000 7f470e508640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47081029f0 0x7f470806bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.614+0000 7f470e508640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4708103c30 0x7f47081a1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.614+0000 7f470e508640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4708108930 0x7f47081a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.614+0000 7f470e508640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f470810e400 con 0x7f4708108930 2024-04-25T11:25:34.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.614+0000 7f470e508640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f470810e280 con 0x7f47081029f0 2024-04-25T11:25:34.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.614+0000 7f470e508640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f470810e580 con 0x7f4708103c30 2024-04-25T11:25:34.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.614+0000 7f470caa5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4708108930 0x7f47081a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.614+0000 7f470caa5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4708108930 0x7f47081a3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55116/0 (socket says 172.21.15.69:55116) 2024-04-25T11:25:34.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.614+0000 7f470caa5640 1 -- 172.21.15.69:0/2141907420 learned_addr learned my addr 172.21.15.69:0/2141907420 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:34.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.614+0000 7f4707fff640 1 --2- 172.21.15.69:0/2141907420 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47081029f0 0x7f470806bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.614+0000 7f47077fe640 1 --2- 172.21.15.69:0/2141907420 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4708103c30 0x7f47081a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.614+0000 7f470caa5640 1 -- 172.21.15.69:0/2141907420 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4708103c30 msgr2=0x7f47081a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:34.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.614+0000 7f470caa5640 1 --2- 172.21.15.69:0/2141907420 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4708103c30 0x7f47081a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.614+0000 7f470caa5640 1 -- 172.21.15.69:0/2141907420 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47081029f0 msgr2=0x7f470806bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:34.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.614+0000 7f470caa5640 1 --2- 172.21.15.69:0/2141907420 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47081029f0 0x7f470806bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.614+0000 7f470caa5640 1 -- 172.21.15.69:0/2141907420 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47081a3e20 con 0x7f4708108930 2024-04-25T11:25:34.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.614+0000 7f470caa5640 1 --2- 172.21.15.69:0/2141907420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4708108930 0x7f47081a3890 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f470002aeb0 tx=0x7f4700003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:34.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.614+0000 7f47057fa640 1 -- 172.21.15.69:0/2141907420 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4700032ed0 con 0x7f4708108930 2024-04-25T11:25:34.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.614+0000 7f47057fa640 1 -- 172.21.15.69:0/2141907420 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4700003e10 con 0x7f4708108930 2024-04-25T11:25:34.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.614+0000 7f470e508640 1 -- 172.21.15.69:0/2141907420 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47081a40b0 con 0x7f4708108930 2024-04-25T11:25:34.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.614+0000 7f47057fa640 1 -- 172.21.15.69:0/2141907420 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4700030be0 con 0x7f4708108930 2024-04-25T11:25:34.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.614+0000 7f470e508640 1 -- 172.21.15.69:0/2141907420 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47081a4600 con 0x7f4708108930 2024-04-25T11:25:34.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.617+0000 7f47153b7740 1 Processor -- start 2024-04-25T11:25:34.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.617+0000 7f47057fa640 1 -- 172.21.15.69:0/2141907420 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f4700004240 con 0x7f4708108930 2024-04-25T11:25:34.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.617+0000 7f47153b7740 1 -- start start 2024-04-25T11:25:34.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.617+0000 7f47153b7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d15022270 0x558d153c3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.617+0000 7f47153b7740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d14ff93b0 0x558d153c38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.617+0000 7f47153b7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558d15001590 0x558d153ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.617+0000 7f47153b7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558d15328740 con 0x558d15022270 2024-04-25T11:25:34.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.617+0000 7f47153b7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558d152c43a0 con 0x558d15001590 2024-04-25T11:25:34.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.617+0000 7f47153b7740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x558d15324700 con 0x558d14ff93b0 2024-04-25T11:25:34.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.617+0000 7f47077fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d15022270 0x558d153c3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.617+0000 7f47077fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d15022270 0x558d153c3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55122/0 (socket says 172.21.15.69:55122) 2024-04-25T11:25:34.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.617+0000 7f47077fe640 1 -- 172.21.15.69:0/1707350574 learned_addr learned my addr 172.21.15.69:0/1707350574 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:34.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.618+0000 7f47077fe640 1 -- 172.21.15.69:0/1707350574 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d14ff93b0 msgr2=0x558d153c38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:25:34.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.618+0000 7f4707fff640 1 --2- 172.21.15.69:0/1707350574 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d14ff93b0 0x558d153c38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.618+0000 7f47057fa640 1 --2- 172.21.15.69:0/2141907420 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f46f00afaf0 0x7f46f00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.618+0000 7f47077fe640 1 --2- 172.21.15.69:0/1707350574 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d14ff93b0 0x558d153c38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.618+0000 7f47077fe640 1 -- 172.21.15.69:0/1707350574 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558d15001590 msgr2=0x558d153ca7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:25:34.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.618+0000 7f47077fe640 1 --2- 172.21.15.69:0/1707350574 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558d15001590 0x558d153ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.618+0000 7f47077fe640 1 -- 172.21.15.69:0/1707350574 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d152e0750 con 0x558d15022270 2024-04-25T11:25:34.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.618+0000 7f470caa5640 1 --2- 172.21.15.69:0/2141907420 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f46f00afaf0 0x7f46f00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.618+0000 7f47077fe640 1 --2- 172.21.15.69:0/1707350574 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d15022270 0x558d153c3370 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f46fc00bda0 tx=0x7f46fc0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7049ec8c49a909e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:34.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.618+0000 7f47057fa640 1 -- 172.21.15.69:0/2141907420 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f4700100500 con 0x7f4708108930 2024-04-25T11:25:34.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.618+0000 7f46ee7fc640 1 -- 172.21.15.69:0/1707350574 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46fc004220 con 0x558d15022270 2024-04-25T11:25:34.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.618+0000 7f46ee7fc640 1 -- 172.21.15.69:0/1707350574 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46fc0043c0 con 0x558d15022270 2024-04-25T11:25:34.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.618+0000 7f46ee7fc640 1 -- 172.21.15.69:0/1707350574 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46fc005540 con 0x558d15022270 2024-04-25T11:25:34.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.619+0000 7f47153b7740 1 -- 172.21.15.69:0/1707350574 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d15022270 msgr2=0x558d153c3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:34.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.619+0000 7f470caa5640 1 --2- 172.21.15.69:0/2141907420 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f46f00afaf0 0x7f46f00b2000 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f47000c85a0 tx=0x7f4700097160 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:34.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.619+0000 7f47153b7740 1 --2- 172.21.15.69:0/1707350574 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d15022270 0x558d153c3370 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f46fc00bda0 tx=0x7f46fc0107a0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.619+0000 7f47153b7740 1 -- 172.21.15.69:0/1707350574 shutdown_connections 2024-04-25T11:25:34.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.619+0000 7f47153b7740 1 --2- 172.21.15.69:0/1707350574 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558d15001590 0x558d153ca7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.619+0000 7f47153b7740 1 --2- 172.21.15.69:0/1707350574 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d14ff93b0 0x558d153c38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.619+0000 7f47153b7740 1 --2- 172.21.15.69:0/1707350574 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d15022270 0x558d153c3370 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.619+0000 7f47153b7740 1 -- 172.21.15.69:0/1707350574 >> 172.21.15.69:0/1707350574 conn(0x558d150163c0 msgr2=0x558d153bed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:34.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.619+0000 7f47153b7740 1 -- 172.21.15.69:0/1707350574 shutdown_connections 2024-04-25T11:25:34.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.619+0000 7f47153b7740 1 -- 172.21.15.69:0/1707350574 wait complete. 2024-04-25T11:25:34.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.619+0000 7f47153b7740 1 -- start start 2024-04-25T11:25:34.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.620+0000 7f47153b7740 1 Processor -- start 2024-04-25T11:25:34.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.620+0000 7f47153b7740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d14ff93b0 0x558d153cd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.620+0000 7f47153b7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558d15001590 0x558d153cd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.620+0000 7f47153b7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d15022270 0x558d153d4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.620+0000 7f47153b7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558d153cb240 con 0x558d15022270 2024-04-25T11:25:34.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.620+0000 7f47153b7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558d152affe0 con 0x558d15001590 2024-04-25T11:25:34.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.620+0000 7f47153b7740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x558d153cb3c0 con 0x558d14ff93b0 2024-04-25T11:25:34.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.620+0000 7f47153b7740 10 client.0 _ll_register_callbacks cb 0x558d152c6150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:25:34.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.620+0000 7f47077fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d15022270 0x558d153d4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.620+0000 7f47077fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d15022270 0x558d153d4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55124/0 (socket says 172.21.15.69:55124) 2024-04-25T11:25:34.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.620+0000 7f47077fe640 1 -- 172.21.15.69:0/2792435178 learned_addr learned my addr 172.21.15.69:0/2792435178 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:34.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.620+0000 7f4707fff640 1 --2- 172.21.15.69:0/2792435178 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558d15001590 0x558d153cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.620+0000 7f47077fe640 1 --2- 172.21.15.69:0/2792435178 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d14ff93b0 0x558d153cd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.620+0000 7f47077fe640 1 -- 172.21.15.69:0/2792435178 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d14ff93b0 msgr2=0x558d153cd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:25:34.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.620+0000 7f47077fe640 1 --2- 172.21.15.69:0/2792435178 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d14ff93b0 0x558d153cd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.620+0000 7f47077fe640 1 -- 172.21.15.69:0/2792435178 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558d15001590 msgr2=0x558d153cd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:25:34.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.620+0000 7f47077fe640 1 --2- 172.21.15.69:0/2792435178 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558d15001590 0x558d153cd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.620+0000 7f47077fe640 1 -- 172.21.15.69:0/2792435178 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d1503a2d0 con 0x558d15022270 2024-04-25T11:25:34.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.620+0000 7f47077fe640 1 --2- 172.21.15.69:0/2792435178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d15022270 0x558d153d4760 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f46fc004610 tx=0x7f46fc00b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc5277db2335713 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:34.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.621+0000 7f46e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-25T11:25:34.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.621+0000 7f46e77fe640 1 -- 172.21.15.69:0/2792435178 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46fc0046c0 con 0x558d15022270 2024-04-25T11:25:34.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.621+0000 7f46e77fe640 1 -- 172.21.15.69:0/2792435178 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f46fc004860 con 0x558d15022270 2024-04-25T11:25:34.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.621+0000 7f47153b7740 1 -- 172.21.15.69:0/2792435178 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46fc0043c0 con 0x558d15022270 2024-04-25T11:25:34.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.621+0000 7f46e77fe640 1 -- 172.21.15.69:0/2792435178 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f46fc032050 con 0x558d15022270 2024-04-25T11:25:34.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.621+0000 7f47153b7740 10 client.15234 fetch_fsmap learned FSMap version 28 2024-04-25T11:25:34.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.621+0000 7f47153b7740 1 -- 172.21.15.69:0/2792435178 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558d151046b0 con 0x558d15022270 2024-04-25T11:25:34.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.621+0000 7f46e77fe640 1 -- 172.21.15.69:0/2792435178 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46fc0043c0 con 0x558d15022270 2024-04-25T11:25:34.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.621+0000 7f46e77fe640 1 -- 172.21.15.69:0/2792435178 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f46fc011440 con 0x558d15022270 2024-04-25T11:25:34.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.621+0000 7f47153b7740 10 client.15234 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T11:25:34.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.621+0000 7f47153b7740 10 client.15234 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-25T11:25:34.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.621+0000 7f47153b7740 20 client.15234 populate_metadata read hostname 'smithi069' 2024-04-25T11:25:34.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.621+0000 7f47153b7740 1 --2- 172.21.15.69:0/2792435178 >> [v2:172.21.15.146:6834/1876533699,v1:172.21.15.146:6835/1876533699] conn(0x558d14fe0e40 0x558d153dae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.621+0000 7f47153b7740 4 client.15234 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:25:34.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.621+0000 7f47153b7740 1 -- 172.21.15.69:0/2792435178 --> [v2:172.21.15.146:6834/1876533699,v1:172.21.15.146:6835/1876533699] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d15336350 con 0x558d14fe0e40 2024-04-25T11:25:34.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.622+0000 7f4707fff640 1 --2- 172.21.15.69:0/2792435178 >> [v2:172.21.15.146:6834/1876533699,v1:172.21.15.146:6835/1876533699] conn(0x558d14fe0e40 0x558d153dae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.622+0000 7f46e77fe640 10 client.15234 ms_handle_connect on v2:172.21.15.146:6834/1876533699 2024-04-25T11:25:34.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.622+0000 7f4707fff640 1 --2- 172.21.15.69:0/2792435178 >> [v2:172.21.15.146:6834/1876533699,v1:172.21.15.146:6835/1876533699] conn(0x558d14fe0e40 0x558d153dae10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d55798c37e819e56 server_cookie=fe965ae807ff17fd in_seq=0 out_seq=0 2024-04-25T11:25:34.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.623+0000 7f46e77fe640 1 -- 172.21.15.69:0/2792435178 <== mds.3 v2:172.21.15.146:6834/1876533699 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558d15336350 con 0x558d14fe0e40 2024-04-25T11:25:34.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.623+0000 7f46e77fe640 10 client.15234 handle_command_reply: tid=0 2024-04-25T11:25:34.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.623+0000 7f47153b7740 1 client.15234 shutdown 2024-04-25T11:25:34.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.623+0000 7f47153b7740 1 -- 172.21.15.69:0/2792435178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d15022270 msgr2=0x558d153d4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:34.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.623+0000 7f47153b7740 1 --2- 172.21.15.69:0/2792435178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d15022270 0x558d153d4760 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f46fc004610 tx=0x7f46fc00b410 comp rx=0 tx=0).stop 2024-04-25T11:25:34.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.623+0000 7f47153b7740 1 -- 172.21.15.69:0/2792435178 shutdown_connections 2024-04-25T11:25:34.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.623+0000 7f47153b7740 1 --2- 172.21.15.69:0/2792435178 >> [v2:172.21.15.146:6834/1876533699,v1:172.21.15.146:6835/1876533699] conn(0x558d14fe0e40 0x558d153dae10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.623+0000 7f47153b7740 1 --2- 172.21.15.69:0/2792435178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d15022270 0x558d153d4760 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.623+0000 7f47153b7740 1 --2- 172.21.15.69:0/2792435178 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558d15001590 0x558d153cd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.623+0000 7f46e77fe640 0 client.15234 ms_handle_reset on v2:172.21.15.146:6834/1876533699 2024-04-25T11:25:34.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.623+0000 7f47153b7740 1 --2- 172.21.15.69:0/2792435178 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d14ff93b0 0x558d153cd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.623+0000 7f47153b7740 1 -- 172.21.15.69:0/2792435178 >> 172.21.15.69:0/2792435178 conn(0x558d150163c0 msgr2=0x558d153bdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:34.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.623+0000 7f47153b7740 1 -- 172.21.15.69:0/2792435178 shutdown_connections 2024-04-25T11:25:34.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.623+0000 7f47153b7740 1 -- 172.21.15.69:0/2792435178 wait complete. 2024-04-25T11:25:34.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.623+0000 7f47153b7740 20 client.15234 trim_cache size 0 max 16384 2024-04-25T11:25:34.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.641+0000 7f47153b7740 1 Processor -- start 2024-04-25T11:25:34.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.641+0000 7f47153b7740 1 -- start start 2024-04-25T11:25:34.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.641+0000 7f47153b7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d14fe0e40 0x558d153c7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.641+0000 7f47153b7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558d14ff93b0 0x558d153c7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.642+0000 7f47153b7740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d15001590 0x558d153cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.642+0000 7f47153b7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558d153d9150 con 0x558d14fe0e40 2024-04-25T11:25:34.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.642+0000 7f47153b7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558d153d9820 con 0x558d14ff93b0 2024-04-25T11:25:34.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.642+0000 7f47153b7740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x558d153d9ef0 con 0x558d15001590 2024-04-25T11:25:34.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.642+0000 7f47077fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d14fe0e40 0x558d153c7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.642+0000 7f47077fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d14fe0e40 0x558d153c7710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55138/0 (socket says 172.21.15.69:55138) 2024-04-25T11:25:34.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.642+0000 7f47077fe640 1 -- 172.21.15.69:0/3871828089 learned_addr learned my addr 172.21.15.69:0/3871828089 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:34.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.642+0000 7f47077fe640 1 -- 172.21.15.69:0/3871828089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d15001590 msgr2=0x558d153cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:25:34.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.642+0000 7f47077fe640 1 --2- 172.21.15.69:0/3871828089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d15001590 0x558d153cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.642+0000 7f47077fe640 1 -- 172.21.15.69:0/3871828089 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558d14ff93b0 msgr2=0x558d153c7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:25:34.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.642+0000 7f47077fe640 1 --2- 172.21.15.69:0/3871828089 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558d14ff93b0 0x558d153c7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.642+0000 7f47077fe640 1 -- 172.21.15.69:0/3871828089 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d153c6790 con 0x558d14fe0e40 2024-04-25T11:25:34.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.642+0000 7f47077fe640 1 --2- 172.21.15.69:0/3871828089 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d14fe0e40 0x558d153c7710 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f46fc000c00 tx=0x7f46fc004130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c79712f900644432 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:34.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.642+0000 7f46edffb640 1 -- 172.21.15.69:0/3871828089 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46fc03c070 con 0x558d14fe0e40 2024-04-25T11:25:34.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.642+0000 7f46edffb640 1 -- 172.21.15.69:0/3871828089 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46fc010ca0 con 0x558d14fe0e40 2024-04-25T11:25:34.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.642+0000 7f46edffb640 1 -- 172.21.15.69:0/3871828089 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46fc0042b0 con 0x558d14fe0e40 2024-04-25T11:25:34.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.643+0000 7f47153b7740 1 -- 172.21.15.69:0/3871828089 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d14fe0e40 msgr2=0x558d153c7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:34.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.643+0000 7f47153b7740 1 --2- 172.21.15.69:0/3871828089 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d14fe0e40 0x558d153c7710 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f46fc000c00 tx=0x7f46fc004130 comp rx=0 tx=0).stop 2024-04-25T11:25:34.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.643+0000 7f47153b7740 1 -- 172.21.15.69:0/3871828089 shutdown_connections 2024-04-25T11:25:34.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.643+0000 7f47153b7740 1 --2- 172.21.15.69:0/3871828089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d15001590 0x558d153cb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.643+0000 7f47153b7740 1 --2- 172.21.15.69:0/3871828089 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558d14ff93b0 0x558d153c7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.643+0000 7f47153b7740 1 --2- 172.21.15.69:0/3871828089 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d14fe0e40 0x558d153c7710 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.643+0000 7f47153b7740 1 -- 172.21.15.69:0/3871828089 >> 172.21.15.69:0/3871828089 conn(0x558d150163c0 msgr2=0x558d153c69c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:34.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.643+0000 7f47153b7740 1 -- 172.21.15.69:0/3871828089 shutdown_connections 2024-04-25T11:25:34.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.643+0000 7f47153b7740 1 -- 172.21.15.69:0/3871828089 wait complete. 2024-04-25T11:25:34.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.643+0000 7f47153b7740 1 -- start start 2024-04-25T11:25:34.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.644+0000 7f47153b7740 1 Processor -- start 2024-04-25T11:25:34.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.644+0000 7f47153b7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d14fe0e40 0x558d153cf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.644+0000 7f47153b7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558d14ff93b0 0x558d153cf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.644+0000 7f47153b7740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d15001590 0x558d153d2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.644+0000 7f47153b7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558d153dac90 con 0x558d14fe0e40 2024-04-25T11:25:34.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.644+0000 7f47153b7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558d153da5c0 con 0x558d14ff93b0 2024-04-25T11:25:34.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.644+0000 7f47153b7740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x558d153db360 con 0x558d15001590 2024-04-25T11:25:34.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.644+0000 7f47153b7740 10 client.0 _ll_register_callbacks cb 0x7f46fc00bc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:25:34.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.644+0000 7f47077fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d14fe0e40 0x558d153cf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.644+0000 7f47077fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d14fe0e40 0x558d153cf400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55154/0 (socket says 172.21.15.69:55154) 2024-04-25T11:25:34.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.644+0000 7f47077fe640 1 -- 172.21.15.69:0/823318457 learned_addr learned my addr 172.21.15.69:0/823318457 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:34.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.644+0000 7f4707fff640 1 --2- 172.21.15.69:0/823318457 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558d14ff93b0 0x558d153cf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.644+0000 7f47077fe640 1 --2- 172.21.15.69:0/823318457 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d15001590 0x558d153d2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.644+0000 7f47077fe640 1 -- 172.21.15.69:0/823318457 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d15001590 msgr2=0x558d153d2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:25:34.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.644+0000 7f47077fe640 1 --2- 172.21.15.69:0/823318457 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d15001590 0x558d153d2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.644+0000 7f47077fe640 1 -- 172.21.15.69:0/823318457 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558d14ff93b0 msgr2=0x558d153cf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:25:34.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.644+0000 7f47077fe640 1 --2- 172.21.15.69:0/823318457 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558d14ff93b0 0x558d153cf940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.644+0000 7f47077fe640 1 -- 172.21.15.69:0/823318457 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d153d2e40 con 0x558d14fe0e40 2024-04-25T11:25:34.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.644+0000 7f4707fff640 1 --2- 172.21.15.69:0/823318457 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558d14ff93b0 0x558d153cf940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:25:34.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.644+0000 7f47077fe640 1 --2- 172.21.15.69:0/823318457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d14fe0e40 0x558d153cf400 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f46fc03a040 tx=0x7f46fc0040a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70446aa0f658f0f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:34.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.644+0000 7f46e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-25T11:25:34.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.644+0000 7f46e77fe640 1 -- 172.21.15.69:0/823318457 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46fc03c040 con 0x558d14fe0e40 2024-04-25T11:25:34.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.644+0000 7f46e77fe640 1 -- 172.21.15.69:0/823318457 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f46fc004860 con 0x558d14fe0e40 2024-04-25T11:25:34.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.644+0000 7f47153b7740 1 -- 172.21.15.69:0/823318457 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46fc010ca0 con 0x558d14fe0e40 2024-04-25T11:25:34.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.645+0000 7f46e77fe640 1 -- 172.21.15.69:0/823318457 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f46fc004220 con 0x558d14fe0e40 2024-04-25T11:25:34.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.645+0000 7f47153b7740 10 client.15240 fetch_fsmap learned FSMap version 28 2024-04-25T11:25:34.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.645+0000 7f47153b7740 1 -- 172.21.15.69:0/823318457 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558d153d40c0 con 0x558d14fe0e40 2024-04-25T11:25:34.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.645+0000 7f46e77fe640 1 -- 172.21.15.69:0/823318457 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46fc010ca0 con 0x558d14fe0e40 2024-04-25T11:25:34.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.645+0000 7f46e77fe640 1 -- 172.21.15.69:0/823318457 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f46fc042450 con 0x558d14fe0e40 2024-04-25T11:25:34.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.645+0000 7f47153b7740 10 client.15240 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T11:25:34.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.645+0000 7f47153b7740 10 client.15240 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-25T11:25:34.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.645+0000 7f47153b7740 20 client.15240 populate_metadata read hostname 'smithi069' 2024-04-25T11:25:34.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.645+0000 7f47153b7740 1 --2- 172.21.15.69:0/823318457 >> [v2:172.21.15.146:6834/1876533699,v1:172.21.15.146:6835/1876533699] conn(0x558d15022270 0x558d15446ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.645+0000 7f47153b7740 4 client.15240 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:25:34.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.645+0000 7f47153b7740 1 -- 172.21.15.69:0/823318457 --> [v2:172.21.15.146:6834/1876533699,v1:172.21.15.146:6835/1876533699] -- command(tid 0: {"prefix": "status"}) -- 0x558d153301c0 con 0x558d15022270 2024-04-25T11:25:34.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.645+0000 7f4707fff640 1 --2- 172.21.15.69:0/823318457 >> [v2:172.21.15.146:6834/1876533699,v1:172.21.15.146:6835/1876533699] conn(0x558d15022270 0x558d15446ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.646+0000 7f46e77fe640 10 client.15240 ms_handle_connect on v2:172.21.15.146:6834/1876533699 2024-04-25T11:25:34.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.646+0000 7f4707fff640 1 --2- 172.21.15.69:0/823318457 >> [v2:172.21.15.146:6834/1876533699,v1:172.21.15.146:6835/1876533699] conn(0x558d15022270 0x558d15446ef0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3a6e8dde768e9d0d server_cookie=130b12b12bc44f59 in_seq=0 out_seq=0 2024-04-25T11:25:34.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.646+0000 7f46e77fe640 1 -- 172.21.15.69:0/823318457 <== mds.3 v2:172.21.15.146:6834/1876533699 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x558d153301c0 con 0x558d15022270 2024-04-25T11:25:34.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.646+0000 7f46e77fe640 10 client.15240 handle_command_reply: tid=0 2024-04-25T11:25:34.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.646+0000 7f47153b7740 1 client.15240 shutdown 2024-04-25T11:25:34.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.646+0000 7f47153b7740 1 -- 172.21.15.69:0/823318457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d14fe0e40 msgr2=0x558d153cf400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:34.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.646+0000 7f47153b7740 1 --2- 172.21.15.69:0/823318457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d14fe0e40 0x558d153cf400 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f46fc03a040 tx=0x7f46fc0040a0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.646+0000 7f47153b7740 1 -- 172.21.15.69:0/823318457 shutdown_connections 2024-04-25T11:25:34.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.646+0000 7f47153b7740 1 --2- 172.21.15.69:0/823318457 >> [v2:172.21.15.146:6834/1876533699,v1:172.21.15.146:6835/1876533699] conn(0x558d15022270 0x558d15446ef0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.646+0000 7f47153b7740 1 --2- 172.21.15.69:0/823318457 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d15001590 0x558d153d2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.646+0000 7f46e77fe640 0 client.15240 ms_handle_reset on v2:172.21.15.146:6834/1876533699 2024-04-25T11:25:34.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.646+0000 7f47153b7740 1 --2- 172.21.15.69:0/823318457 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558d14ff93b0 0x558d153cf940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.646+0000 7f47153b7740 1 --2- 172.21.15.69:0/823318457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d14fe0e40 0x558d153cf400 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.646+0000 7f47153b7740 1 -- 172.21.15.69:0/823318457 >> 172.21.15.69:0/823318457 conn(0x558d150163c0 msgr2=0x558d153c7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:34.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.646+0000 7f47153b7740 1 -- 172.21.15.69:0/823318457 shutdown_connections 2024-04-25T11:25:34.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.646+0000 7f47153b7740 1 -- 172.21.15.69:0/823318457 wait complete. 2024-04-25T11:25:34.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.646+0000 7f47153b7740 20 client.15240 trim_cache size 0 max 16384 2024-04-25T11:25:34.646 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-25T11:25:34.646 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "f7ed73dc-02f4-11ef-bc93-c7b262605968", 2024-04-25T11:25:34.646 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 3, 2024-04-25T11:25:34.646 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 24458, 2024-04-25T11:25:34.646 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-04-25T11:25:34.646 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-04-25T11:25:34.646 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-04-25T11:25:34.647 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 86.741373830000001, 2024-04-25T11:25:34.647 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 28, 2024-04-25T11:25:34.647 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 84, 2024-04-25T11:25:34.647 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 80, 2024-04-25T11:25:34.647 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 140.33875278299999 2024-04-25T11:25:34.647 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-25T11:25:34.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.647+0000 7f470e508640 1 -- 172.21.15.69:0/2141907420 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f46f00afaf0 msgr2=0x7f46f00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:34.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.647+0000 7f470e508640 1 --2- 172.21.15.69:0/2141907420 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f46f00afaf0 0x7f46f00b2000 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f47000c85a0 tx=0x7f4700097160 comp rx=0 tx=0).stop 2024-04-25T11:25:34.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.647+0000 7f470e508640 1 -- 172.21.15.69:0/2141907420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4708108930 msgr2=0x7f47081a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:34.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.647+0000 7f470e508640 1 --2- 172.21.15.69:0/2141907420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4708108930 0x7f47081a3890 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f470002aeb0 tx=0x7f4700003c20 comp rx=0 tx=0).stop 2024-04-25T11:25:34.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.647+0000 7f470e508640 1 -- 172.21.15.69:0/2141907420 shutdown_connections 2024-04-25T11:25:34.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.647+0000 7f470e508640 1 --2- 172.21.15.69:0/2141907420 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f46f00afaf0 0x7f46f00b2000 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.647+0000 7f470e508640 1 --2- 172.21.15.69:0/2141907420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4708108930 0x7f47081a3890 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.647+0000 7f470e508640 1 --2- 172.21.15.69:0/2141907420 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4708103c30 0x7f47081a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.647+0000 7f470e508640 1 --2- 172.21.15.69:0/2141907420 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47081029f0 0x7f470806bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.647+0000 7f470e508640 1 -- 172.21.15.69:0/2141907420 >> 172.21.15.69:0/2141907420 conn(0x7f47080fdbe0 msgr2=0x7f4708104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:34.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.647+0000 7f470e508640 1 -- 172.21.15.69:0/2141907420 shutdown_connections 2024-04-25T11:25:34.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.647+0000 7f470e508640 1 -- 172.21.15.69:0/2141907420 wait complete. 2024-04-25T11:25:34.651 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-25T11:25:34.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.766+0000 7f1174b9d640 1 -- 172.21.15.69:0/2926497844 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1170106f30 msgr2=0x7f1170109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:34.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.766+0000 7f1174b9d640 1 --2- 172.21.15.69:0/2926497844 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1170106f30 0x7f1170109350 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f1164009900 tx=0x7f116402a410 comp rx=0 tx=0).stop 2024-04-25T11:25:34.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.766+0000 7f1174b9d640 1 -- 172.21.15.69:0/2926497844 shutdown_connections 2024-04-25T11:25:34.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.766+0000 7f1174b9d640 1 --2- 172.21.15.69:0/2926497844 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1170109890 0x7f117010bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.766+0000 7f1174b9d640 1 --2- 172.21.15.69:0/2926497844 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1170106f30 0x7f1170109350 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.766+0000 7f1174b9d640 1 --2- 172.21.15.69:0/2926497844 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11701045d0 0x7f11701069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.766+0000 7f1174b9d640 1 -- 172.21.15.69:0/2926497844 >> 172.21.15.69:0/2926497844 conn(0x7f11700fdbd0 msgr2=0x7f1170100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:34.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.766+0000 7f1174b9d640 1 -- 172.21.15.69:0/2926497844 shutdown_connections 2024-04-25T11:25:34.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.766+0000 7f1174b9d640 1 -- 172.21.15.69:0/2926497844 wait complete. 2024-04-25T11:25:34.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.767+0000 7f1174b9d640 1 Processor -- start 2024-04-25T11:25:34.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.767+0000 7f1174b9d640 1 -- start start 2024-04-25T11:25:34.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.767+0000 7f1174b9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11701045d0 0x7f117019a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.767+0000 7f1174b9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1170106f30 0x7f117019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.767+0000 7f1174b9d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1170109890 0x7f11701a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.767+0000 7f1174b9d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f117010f4c0 con 0x7f1170106f30 2024-04-25T11:25:34.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.767+0000 7f1174b9d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f117010f340 con 0x7f1170109890 2024-04-25T11:25:34.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.767+0000 7f1174b9d640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f117010f640 con 0x7f11701045d0 2024-04-25T11:25:34.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.768+0000 7f116dd9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1170106f30 0x7f117019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.768+0000 7f116dd9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1170106f30 0x7f117019a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55176/0 (socket says 172.21.15.69:55176) 2024-04-25T11:25:34.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.768+0000 7f116dd9b640 1 -- 172.21.15.69:0/2258882474 learned_addr learned my addr 172.21.15.69:0/2258882474 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:34.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.768+0000 7f116ed9d640 1 --2- 172.21.15.69:0/2258882474 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1170109890 0x7f11701a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.768+0000 7f116e59c640 1 --2- 172.21.15.69:0/2258882474 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11701045d0 0x7f117019a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.768+0000 7f116dd9b640 1 -- 172.21.15.69:0/2258882474 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11701045d0 msgr2=0x7f117019a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:34.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.768+0000 7f116dd9b640 1 --2- 172.21.15.69:0/2258882474 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11701045d0 0x7f117019a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.768+0000 7f116dd9b640 1 -- 172.21.15.69:0/2258882474 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1170109890 msgr2=0x7f11701a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:34.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.768+0000 7f116dd9b640 1 --2- 172.21.15.69:0/2258882474 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1170109890 0x7f11701a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.768+0000 7f116dd9b640 1 -- 172.21.15.69:0/2258882474 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11701a1d50 con 0x7f1170106f30 2024-04-25T11:25:34.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.768+0000 7f116dd9b640 1 --2- 172.21.15.69:0/2258882474 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1170106f30 0x7f117019a750 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f1164009230 tx=0x7f11640042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:34.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.768+0000 7f115f7fe640 1 -- 172.21.15.69:0/2258882474 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116403f030 con 0x7f1170106f30 2024-04-25T11:25:34.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.768+0000 7f115f7fe640 1 -- 172.21.15.69:0/2258882474 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1164003c90 con 0x7f1170106f30 2024-04-25T11:25:34.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.768+0000 7f1174b9d640 1 -- 172.21.15.69:0/2258882474 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11701a1fe0 con 0x7f1170106f30 2024-04-25T11:25:34.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.768+0000 7f115f7fe640 1 -- 172.21.15.69:0/2258882474 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164035070 con 0x7f1170106f30 2024-04-25T11:25:34.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.768+0000 7f1174b9d640 1 -- 172.21.15.69:0/2258882474 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11701a23f0 con 0x7f1170106f30 2024-04-25T11:25:34.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.771+0000 7f115f7fe640 1 -- 172.21.15.69:0/2258882474 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f1164042030 con 0x7f1170106f30 2024-04-25T11:25:34.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.771+0000 7f117ba4e740 1 Processor -- start 2024-04-25T11:25:34.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.771+0000 7f117ba4e740 1 -- start start 2024-04-25T11:25:34.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.771+0000 7f117ba4e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3b8137270 0x55a3b84d8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.771+0000 7f117ba4e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3b810e3b0 0x55a3b84d88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.772+0000 7f117ba4e740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a3b8116590 0x55a3b84df7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.772+0000 7f117ba4e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a3b843d740 con 0x55a3b8137270 2024-04-25T11:25:34.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.772+0000 7f117ba4e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55a3b83d93a0 con 0x55a3b810e3b0 2024-04-25T11:25:34.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.772+0000 7f117ba4e740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55a3b8439700 con 0x55a3b8116590 2024-04-25T11:25:34.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.772+0000 7f116ed9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3b8137270 0x55a3b84d8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.772+0000 7f116ed9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3b8137270 0x55a3b84d8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55180/0 (socket says 172.21.15.69:55180) 2024-04-25T11:25:34.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.772+0000 7f116ed9d640 1 -- 172.21.15.69:0/597248017 learned_addr learned my addr 172.21.15.69:0/597248017 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:34.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.772+0000 7f115f7fe640 1 --2- 172.21.15.69:0/2258882474 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f11580afaf0 0x7f11580b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.772+0000 7f116dd9b640 1 --2- 172.21.15.69:0/597248017 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a3b8116590 0x55a3b84df7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.772+0000 7f116e59c640 1 --2- 172.21.15.69:0/597248017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3b810e3b0 0x55a3b84d88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.772+0000 7f116ed9d640 1 --2- 172.21.15.69:0/2258882474 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f11580afaf0 0x7f11580b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.772+0000 7f115f7fe640 1 -- 172.21.15.69:0/2258882474 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f11640ff630 con 0x7f1170106f30 2024-04-25T11:25:34.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.772+0000 7f116ed9d640 1 -- 172.21.15.69:0/597248017 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a3b8116590 msgr2=0x55a3b84df7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:25:34.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.772+0000 7f116ed9d640 1 --2- 172.21.15.69:0/597248017 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a3b8116590 0x55a3b84df7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.772+0000 7f116ed9d640 1 -- 172.21.15.69:0/597248017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3b810e3b0 msgr2=0x55a3b84d88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:25:34.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.772+0000 7f116ed9d640 1 --2- 172.21.15.69:0/597248017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3b810e3b0 0x55a3b84d88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.772+0000 7f116ed9d640 1 -- 172.21.15.69:0/597248017 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3b83f5750 con 0x55a3b8137270 2024-04-25T11:25:34.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.772+0000 7f116ed9d640 1 --2- 172.21.15.69:0/597248017 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3b8137270 0x55a3b84d8370 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1168014b30 tx=0x7f1168012390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20309c6c2e191f89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:34.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.772+0000 7f115cff9640 1 -- 172.21.15.69:0/597248017 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116801ed90 con 0x55a3b8137270 2024-04-25T11:25:34.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.772+0000 7f115cff9640 1 -- 172.21.15.69:0/597248017 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1168023070 con 0x55a3b8137270 2024-04-25T11:25:34.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.772+0000 7f115cff9640 1 -- 172.21.15.69:0/597248017 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116800fb50 con 0x55a3b8137270 2024-04-25T11:25:34.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.773+0000 7f117ba4e740 1 -- 172.21.15.69:0/597248017 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3b8137270 msgr2=0x55a3b84d8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:34.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.773+0000 7f117ba4e740 1 --2- 172.21.15.69:0/597248017 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3b8137270 0x55a3b84d8370 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1168014b30 tx=0x7f1168012390 comp rx=0 tx=0).stop 2024-04-25T11:25:34.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.773+0000 7f116ed9d640 1 --2- 172.21.15.69:0/2258882474 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f11580afaf0 0x7f11580b2000 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1168022000 tx=0x7f1168020000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:34.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.773+0000 7f117ba4e740 1 -- 172.21.15.69:0/597248017 shutdown_connections 2024-04-25T11:25:34.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.773+0000 7f117ba4e740 1 --2- 172.21.15.69:0/597248017 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a3b8116590 0x55a3b84df7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.773+0000 7f117ba4e740 1 --2- 172.21.15.69:0/597248017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3b810e3b0 0x55a3b84d88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.773+0000 7f117ba4e740 1 --2- 172.21.15.69:0/597248017 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3b8137270 0x55a3b84d8370 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.773+0000 7f117ba4e740 1 -- 172.21.15.69:0/597248017 >> 172.21.15.69:0/597248017 conn(0x55a3b812b3c0 msgr2=0x55a3b84d3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:34.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.773+0000 7f117ba4e740 1 -- 172.21.15.69:0/597248017 shutdown_connections 2024-04-25T11:25:34.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.773+0000 7f117ba4e740 1 -- 172.21.15.69:0/597248017 wait complete. 2024-04-25T11:25:34.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.774+0000 7f117ba4e740 1 -- start start 2024-04-25T11:25:34.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.774+0000 7f117ba4e740 1 Processor -- start 2024-04-25T11:25:34.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.774+0000 7f117ba4e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3b810e3b0 0x55a3b84e22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.774+0000 7f117ba4e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3b8116590 0x55a3b84e27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.774+0000 7f117ba4e740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a3b8137270 0x55a3b84e9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.774+0000 7f117ba4e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a3b84e0240 con 0x55a3b810e3b0 2024-04-25T11:25:34.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.774+0000 7f117ba4e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55a3b83c4fe0 con 0x55a3b8116590 2024-04-25T11:25:34.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.774+0000 7f117ba4e740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55a3b84e03c0 con 0x55a3b8137270 2024-04-25T11:25:34.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.774+0000 7f116e59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3b810e3b0 0x55a3b84e22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.774+0000 7f117ba4e740 10 client.0 _ll_register_callbacks cb 0x55a3b83db150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:25:34.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.774+0000 7f116e59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3b810e3b0 0x55a3b84e22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55194/0 (socket says 172.21.15.69:55194) 2024-04-25T11:25:34.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.774+0000 7f116e59c640 1 -- 172.21.15.69:0/3301977459 learned_addr learned my addr 172.21.15.69:0/3301977459 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:34.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.774+0000 7f116dd9b640 1 --2- 172.21.15.69:0/3301977459 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3b8116590 0x55a3b84e27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.774+0000 7f116e59c640 1 -- 172.21.15.69:0/3301977459 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a3b8137270 msgr2=0x55a3b84e9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:25:34.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.774+0000 7f116e59c640 1 --2- 172.21.15.69:0/3301977459 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a3b8137270 0x55a3b84e9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.774+0000 7f116e59c640 1 -- 172.21.15.69:0/3301977459 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3b8116590 msgr2=0x55a3b84e27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:25:34.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.774+0000 7f116e59c640 1 --2- 172.21.15.69:0/3301977459 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3b8116590 0x55a3b84e27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.774+0000 7f116e59c640 1 -- 172.21.15.69:0/3301977459 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3b814f2d0 con 0x55a3b810e3b0 2024-04-25T11:25:34.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.775+0000 7f116e59c640 1 --2- 172.21.15.69:0/3301977459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3b810e3b0 0x55a3b84e22a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f116000b920 tx=0x7f116000bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17c6d6f59f7a1d81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:34.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.775+0000 7f1155ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-25T11:25:34.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.775+0000 7f1155ffb640 1 -- 172.21.15.69:0/3301977459 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1160003ea0 con 0x55a3b810e3b0 2024-04-25T11:25:34.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.775+0000 7f1155ffb640 1 -- 172.21.15.69:0/3301977459 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f116001b640 con 0x55a3b810e3b0 2024-04-25T11:25:34.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.775+0000 7f117ba4e740 1 -- 172.21.15.69:0/3301977459 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1168023070 con 0x55a3b810e3b0 2024-04-25T11:25:34.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.775+0000 7f1155ffb640 1 -- 172.21.15.69:0/3301977459 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1160024050 con 0x55a3b810e3b0 2024-04-25T11:25:34.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.775+0000 7f117ba4e740 10 client.15252 fetch_fsmap learned FSMap version 28 2024-04-25T11:25:34.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.775+0000 7f117ba4e740 1 -- 172.21.15.69:0/3301977459 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a3b82196b0 con 0x55a3b810e3b0 2024-04-25T11:25:34.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.775+0000 7f1155ffb640 1 -- 172.21.15.69:0/3301977459 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1168023070 con 0x55a3b810e3b0 2024-04-25T11:25:34.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.775+0000 7f1155ffb640 1 -- 172.21.15.69:0/3301977459 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1160019020 con 0x55a3b810e3b0 2024-04-25T11:25:34.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.775+0000 7f117ba4e740 10 client.15252 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T11:25:34.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.775+0000 7f117ba4e740 10 client.15252 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T11:25:34.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.775+0000 7f117ba4e740 20 client.15252 populate_metadata read hostname 'smithi069' 2024-04-25T11:25:34.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.775+0000 7f117ba4e740 1 --2- 172.21.15.69:0/3301977459 >> [v2:172.21.15.152:6838/3008615657,v1:172.21.15.152:6839/3008615657] conn(0x55a3b80f5e40 0x55a3b84f0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.775+0000 7f117ba4e740 4 client.15252 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:25:34.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.775+0000 7f117ba4e740 1 -- 172.21.15.69:0/3301977459 --> [v2:172.21.15.152:6838/3008615657,v1:172.21.15.152:6839/3008615657] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3b844b350 con 0x55a3b80f5e40 2024-04-25T11:25:34.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.776+0000 7f116dd9b640 1 --2- 172.21.15.69:0/3301977459 >> [v2:172.21.15.152:6838/3008615657,v1:172.21.15.152:6839/3008615657] conn(0x55a3b80f5e40 0x55a3b84f0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.776+0000 7f1155ffb640 10 client.15252 ms_handle_connect on v2:172.21.15.152:6838/3008615657 2024-04-25T11:25:34.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.776+0000 7f116dd9b640 1 --2- 172.21.15.69:0/3301977459 >> [v2:172.21.15.152:6838/3008615657,v1:172.21.15.152:6839/3008615657] conn(0x55a3b80f5e40 0x55a3b84f0090 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=406ab06fb7215767 server_cookie=773d74466a45726f in_seq=0 out_seq=0 2024-04-25T11:25:34.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.777+0000 7f1155ffb640 1 -- 172.21.15.69:0/3301977459 <== mds.2 v2:172.21.15.152:6838/3008615657 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a3b844b350 con 0x55a3b80f5e40 2024-04-25T11:25:34.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.777+0000 7f1155ffb640 10 client.15252 handle_command_reply: tid=0 2024-04-25T11:25:34.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.777+0000 7f117ba4e740 1 client.15252 shutdown 2024-04-25T11:25:34.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.777+0000 7f117ba4e740 1 -- 172.21.15.69:0/3301977459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3b810e3b0 msgr2=0x55a3b84e22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:34.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.777+0000 7f117ba4e740 1 --2- 172.21.15.69:0/3301977459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3b810e3b0 0x55a3b84e22a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f116000b920 tx=0x7f116000bde0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.777+0000 7f117ba4e740 1 -- 172.21.15.69:0/3301977459 shutdown_connections 2024-04-25T11:25:34.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.777+0000 7f117ba4e740 1 --2- 172.21.15.69:0/3301977459 >> [v2:172.21.15.152:6838/3008615657,v1:172.21.15.152:6839/3008615657] conn(0x55a3b80f5e40 0x55a3b84f0090 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.777+0000 7f117ba4e740 1 --2- 172.21.15.69:0/3301977459 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a3b8137270 0x55a3b84e9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.777+0000 7f117ba4e740 1 --2- 172.21.15.69:0/3301977459 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3b8116590 0x55a3b84e27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.777+0000 7f117ba4e740 1 --2- 172.21.15.69:0/3301977459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3b810e3b0 0x55a3b84e22a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.777+0000 7f117ba4e740 1 -- 172.21.15.69:0/3301977459 >> 172.21.15.69:0/3301977459 conn(0x55a3b812b3c0 msgr2=0x55a3b84d2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:34.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.777+0000 7f1155ffb640 0 client.15252 ms_handle_reset on v2:172.21.15.152:6838/3008615657 2024-04-25T11:25:34.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.777+0000 7f117ba4e740 1 -- 172.21.15.69:0/3301977459 shutdown_connections 2024-04-25T11:25:34.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.777+0000 7f117ba4e740 1 -- 172.21.15.69:0/3301977459 wait complete. 2024-04-25T11:25:34.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.777+0000 7f117ba4e740 20 client.15252 trim_cache size 0 max 16384 2024-04-25T11:25:34.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.795+0000 7f117ba4e740 1 Processor -- start 2024-04-25T11:25:34.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.795+0000 7f117ba4e740 1 -- start start 2024-04-25T11:25:34.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.795+0000 7f117ba4e740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a3b810e3b0 0x55a3b84d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.795+0000 7f117ba4e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3b8116590 0x55a3b84e9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.796+0000 7f117ba4e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3b8137270 0x55a3b84d3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.796+0000 7f117ba4e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a3b84ee3d0 con 0x55a3b8137270 2024-04-25T11:25:34.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.796+0000 7f117ba4e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55a3b84eeaa0 con 0x55a3b8116590 2024-04-25T11:25:34.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.796+0000 7f117ba4e740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55a3b84ef170 con 0x55a3b810e3b0 2024-04-25T11:25:34.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.796+0000 7f116ed9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3b8137270 0x55a3b84d3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.796+0000 7f116ed9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3b8137270 0x55a3b84d3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55206/0 (socket says 172.21.15.69:55206) 2024-04-25T11:25:34.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.796+0000 7f116ed9d640 1 -- 172.21.15.69:0/1596846381 learned_addr learned my addr 172.21.15.69:0/1596846381 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:34.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.796+0000 7f116dd9b640 1 --2- 172.21.15.69:0/1596846381 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3b8116590 0x55a3b84e9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.796+0000 7f116e59c640 1 --2- 172.21.15.69:0/1596846381 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a3b810e3b0 0x55a3b84d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.796+0000 7f116ed9d640 1 -- 172.21.15.69:0/1596846381 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a3b810e3b0 msgr2=0x55a3b84d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:25:34.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.796+0000 7f116ed9d640 1 --2- 172.21.15.69:0/1596846381 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a3b810e3b0 0x55a3b84d36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.796+0000 7f116ed9d640 1 -- 172.21.15.69:0/1596846381 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3b8116590 msgr2=0x55a3b84e9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:25:34.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.796+0000 7f116ed9d640 1 --2- 172.21.15.69:0/1596846381 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3b8116590 0x55a3b84e9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.796+0000 7f116ed9d640 1 -- 172.21.15.69:0/1596846381 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3b84dd0f0 con 0x55a3b8137270 2024-04-25T11:25:34.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.796+0000 7f116ed9d640 1 --2- 172.21.15.69:0/1596846381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3b8137270 0x55a3b84d3c30 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f11680149d0 tx=0x7f1168005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c962a957b4b394 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:34.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.796+0000 7f1157fff640 1 -- 172.21.15.69:0/1596846381 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116800fe20 con 0x55a3b8137270 2024-04-25T11:25:34.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.796+0000 7f1157fff640 1 -- 172.21.15.69:0/1596846381 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1168035070 con 0x55a3b8137270 2024-04-25T11:25:34.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.796+0000 7f1157fff640 1 -- 172.21.15.69:0/1596846381 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116800f8d0 con 0x55a3b8137270 2024-04-25T11:25:34.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.797+0000 7f117ba4e740 1 -- 172.21.15.69:0/1596846381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3b8137270 msgr2=0x55a3b84d3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:34.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.797+0000 7f117ba4e740 1 --2- 172.21.15.69:0/1596846381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3b8137270 0x55a3b84d3c30 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f11680149d0 tx=0x7f1168005c00 comp rx=0 tx=0).stop 2024-04-25T11:25:34.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.797+0000 7f117ba4e740 1 -- 172.21.15.69:0/1596846381 shutdown_connections 2024-04-25T11:25:34.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.797+0000 7f117ba4e740 1 --2- 172.21.15.69:0/1596846381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3b8137270 0x55a3b84d3c30 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.797+0000 7f117ba4e740 1 --2- 172.21.15.69:0/1596846381 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3b8116590 0x55a3b84e9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.797+0000 7f117ba4e740 1 --2- 172.21.15.69:0/1596846381 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a3b810e3b0 0x55a3b84d36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.797+0000 7f117ba4e740 1 -- 172.21.15.69:0/1596846381 >> 172.21.15.69:0/1596846381 conn(0x55a3b812b3c0 msgr2=0x55a3b84e08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:34.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.797+0000 7f117ba4e740 1 -- 172.21.15.69:0/1596846381 shutdown_connections 2024-04-25T11:25:34.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.797+0000 7f117ba4e740 1 -- 172.21.15.69:0/1596846381 wait complete. 2024-04-25T11:25:34.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.797+0000 7f117ba4e740 1 -- start start 2024-04-25T11:25:34.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.797+0000 7f117ba4e740 1 Processor -- start 2024-04-25T11:25:34.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.798+0000 7f117ba4e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3b810e3b0 0x55a3b84e4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.798+0000 7f117ba4e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3b8116590 0x55a3b84e4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.798+0000 7f117ba4e740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a3b8137270 0x55a3b84e7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.798+0000 7f117ba4e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a3b84eff10 con 0x55a3b8116590 2024-04-25T11:25:34.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.798+0000 7f117ba4e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55a3b84ef840 con 0x55a3b810e3b0 2024-04-25T11:25:34.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.798+0000 7f117ba4e740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55a3b84f05e0 con 0x55a3b8137270 2024-04-25T11:25:34.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.798+0000 7f117ba4e740 10 client.0 _ll_register_callbacks cb 0x7f1160004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:25:34.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.798+0000 7f116dd9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3b8116590 0x55a3b84e4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.798+0000 7f116dd9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3b8116590 0x55a3b84e4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55214/0 (socket says 172.21.15.69:55214) 2024-04-25T11:25:34.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.798+0000 7f116dd9b640 1 -- 172.21.15.69:0/2028645821 learned_addr learned my addr 172.21.15.69:0/2028645821 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:34.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.798+0000 7f116e59c640 1 --2- 172.21.15.69:0/2028645821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3b810e3b0 0x55a3b84e4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.798+0000 7f116ed9d640 1 --2- 172.21.15.69:0/2028645821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a3b8137270 0x55a3b84e7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.798+0000 7f116dd9b640 1 -- 172.21.15.69:0/2028645821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a3b8137270 msgr2=0x55a3b84e7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:25:34.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.798+0000 7f116dd9b640 1 --2- 172.21.15.69:0/2028645821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a3b8137270 0x55a3b84e7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.798+0000 7f116dd9b640 1 -- 172.21.15.69:0/2028645821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3b810e3b0 msgr2=0x55a3b84e4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:25:34.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.798+0000 7f116dd9b640 1 --2- 172.21.15.69:0/2028645821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3b810e3b0 0x55a3b84e4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.798+0000 7f116dd9b640 1 -- 172.21.15.69:0/2028645821 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3b84e8380 con 0x55a3b8116590 2024-04-25T11:25:34.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.798+0000 7f116ed9d640 1 --2- 172.21.15.69:0/2028645821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a3b8137270 0x55a3b84e7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:25:34.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.798+0000 7f116dd9b640 1 --2- 172.21.15.69:0/2028645821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3b8116590 0x55a3b84e4e80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f11640fcd60 tx=0x7f11640930c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85e495f2acd1cb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:34.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.798+0000 7f1155ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-25T11:25:34.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.798+0000 7f1155ffb640 1 -- 172.21.15.69:0/2028645821 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164049040 con 0x55a3b8116590 2024-04-25T11:25:34.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.798+0000 7f1155ffb640 1 -- 172.21.15.69:0/2028645821 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f1164003800 con 0x55a3b8116590 2024-04-25T11:25:34.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.798+0000 7f117ba4e740 1 -- 172.21.15.69:0/2028645821 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1168035070 con 0x55a3b8116590 2024-04-25T11:25:34.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.799+0000 7f1155ffb640 1 -- 172.21.15.69:0/2028645821 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1164003e40 con 0x55a3b8116590 2024-04-25T11:25:34.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.799+0000 7f117ba4e740 10 client.15258 fetch_fsmap learned FSMap version 28 2024-04-25T11:25:34.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.799+0000 7f117ba4e740 1 -- 172.21.15.69:0/2028645821 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a3b84e9600 con 0x55a3b8116590 2024-04-25T11:25:34.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.799+0000 7f1155ffb640 1 -- 172.21.15.69:0/2028645821 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1168035070 con 0x55a3b8116590 2024-04-25T11:25:34.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.799+0000 7f1155ffb640 1 -- 172.21.15.69:0/2028645821 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f11640c3b40 con 0x55a3b8116590 2024-04-25T11:25:34.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.799+0000 7f117ba4e740 10 client.15258 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T11:25:34.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.799+0000 7f117ba4e740 10 client.15258 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T11:25:34.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.799+0000 7f117ba4e740 20 client.15258 populate_metadata read hostname 'smithi069' 2024-04-25T11:25:34.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.799+0000 7f117ba4e740 1 --2- 172.21.15.69:0/2028645821 >> [v2:172.21.15.152:6838/3008615657,v1:172.21.15.152:6839/3008615657] conn(0x55a3b80cb280 0x55a3b855c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.799+0000 7f117ba4e740 4 client.15258 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:25:34.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.799+0000 7f117ba4e740 1 -- 172.21.15.69:0/2028645821 --> [v2:172.21.15.152:6838/3008615657,v1:172.21.15.152:6839/3008615657] -- command(tid 0: {"prefix": "status"}) -- 0x55a3b84451c0 con 0x55a3b80cb280 2024-04-25T11:25:34.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.799+0000 7f116e59c640 1 --2- 172.21.15.69:0/2028645821 >> [v2:172.21.15.152:6838/3008615657,v1:172.21.15.152:6839/3008615657] conn(0x55a3b80cb280 0x55a3b855c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.800+0000 7f1155ffb640 10 client.15258 ms_handle_connect on v2:172.21.15.152:6838/3008615657 2024-04-25T11:25:34.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.800+0000 7f116e59c640 1 --2- 172.21.15.69:0/2028645821 >> [v2:172.21.15.152:6838/3008615657,v1:172.21.15.152:6839/3008615657] conn(0x55a3b80cb280 0x55a3b855c240 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1d5e312ab0639735 server_cookie=3780cff7f5a8c1eb in_seq=0 out_seq=0 2024-04-25T11:25:34.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.800+0000 7f1155ffb640 1 -- 172.21.15.69:0/2028645821 <== mds.2 v2:172.21.15.152:6838/3008615657 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a3b84451c0 con 0x55a3b80cb280 2024-04-25T11:25:34.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.800+0000 7f1155ffb640 10 client.15258 handle_command_reply: tid=0 2024-04-25T11:25:34.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.800+0000 7f117ba4e740 1 client.15258 shutdown 2024-04-25T11:25:34.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.800+0000 7f117ba4e740 1 -- 172.21.15.69:0/2028645821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3b8116590 msgr2=0x55a3b84e4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:34.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.800+0000 7f117ba4e740 1 --2- 172.21.15.69:0/2028645821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3b8116590 0x55a3b84e4e80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f11640fcd60 tx=0x7f11640930c0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.800+0000 7f117ba4e740 1 -- 172.21.15.69:0/2028645821 shutdown_connections 2024-04-25T11:25:34.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.800+0000 7f117ba4e740 1 --2- 172.21.15.69:0/2028645821 >> [v2:172.21.15.152:6838/3008615657,v1:172.21.15.152:6839/3008615657] conn(0x55a3b80cb280 0x55a3b855c240 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.800+0000 7f117ba4e740 1 --2- 172.21.15.69:0/2028645821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a3b8137270 0x55a3b84e7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.800+0000 7f117ba4e740 1 --2- 172.21.15.69:0/2028645821 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3b8116590 0x55a3b84e4e80 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.800+0000 7f1155ffb640 0 client.15258 ms_handle_reset on v2:172.21.15.152:6838/3008615657 2024-04-25T11:25:34.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.800+0000 7f117ba4e740 1 --2- 172.21.15.69:0/2028645821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a3b810e3b0 0x55a3b84e4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.800+0000 7f117ba4e740 1 -- 172.21.15.69:0/2028645821 >> 172.21.15.69:0/2028645821 conn(0x55a3b812b3c0 msgr2=0x55a3b84d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:34.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.800+0000 7f117ba4e740 1 -- 172.21.15.69:0/2028645821 shutdown_connections 2024-04-25T11:25:34.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.800+0000 7f117ba4e740 1 -- 172.21.15.69:0/2028645821 wait complete. 2024-04-25T11:25:34.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.800+0000 7f117ba4e740 20 client.15258 trim_cache size 0 max 16384 2024-04-25T11:25:34.801 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-25T11:25:34.801 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "f7ed73dc-02f4-11ef-bc93-c7b262605968", 2024-04-25T11:25:34.801 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 2, 2024-04-25T11:25:34.801 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 24475, 2024-04-25T11:25:34.801 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-04-25T11:25:34.801 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-04-25T11:25:34.801 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-04-25T11:25:34.801 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 87.908227796000006, 2024-04-25T11:25:34.801 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 28, 2024-04-25T11:25:34.801 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 84, 2024-04-25T11:25:34.801 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 80, 2024-04-25T11:25:34.802 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 128.84931141800001 2024-04-25T11:25:34.802 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-25T11:25:34.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.801+0000 7f1174b9d640 1 -- 172.21.15.69:0/2258882474 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f11580afaf0 msgr2=0x7f11580b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:34.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.801+0000 7f1174b9d640 1 --2- 172.21.15.69:0/2258882474 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f11580afaf0 0x7f11580b2000 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1168022000 tx=0x7f1168020000 comp rx=0 tx=0).stop 2024-04-25T11:25:34.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.801+0000 7f1174b9d640 1 -- 172.21.15.69:0/2258882474 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1170106f30 msgr2=0x7f117019a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:34.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.801+0000 7f1174b9d640 1 --2- 172.21.15.69:0/2258882474 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1170106f30 0x7f117019a750 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f1164009230 tx=0x7f11640042f0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.801+0000 7f1174b9d640 1 -- 172.21.15.69:0/2258882474 shutdown_connections 2024-04-25T11:25:34.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.801+0000 7f1174b9d640 1 --2- 172.21.15.69:0/2258882474 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f11580afaf0 0x7f11580b2000 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.801+0000 7f1174b9d640 1 --2- 172.21.15.69:0/2258882474 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1170109890 0x7f11701a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.801+0000 7f1174b9d640 1 --2- 172.21.15.69:0/2258882474 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1170106f30 0x7f117019a750 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.801+0000 7f1174b9d640 1 --2- 172.21.15.69:0/2258882474 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11701045d0 0x7f117019a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.801+0000 7f1174b9d640 1 -- 172.21.15.69:0/2258882474 >> 172.21.15.69:0/2258882474 conn(0x7f11700fdbd0 msgr2=0x7f11701066b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:34.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.801+0000 7f1174b9d640 1 -- 172.21.15.69:0/2258882474 shutdown_connections 2024-04-25T11:25:34.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.801+0000 7f1174b9d640 1 -- 172.21.15.69:0/2258882474 wait complete. 2024-04-25T11:25:34.805 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-25T11:25:34.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.921+0000 7f85d036c640 1 -- 172.21.15.69:0/3906884912 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85c8101750 msgr2=0x7f85c8101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:34.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.921+0000 7f85d036c640 1 --2- 172.21.15.69:0/3906884912 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85c8101750 0x7f85c8101bc0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f85b8009900 tx=0x7f85b802a410 comp rx=0 tx=0).stop 2024-04-25T11:25:34.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.921+0000 7f85d036c640 1 -- 172.21.15.69:0/3906884912 shutdown_connections 2024-04-25T11:25:34.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.921+0000 7f85d036c640 1 --2- 172.21.15.69:0/3906884912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85c8102250 0x7f85c810bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.921+0000 7f85d036c640 1 --2- 172.21.15.69:0/3906884912 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85c8101750 0x7f85c8101bc0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.921+0000 7f85d036c640 1 --2- 172.21.15.69:0/3906884912 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85c81050a0 0x7f85c81054b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.921+0000 7f85d036c640 1 -- 172.21.15.69:0/3906884912 >> 172.21.15.69:0/3906884912 conn(0x7f85c80fd350 msgr2=0x7f85c80ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:34.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.921+0000 7f85d036c640 1 -- 172.21.15.69:0/3906884912 shutdown_connections 2024-04-25T11:25:34.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.921+0000 7f85d036c640 1 -- 172.21.15.69:0/3906884912 wait complete. 2024-04-25T11:25:34.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.922+0000 7f85d036c640 1 Processor -- start 2024-04-25T11:25:34.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.922+0000 7f85d036c640 1 -- start start 2024-04-25T11:25:34.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.922+0000 7f85d036c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85c8101750 0x7f85c819a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.922+0000 7f85d036c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85c8102250 0x7f85c819a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.922+0000 7f85d036c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85c81050a0 0x7f85c81a15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.922+0000 7f85d036c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f85c810f4c0 con 0x7f85c8102250 2024-04-25T11:25:34.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.922+0000 7f85d036c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f85c810f340 con 0x7f85c8101750 2024-04-25T11:25:34.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.922+0000 7f85d036c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f85c810f640 con 0x7f85c81050a0 2024-04-25T11:25:34.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.922+0000 7f85cd907640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85c8102250 0x7f85c819a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.922+0000 7f85cd907640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85c8102250 0x7f85c819a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55234/0 (socket says 172.21.15.69:55234) 2024-04-25T11:25:34.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.922+0000 7f85cd907640 1 -- 172.21.15.69:0/3052512245 learned_addr learned my addr 172.21.15.69:0/3052512245 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:34.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.922+0000 7f85ce108640 1 --2- 172.21.15.69:0/3052512245 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85c8101750 0x7f85c819a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.922+0000 7f85cd907640 1 -- 172.21.15.69:0/3052512245 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85c81050a0 msgr2=0x7f85c81a15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:34.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.922+0000 7f85cd907640 1 --2- 172.21.15.69:0/3052512245 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85c81050a0 0x7f85c81a15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.922+0000 7f85cd907640 1 -- 172.21.15.69:0/3052512245 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85c8101750 msgr2=0x7f85c819a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:34.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.922+0000 7f85cd907640 1 --2- 172.21.15.69:0/3052512245 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85c8101750 0x7f85c819a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.922+0000 7f85cd907640 1 -- 172.21.15.69:0/3052512245 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85c81a1c60 con 0x7f85c8102250 2024-04-25T11:25:34.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.923+0000 7f85cd907640 1 --2- 172.21.15.69:0/3052512245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85c8102250 0x7f85c819a660 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f85b8009230 tx=0x7f85b80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:34.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.923+0000 7f85bf7fe640 1 -- 172.21.15.69:0/3052512245 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b803f030 con 0x7f85c8102250 2024-04-25T11:25:34.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.923+0000 7f85bf7fe640 1 -- 172.21.15.69:0/3052512245 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f85b8003c90 con 0x7f85c8102250 2024-04-25T11:25:34.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.923+0000 7f85d036c640 1 -- 172.21.15.69:0/3052512245 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85c81a1ef0 con 0x7f85c8102250 2024-04-25T11:25:34.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.923+0000 7f85bf7fe640 1 -- 172.21.15.69:0/3052512245 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b8035070 con 0x7f85c8102250 2024-04-25T11:25:34.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.923+0000 7f85d036c640 1 -- 172.21.15.69:0/3052512245 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85c81a2350 con 0x7f85c8102250 2024-04-25T11:25:34.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.925+0000 7f85bf7fe640 1 -- 172.21.15.69:0/3052512245 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f85b8042030 con 0x7f85c8102250 2024-04-25T11:25:34.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.925+0000 7f85d7216740 1 Processor -- start 2024-04-25T11:25:34.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.926+0000 7f85d7216740 1 -- start start 2024-04-25T11:25:34.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.926+0000 7f85d7216740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5576e5a3e270 0x5576e5ddf370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.926+0000 7f85d7216740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5576e5a153b0 0x5576e5ddf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.926+0000 7f85d7216740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576e5a1d590 0x5576e5de67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.926+0000 7f85d7216740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5576e5d44740 con 0x5576e5a1d590 2024-04-25T11:25:34.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.926+0000 7f85d7216740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5576e5ce03a0 con 0x5576e5a153b0 2024-04-25T11:25:34.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.926+0000 7f85d7216740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5576e5d40700 con 0x5576e5a3e270 2024-04-25T11:25:34.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.926+0000 7f85bf7fe640 1 --2- 172.21.15.69:0/3052512245 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f85b40afaf0 0x7f85b40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.926+0000 7f85cd907640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576e5a1d590 0x5576e5de67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.926+0000 7f85cd907640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576e5a1d590 0x5576e5de67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55248/0 (socket says 172.21.15.69:55248) 2024-04-25T11:25:34.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.926+0000 7f85cd907640 1 -- 172.21.15.69:0/132535935 learned_addr learned my addr 172.21.15.69:0/132535935 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:34.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.926+0000 7f85ce909640 1 --2- 172.21.15.69:0/3052512245 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f85b40afaf0 0x7f85b40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.926+0000 7f85ce909640 1 --2- 172.21.15.69:0/132535935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5576e5a3e270 0x5576e5ddf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.926+0000 7f85ce108640 1 --2- 172.21.15.69:0/132535935 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5576e5a153b0 0x5576e5ddf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.926+0000 7f85bf7fe640 1 -- 172.21.15.69:0/3052512245 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f85b80ff5c0 con 0x7f85c8102250 2024-04-25T11:25:34.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.926+0000 7f85cd907640 1 -- 172.21.15.69:0/132535935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5576e5a3e270 msgr2=0x5576e5ddf370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:25:34.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.926+0000 7f85cd907640 1 --2- 172.21.15.69:0/132535935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5576e5a3e270 0x5576e5ddf370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.926+0000 7f85cd907640 1 -- 172.21.15.69:0/132535935 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5576e5a153b0 msgr2=0x5576e5ddf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:25:34.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.926+0000 7f85cd907640 1 --2- 172.21.15.69:0/132535935 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5576e5a153b0 0x5576e5ddf8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.927+0000 7f85cd907640 1 -- 172.21.15.69:0/132535935 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576e5cfc750 con 0x5576e5a1d590 2024-04-25T11:25:34.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.927+0000 7f85ce108640 1 --2- 172.21.15.69:0/132535935 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5576e5a153b0 0x5576e5ddf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:25:34.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.927+0000 7f85ce909640 1 --2- 172.21.15.69:0/3052512245 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f85b40afaf0 0x7f85b40b2000 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f85c0005c40 tx=0x7f85c0009c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:34.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.927+0000 7f85cd907640 1 --2- 172.21.15.69:0/132535935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576e5a1d590 0x5576e5de67f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f85b80fccf0 tx=0x7f85b8032f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70f75f23bd32a887 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:34.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.927+0000 7f85bcbfe640 1 -- 172.21.15.69:0/132535935 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b8049070 con 0x5576e5a1d590 2024-04-25T11:25:34.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.927+0000 7f85bcbfe640 1 -- 172.21.15.69:0/132535935 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f85b8003e40 con 0x5576e5a1d590 2024-04-25T11:25:34.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.927+0000 7f85bcbfe640 1 -- 172.21.15.69:0/132535935 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b80cba80 con 0x5576e5a1d590 2024-04-25T11:25:34.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.928+0000 7f85d7216740 1 -- 172.21.15.69:0/132535935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576e5a1d590 msgr2=0x5576e5de67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:34.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.928+0000 7f85d7216740 1 --2- 172.21.15.69:0/132535935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576e5a1d590 0x5576e5de67f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f85b80fccf0 tx=0x7f85b8032f30 comp rx=0 tx=0).stop 2024-04-25T11:25:34.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.928+0000 7f85d7216740 1 -- 172.21.15.69:0/132535935 shutdown_connections 2024-04-25T11:25:34.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.928+0000 7f85d7216740 1 --2- 172.21.15.69:0/132535935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576e5a1d590 0x5576e5de67f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.928+0000 7f85d7216740 1 --2- 172.21.15.69:0/132535935 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5576e5a153b0 0x5576e5ddf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.928+0000 7f85d7216740 1 --2- 172.21.15.69:0/132535935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5576e5a3e270 0x5576e5ddf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.928+0000 7f85d7216740 1 -- 172.21.15.69:0/132535935 >> 172.21.15.69:0/132535935 conn(0x5576e5a323c0 msgr2=0x5576e5ddad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:34.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.928+0000 7f85d7216740 1 -- 172.21.15.69:0/132535935 shutdown_connections 2024-04-25T11:25:34.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.928+0000 7f85d7216740 1 -- 172.21.15.69:0/132535935 wait complete. 2024-04-25T11:25:34.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.928+0000 7f85d7216740 1 -- start start 2024-04-25T11:25:34.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.928+0000 7f85d7216740 1 Processor -- start 2024-04-25T11:25:34.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.928+0000 7f85d7216740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5576e5a153b0 0x5576e5de8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.928+0000 7f85d7216740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5576e5a1d590 0x5576e5de94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.928+0000 7f85d7216740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576e5a3e270 0x5576e5df0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.928+0000 7f85d7216740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5576e5de7000 con 0x5576e5a3e270 2024-04-25T11:25:34.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.928+0000 7f85d7216740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5576e5ccbfe0 con 0x5576e5a1d590 2024-04-25T11:25:34.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.928+0000 7f85d7216740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5576e5de7180 con 0x5576e5a153b0 2024-04-25T11:25:34.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.929+0000 7f85d7216740 10 client.0 _ll_register_callbacks cb 0x5576e5ce2150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:25:34.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.929+0000 7f85ce909640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576e5a3e270 0x5576e5df0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.929+0000 7f85ce909640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576e5a3e270 0x5576e5df0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55262/0 (socket says 172.21.15.69:55262) 2024-04-25T11:25:34.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.929+0000 7f85ce909640 1 -- 172.21.15.69:0/2340904009 learned_addr learned my addr 172.21.15.69:0/2340904009 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:34.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.929+0000 7f85ce108640 1 --2- 172.21.15.69:0/2340904009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5576e5a153b0 0x5576e5de8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.929+0000 7f85cd907640 1 --2- 172.21.15.69:0/2340904009 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5576e5a1d590 0x5576e5de94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.929+0000 7f85ce909640 1 -- 172.21.15.69:0/2340904009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5576e5a153b0 msgr2=0x5576e5de8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:25:34.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.929+0000 7f85ce909640 1 --2- 172.21.15.69:0/2340904009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5576e5a153b0 0x5576e5de8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.929+0000 7f85ce909640 1 -- 172.21.15.69:0/2340904009 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5576e5a1d590 msgr2=0x5576e5de94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:25:34.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.929+0000 7f85ce909640 1 --2- 172.21.15.69:0/2340904009 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5576e5a1d590 0x5576e5de94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.929+0000 7f85ce909640 1 -- 172.21.15.69:0/2340904009 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576e5a562d0 con 0x5576e5a3e270 2024-04-25T11:25:34.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.929+0000 7f85ce909640 1 --2- 172.21.15.69:0/2340904009 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576e5a3e270 0x5576e5df0430 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f85c0013010 tx=0x7f85c0005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=659cd0251dd8ca2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:34.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.929+0000 7f85b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-25T11:25:34.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.929+0000 7f85b1ffb640 1 -- 172.21.15.69:0/2340904009 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c0004d20 con 0x5576e5a3e270 2024-04-25T11:25:34.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.929+0000 7f85d7216740 1 -- 172.21.15.69:0/2340904009 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5576e5d52350 con 0x5576e5a3e270 2024-04-25T11:25:34.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.929+0000 7f85b1ffb640 1 -- 172.21.15.69:0/2340904009 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f85c00043c0 con 0x5576e5a3e270 2024-04-25T11:25:34.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.929+0000 7f85b1ffb640 1 -- 172.21.15.69:0/2340904009 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f85c00045c0 con 0x5576e5a3e270 2024-04-25T11:25:34.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.929+0000 7f85d7216740 10 client.15270 fetch_fsmap learned FSMap version 28 2024-04-25T11:25:34.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.929+0000 7f85d7216740 1 -- 172.21.15.69:0/2340904009 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5576e5b206b0 con 0x5576e5a3e270 2024-04-25T11:25:34.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.930+0000 7f85b1ffb640 1 -- 172.21.15.69:0/2340904009 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5576e5d52350 con 0x5576e5a3e270 2024-04-25T11:25:34.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.930+0000 7f85b1ffb640 1 -- 172.21.15.69:0/2340904009 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f85c0027b20 con 0x5576e5a3e270 2024-04-25T11:25:34.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.930+0000 7f85d7216740 10 client.15270 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T11:25:34.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.930+0000 7f85d7216740 10 client.15270 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T11:25:34.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.930+0000 7f85d7216740 20 client.15270 populate_metadata read hostname 'smithi069' 2024-04-25T11:25:34.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.930+0000 7f85d7216740 1 --2- 172.21.15.69:0/2340904009 >> [v2:172.21.15.146:6836/2730317134,v1:172.21.15.146:6837/2730317134] conn(0x5576e59fce40 0x5576e5df6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.930+0000 7f85d7216740 4 client.15270 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:25:34.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.930+0000 7f85d7216740 1 -- 172.21.15.69:0/2340904009 --> [v2:172.21.15.146:6836/2730317134,v1:172.21.15.146:6837/2730317134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576e5d4c1c0 con 0x5576e59fce40 2024-04-25T11:25:34.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.930+0000 7f85ce108640 1 --2- 172.21.15.69:0/2340904009 >> [v2:172.21.15.146:6836/2730317134,v1:172.21.15.146:6837/2730317134] conn(0x5576e59fce40 0x5576e5df6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.931+0000 7f85ce108640 1 --2- 172.21.15.69:0/2340904009 >> [v2:172.21.15.146:6836/2730317134,v1:172.21.15.146:6837/2730317134] conn(0x5576e59fce40 0x5576e5df6d00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dc3c0a19e289abe9 server_cookie=1ac38339c8aaf9df in_seq=0 out_seq=0 2024-04-25T11:25:34.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.931+0000 7f85b1ffb640 10 client.15270 ms_handle_connect on v2:172.21.15.146:6836/2730317134 2024-04-25T11:25:34.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.931+0000 7f85b1ffb640 1 -- 172.21.15.69:0/2340904009 <== mds.1 v2:172.21.15.146:6836/2730317134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5576e5d4c1c0 con 0x5576e59fce40 2024-04-25T11:25:34.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.931+0000 7f85b1ffb640 10 client.15270 handle_command_reply: tid=0 2024-04-25T11:25:34.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.931+0000 7f85d7216740 1 client.15270 shutdown 2024-04-25T11:25:34.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.931+0000 7f85d7216740 1 -- 172.21.15.69:0/2340904009 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576e5a3e270 msgr2=0x5576e5df0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:34.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.931+0000 7f85d7216740 1 --2- 172.21.15.69:0/2340904009 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576e5a3e270 0x5576e5df0430 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f85c0013010 tx=0x7f85c0005bd0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.932+0000 7f85d7216740 1 -- 172.21.15.69:0/2340904009 shutdown_connections 2024-04-25T11:25:34.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.932+0000 7f85d7216740 1 --2- 172.21.15.69:0/2340904009 >> [v2:172.21.15.146:6836/2730317134,v1:172.21.15.146:6837/2730317134] conn(0x5576e59fce40 0x5576e5df6d00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.932+0000 7f85d7216740 1 --2- 172.21.15.69:0/2340904009 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576e5a3e270 0x5576e5df0430 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.932+0000 7f85b1ffb640 0 client.15270 ms_handle_reset on v2:172.21.15.146:6836/2730317134 2024-04-25T11:25:34.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.932+0000 7f85d7216740 1 --2- 172.21.15.69:0/2340904009 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5576e5a1d590 0x5576e5de94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.932+0000 7f85d7216740 1 --2- 172.21.15.69:0/2340904009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5576e5a153b0 0x5576e5de8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.932+0000 7f85d7216740 1 -- 172.21.15.69:0/2340904009 >> 172.21.15.69:0/2340904009 conn(0x5576e5a323c0 msgr2=0x5576e5dd9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:34.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.932+0000 7f85d7216740 1 -- 172.21.15.69:0/2340904009 shutdown_connections 2024-04-25T11:25:34.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.932+0000 7f85d7216740 1 -- 172.21.15.69:0/2340904009 wait complete. 2024-04-25T11:25:34.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.932+0000 7f85d7216740 20 client.15270 trim_cache size 0 max 16384 2024-04-25T11:25:34.942 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:34 smithi069 ceph-mon[29188]: pgmap v344: 129 pgs: 1 peering, 128 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T11:25:34.942 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:34 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/3494840236' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:25:34.942 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:34 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/793316877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:25:34.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.950+0000 7f85d7216740 1 Processor -- start 2024-04-25T11:25:34.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.950+0000 7f85d7216740 1 -- start start 2024-04-25T11:25:34.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.951+0000 7f85d7216740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576e59fce40 0x5576e5dd5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.951+0000 7f85d7216740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5576e5a153b0 0x5576e5df0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.951+0000 7f85d7216740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5576e5a1d590 0x5576e5dda6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.951+0000 7f85d7216740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5576e5df5040 con 0x5576e59fce40 2024-04-25T11:25:34.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.951+0000 7f85d7216740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5576e5df5710 con 0x5576e5a1d590 2024-04-25T11:25:34.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.951+0000 7f85d7216740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5576e5df5de0 con 0x5576e5a153b0 2024-04-25T11:25:34.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.951+0000 7f85ce108640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576e59fce40 0x5576e5dd5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.951+0000 7f85ce108640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576e59fce40 0x5576e5dd5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55266/0 (socket says 172.21.15.69:55266) 2024-04-25T11:25:34.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.951+0000 7f85ce108640 1 -- 172.21.15.69:0/2989376140 learned_addr learned my addr 172.21.15.69:0/2989376140 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:34.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.951+0000 7f85cd907640 1 --2- 172.21.15.69:0/2989376140 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5576e5a153b0 0x5576e5df0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.951+0000 7f85ce108640 1 -- 172.21.15.69:0/2989376140 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5576e5a153b0 msgr2=0x5576e5df0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:25:34.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.951+0000 7f85ce909640 1 --2- 172.21.15.69:0/2989376140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5576e5a1d590 0x5576e5dda6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.951+0000 7f85ce108640 1 --2- 172.21.15.69:0/2989376140 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5576e5a153b0 0x5576e5df0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.951+0000 7f85ce108640 1 -- 172.21.15.69:0/2989376140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5576e5a1d590 msgr2=0x5576e5dda6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:25:34.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.951+0000 7f85ce108640 1 --2- 172.21.15.69:0/2989376140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5576e5a1d590 0x5576e5dda6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.951+0000 7f85ce108640 1 -- 172.21.15.69:0/2989376140 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576e5de40f0 con 0x5576e59fce40 2024-04-25T11:25:34.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.951+0000 7f85ce108640 1 --2- 172.21.15.69:0/2989376140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576e59fce40 0x5576e5dd5a20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f85c806b4c0 tx=0x7f85c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80793ba9ac954ad2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:34.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.951+0000 7f85b3fff640 1 -- 172.21.15.69:0/2989376140 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c4004030 con 0x5576e59fce40 2024-04-25T11:25:34.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.951+0000 7f85b3fff640 1 -- 172.21.15.69:0/2989376140 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f85c4005020 con 0x5576e59fce40 2024-04-25T11:25:34.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.951+0000 7f85b3fff640 1 -- 172.21.15.69:0/2989376140 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c401d890 con 0x5576e59fce40 2024-04-25T11:25:34.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.952+0000 7f85d7216740 1 -- 172.21.15.69:0/2989376140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576e59fce40 msgr2=0x5576e5dd5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:34.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.952+0000 7f85d7216740 1 --2- 172.21.15.69:0/2989376140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576e59fce40 0x5576e5dd5a20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f85c806b4c0 tx=0x7f85c40046c0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.952+0000 7f85d7216740 1 -- 172.21.15.69:0/2989376140 shutdown_connections 2024-04-25T11:25:34.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.952+0000 7f85d7216740 1 --2- 172.21.15.69:0/2989376140 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5576e5a1d590 0x5576e5dda6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.952+0000 7f85d7216740 1 --2- 172.21.15.69:0/2989376140 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5576e5a153b0 0x5576e5df0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.952+0000 7f85d7216740 1 --2- 172.21.15.69:0/2989376140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576e59fce40 0x5576e5dd5a20 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.952+0000 7f85d7216740 1 -- 172.21.15.69:0/2989376140 >> 172.21.15.69:0/2989376140 conn(0x5576e5a323c0 msgr2=0x5576e5de75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:34.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.952+0000 7f85d7216740 1 -- 172.21.15.69:0/2989376140 shutdown_connections 2024-04-25T11:25:34.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.952+0000 7f85d7216740 1 -- 172.21.15.69:0/2989376140 wait complete. 2024-04-25T11:25:34.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.952+0000 7f85d7216740 1 -- start start 2024-04-25T11:25:34.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.952+0000 7f85d7216740 1 Processor -- start 2024-04-25T11:25:34.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.953+0000 7f85d7216740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576e59fce40 0x5576e5debd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.953+0000 7f85d7216740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5576e5a153b0 0x5576e5dec280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.953+0000 7f85d7216740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5576e5a1d590 0x5576e5def1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.953+0000 7f85d7216740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5576e5df6b80 con 0x5576e59fce40 2024-04-25T11:25:34.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.953+0000 7f85d7216740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5576e5df64b0 con 0x5576e5a153b0 2024-04-25T11:25:34.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.953+0000 7f85d7216740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5576e5df7250 con 0x5576e5a1d590 2024-04-25T11:25:34.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.953+0000 7f85d7216740 10 client.0 _ll_register_callbacks cb 0x7f85c0009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:25:34.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.953+0000 7f85ce108640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576e59fce40 0x5576e5debd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.953+0000 7f85ce108640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576e59fce40 0x5576e5debd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55272/0 (socket says 172.21.15.69:55272) 2024-04-25T11:25:34.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.953+0000 7f85ce108640 1 -- 172.21.15.69:0/4065643581 learned_addr learned my addr 172.21.15.69:0/4065643581 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:34.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.953+0000 7f85ce909640 1 --2- 172.21.15.69:0/4065643581 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5576e5a1d590 0x5576e5def1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.953+0000 7f85cd907640 1 --2- 172.21.15.69:0/4065643581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5576e5a153b0 0x5576e5dec280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.953+0000 7f85ce108640 1 -- 172.21.15.69:0/4065643581 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5576e5a1d590 msgr2=0x5576e5def1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:25:34.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.953+0000 7f85ce108640 1 --2- 172.21.15.69:0/4065643581 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5576e5a1d590 0x5576e5def1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.953+0000 7f85ce108640 1 -- 172.21.15.69:0/4065643581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5576e5a153b0 msgr2=0x5576e5dec280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:25:34.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.953+0000 7f85ce108640 1 --2- 172.21.15.69:0/4065643581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5576e5a153b0 0x5576e5dec280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.953+0000 7f85ce108640 1 -- 172.21.15.69:0/4065643581 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576e5def780 con 0x5576e59fce40 2024-04-25T11:25:34.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.953+0000 7f85ce909640 1 --2- 172.21.15.69:0/4065643581 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5576e5a1d590 0x5576e5def1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:25:34.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.953+0000 7f85cd907640 1 --2- 172.21.15.69:0/4065643581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5576e5a153b0 0x5576e5dec280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:25:34.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.953+0000 7f85ce108640 1 --2- 172.21.15.69:0/4065643581 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576e59fce40 0x5576e5debd40 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f85c400aa50 tx=0x7f85c40130f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b7577ba0c3c7d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:34.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.953+0000 7f85b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-25T11:25:34.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.953+0000 7f85b1ffb640 1 -- 172.21.15.69:0/4065643581 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c4004040 con 0x5576e59fce40 2024-04-25T11:25:34.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.953+0000 7f85b1ffb640 1 -- 172.21.15.69:0/4065643581 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f85c40041e0 con 0x5576e59fce40 2024-04-25T11:25:34.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.953+0000 7f85d7216740 1 -- 172.21.15.69:0/4065643581 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85c4005020 con 0x5576e59fce40 2024-04-25T11:25:34.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.954+0000 7f85b1ffb640 1 -- 172.21.15.69:0/4065643581 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f85c4037050 con 0x5576e59fce40 2024-04-25T11:25:34.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.954+0000 7f85d7216740 10 client.15276 fetch_fsmap learned FSMap version 28 2024-04-25T11:25:34.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.954+0000 7f85d7216740 1 -- 172.21.15.69:0/4065643581 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5576e5e5fd00 con 0x5576e59fce40 2024-04-25T11:25:34.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.954+0000 7f85b1ffb640 1 -- 172.21.15.69:0/4065643581 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f85c4005020 con 0x5576e59fce40 2024-04-25T11:25:34.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.954+0000 7f85b1ffb640 1 -- 172.21.15.69:0/4065643581 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f85c4004950 con 0x5576e59fce40 2024-04-25T11:25:34.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.954+0000 7f85d7216740 10 client.15276 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T11:25:34.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.954+0000 7f85d7216740 10 client.15276 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T11:25:34.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.954+0000 7f85d7216740 20 client.15276 populate_metadata read hostname 'smithi069' 2024-04-25T11:25:34.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.954+0000 7f85d7216740 1 --2- 172.21.15.69:0/4065643581 >> [v2:172.21.15.146:6836/2730317134,v1:172.21.15.146:6837/2730317134] conn(0x5576e5a3e270 0x5576e5e632c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:34.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.954+0000 7f85d7216740 4 client.15276 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:25:34.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.954+0000 7f85d7216740 1 -- 172.21.15.69:0/4065643581 --> [v2:172.21.15.146:6836/2730317134,v1:172.21.15.146:6837/2730317134] -- command(tid 0: {"prefix": "status"}) -- 0x5576e5cc79a0 con 0x5576e5a3e270 2024-04-25T11:25:34.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.954+0000 7f85cd907640 1 --2- 172.21.15.69:0/4065643581 >> [v2:172.21.15.146:6836/2730317134,v1:172.21.15.146:6837/2730317134] conn(0x5576e5a3e270 0x5576e5e632c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:34.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.955+0000 7f85b1ffb640 10 client.15276 ms_handle_connect on v2:172.21.15.146:6836/2730317134 2024-04-25T11:25:34.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.955+0000 7f85cd907640 1 --2- 172.21.15.69:0/4065643581 >> [v2:172.21.15.146:6836/2730317134,v1:172.21.15.146:6837/2730317134] conn(0x5576e5a3e270 0x5576e5e632c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b1ec488d57702519 server_cookie=9ef014a74bfa1d7e in_seq=0 out_seq=0 2024-04-25T11:25:34.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.955+0000 7f85b1ffb640 1 -- 172.21.15.69:0/4065643581 <== mds.1 v2:172.21.15.146:6836/2730317134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5576e5cc79a0 con 0x5576e5a3e270 2024-04-25T11:25:34.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.955+0000 7f85b1ffb640 10 client.15276 handle_command_reply: tid=0 2024-04-25T11:25:34.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.955+0000 7f85d7216740 1 client.15276 shutdown 2024-04-25T11:25:34.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.955+0000 7f85d7216740 1 -- 172.21.15.69:0/4065643581 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576e59fce40 msgr2=0x5576e5debd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:34.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.955+0000 7f85d7216740 1 --2- 172.21.15.69:0/4065643581 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576e59fce40 0x5576e5debd40 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f85c400aa50 tx=0x7f85c40130f0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.955+0000 7f85d7216740 1 -- 172.21.15.69:0/4065643581 shutdown_connections 2024-04-25T11:25:34.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.955+0000 7f85d7216740 1 --2- 172.21.15.69:0/4065643581 >> [v2:172.21.15.146:6836/2730317134,v1:172.21.15.146:6837/2730317134] conn(0x5576e5a3e270 0x5576e5e632c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.955+0000 7f85d7216740 1 --2- 172.21.15.69:0/4065643581 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5576e5a1d590 0x5576e5def1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.955+0000 7f85d7216740 1 --2- 172.21.15.69:0/4065643581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5576e5a153b0 0x5576e5dec280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.955+0000 7f85b1ffb640 0 client.15276 ms_handle_reset on v2:172.21.15.146:6836/2730317134 2024-04-25T11:25:34.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.955+0000 7f85d7216740 1 --2- 172.21.15.69:0/4065643581 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576e59fce40 0x5576e5debd40 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.955+0000 7f85d7216740 1 -- 172.21.15.69:0/4065643581 >> 172.21.15.69:0/4065643581 conn(0x5576e5a323c0 msgr2=0x5576e5dda6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:34.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.955+0000 7f85d7216740 1 -- 172.21.15.69:0/4065643581 shutdown_connections 2024-04-25T11:25:34.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.955+0000 7f85d7216740 1 -- 172.21.15.69:0/4065643581 wait complete. 2024-04-25T11:25:34.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.955+0000 7f85d7216740 20 client.15276 trim_cache size 0 max 16384 2024-04-25T11:25:34.955 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-25T11:25:34.955 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "f7ed73dc-02f4-11ef-bc93-c7b262605968", 2024-04-25T11:25:34.955 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 1, 2024-04-25T11:25:34.955 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 24485, 2024-04-25T11:25:34.956 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-04-25T11:25:34.956 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-04-25T11:25:34.956 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-04-25T11:25:34.956 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 89.072351804999997, 2024-04-25T11:25:34.956 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 28, 2024-04-25T11:25:34.956 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 84, 2024-04-25T11:25:34.956 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 80, 2024-04-25T11:25:34.956 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 131.807397337 2024-04-25T11:25:34.956 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-25T11:25:34.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.956+0000 7f85d036c640 1 -- 172.21.15.69:0/3052512245 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f85b40afaf0 msgr2=0x7f85b40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:34.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.956+0000 7f85d036c640 1 --2- 172.21.15.69:0/3052512245 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f85b40afaf0 0x7f85b40b2000 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f85c0005c40 tx=0x7f85c0009c70 comp rx=0 tx=0).stop 2024-04-25T11:25:34.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.956+0000 7f85d036c640 1 -- 172.21.15.69:0/3052512245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85c8102250 msgr2=0x7f85c819a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:34.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.956+0000 7f85d036c640 1 --2- 172.21.15.69:0/3052512245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85c8102250 0x7f85c819a660 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f85b8009230 tx=0x7f85b80042f0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.956+0000 7f85d036c640 1 -- 172.21.15.69:0/3052512245 shutdown_connections 2024-04-25T11:25:34.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.956+0000 7f85d036c640 1 --2- 172.21.15.69:0/3052512245 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f85b40afaf0 0x7f85b40b2000 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.956+0000 7f85d036c640 1 --2- 172.21.15.69:0/3052512245 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85c81050a0 0x7f85c81a15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.956+0000 7f85d036c640 1 --2- 172.21.15.69:0/3052512245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85c8102250 0x7f85c819a660 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.956+0000 7f85d036c640 1 --2- 172.21.15.69:0/3052512245 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f85c8101750 0x7f85c819a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:34.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.956+0000 7f85d036c640 1 -- 172.21.15.69:0/3052512245 >> 172.21.15.69:0/3052512245 conn(0x7f85c80fd350 msgr2=0x7f85c8105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:34.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.956+0000 7f85d036c640 1 -- 172.21.15.69:0/3052512245 shutdown_connections 2024-04-25T11:25:34.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:34.956+0000 7f85d036c640 1 -- 172.21.15.69:0/3052512245 wait complete. 2024-04-25T11:25:34.960 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T11:25:35.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.075+0000 7f9915620640 1 -- 172.21.15.69:0/1861983622 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9910103930 msgr2=0x7f9910103de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:35.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.075+0000 7f9915620640 1 --2- 172.21.15.69:0/1861983622 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9910103930 0x7f9910103de0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f9900009900 tx=0x7f990002a410 comp rx=0 tx=0).stop 2024-04-25T11:25:35.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.075+0000 7f9915620640 1 -- 172.21.15.69:0/1861983622 shutdown_connections 2024-04-25T11:25:35.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.075+0000 7f9915620640 1 --2- 172.21.15.69:0/1861983622 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9910104470 0x7f991010abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:35.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.075+0000 7f9915620640 1 --2- 172.21.15.69:0/1861983622 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9910103930 0x7f9910103de0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:35.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.075+0000 7f9915620640 1 --2- 172.21.15.69:0/1861983622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9910102f50 0x7f9910103360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:35.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.075+0000 7f9915620640 1 -- 172.21.15.69:0/1861983622 >> 172.21.15.69:0/1861983622 conn(0x7f99100fd350 msgr2=0x7f99100ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:35.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.075+0000 7f9915620640 1 -- 172.21.15.69:0/1861983622 shutdown_connections 2024-04-25T11:25:35.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.075+0000 7f9915620640 1 -- 172.21.15.69:0/1861983622 wait complete. 2024-04-25T11:25:35.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.076+0000 7f9915620640 1 Processor -- start 2024-04-25T11:25:35.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.076+0000 7f9915620640 1 -- start start 2024-04-25T11:25:35.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.076+0000 7f9915620640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9910102f50 0x7f9910199050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:35.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.076+0000 7f9915620640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9910103930 0x7f9910199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:35.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.076+0000 7f9915620640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9910104470 0x7f99101a0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:35.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.076+0000 7f9915620640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f991010e3f0 con 0x7f9910104470 2024-04-25T11:25:35.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.076+0000 7f9915620640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f991010e270 con 0x7f9910102f50 2024-04-25T11:25:35.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.076+0000 7f9915620640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f991010e570 con 0x7f9910103930 2024-04-25T11:25:35.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.076+0000 7f990f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9910104470 0x7f99101a0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:35.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.076+0000 7f990f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9910104470 0x7f99101a0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55284/0 (socket says 172.21.15.69:55284) 2024-04-25T11:25:35.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.076+0000 7f990f7fe640 1 -- 172.21.15.69:0/3866148683 learned_addr learned my addr 172.21.15.69:0/3866148683 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:35.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.076+0000 7f990effd640 1 --2- 172.21.15.69:0/3866148683 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9910102f50 0x7f9910199050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:35.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.076+0000 7f990e7fc640 1 --2- 172.21.15.69:0/3866148683 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9910103930 0x7f9910199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:35.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.077+0000 7f990f7fe640 1 -- 172.21.15.69:0/3866148683 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9910103930 msgr2=0x7f9910199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:35.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.077+0000 7f990f7fe640 1 --2- 172.21.15.69:0/3866148683 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9910103930 0x7f9910199590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:35.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.077+0000 7f990f7fe640 1 -- 172.21.15.69:0/3866148683 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9910102f50 msgr2=0x7f9910199050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:35.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.077+0000 7f990f7fe640 1 --2- 172.21.15.69:0/3866148683 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9910102f50 0x7f9910199050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:35.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.077+0000 7f990f7fe640 1 -- 172.21.15.69:0/3866148683 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99101a0b90 con 0x7f9910104470 2024-04-25T11:25:35.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.077+0000 7f990f7fe640 1 --2- 172.21.15.69:0/3866148683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9910104470 0x7f99101a0520 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f990800cad0 tx=0x7f990800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:35.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.077+0000 7f98f7fff640 1 -- 172.21.15.69:0/3866148683 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990800ae20 con 0x7f9910104470 2024-04-25T11:25:35.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.077+0000 7f98f7fff640 1 -- 172.21.15.69:0/3866148683 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9908005020 con 0x7f9910104470 2024-04-25T11:25:35.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.077+0000 7f9915620640 1 -- 172.21.15.69:0/3866148683 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99101a0e80 con 0x7f9910104470 2024-04-25T11:25:35.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.077+0000 7f98f7fff640 1 -- 172.21.15.69:0/3866148683 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990800f6b0 con 0x7f9910104470 2024-04-25T11:25:35.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.077+0000 7f9915620640 1 -- 172.21.15.69:0/3866148683 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99101a12e0 con 0x7f9910104470 2024-04-25T11:25:35.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.079+0000 7f98f7fff640 1 -- 172.21.15.69:0/3866148683 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f99080051d0 con 0x7f9910104470 2024-04-25T11:25:35.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.080+0000 7f991c5ae740 1 Processor -- start 2024-04-25T11:25:35.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.080+0000 7f991c5ae740 1 -- start start 2024-04-25T11:25:35.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.080+0000 7f991c5ae740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a3d2d0270 0x560a3d671370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:35.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.080+0000 7f991c5ae740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a3d2a73b0 0x560a3d6718f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:35.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.080+0000 7f991c5ae740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560a3d2af590 0x560a3d6787f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:35.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.080+0000 7f991c5ae740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560a3d5d6740 con 0x560a3d2d0270 2024-04-25T11:25:35.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.080+0000 7f991c5ae740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x560a3d5723a0 con 0x560a3d2a73b0 2024-04-25T11:25:35.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.080+0000 7f991c5ae740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x560a3d5d2700 con 0x560a3d2af590 2024-04-25T11:25:35.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.080+0000 7f98f7fff640 1 --2- 172.21.15.69:0/3866148683 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f98f80afaf0 0x7f98f80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:35.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.080+0000 7f990e7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a3d2d0270 0x560a3d671370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:35.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.080+0000 7f990e7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a3d2d0270 0x560a3d671370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55290/0 (socket says 172.21.15.69:55290) 2024-04-25T11:25:35.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.080+0000 7f990e7fc640 1 -- 172.21.15.69:0/1467039292 learned_addr learned my addr 172.21.15.69:0/1467039292 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:35.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.080+0000 7f990f7fe640 1 --2- 172.21.15.69:0/3866148683 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f98f80afaf0 0x7f98f80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:35.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.080+0000 7f98f7fff640 1 -- 172.21.15.69:0/3866148683 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f99080df3b0 con 0x7f9910104470 2024-04-25T11:25:35.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.080+0000 7f990effd640 1 --2- 172.21.15.69:0/1467039292 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a3d2a73b0 0x560a3d6718f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:35.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.080+0000 7f990e7fc640 1 -- 172.21.15.69:0/1467039292 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560a3d2af590 msgr2=0x560a3d6787f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:25:35.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.080+0000 7f990e7fc640 1 --2- 172.21.15.69:0/1467039292 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560a3d2af590 0x560a3d6787f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:35.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.080+0000 7f990e7fc640 1 -- 172.21.15.69:0/1467039292 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a3d2a73b0 msgr2=0x560a3d6718f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:25:35.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.080+0000 7f990e7fc640 1 --2- 172.21.15.69:0/1467039292 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a3d2a73b0 0x560a3d6718f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:35.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.080+0000 7f990e7fc640 1 -- 172.21.15.69:0/1467039292 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a3d58e750 con 0x560a3d2d0270 2024-04-25T11:25:35.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.080+0000 7f990effd640 1 --2- 172.21.15.69:0/1467039292 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a3d2a73b0 0x560a3d6718f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:25:35.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.080+0000 7f990f7fe640 1 --2- 172.21.15.69:0/3866148683 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f98f80afaf0 0x7f98f80b2000 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f99080a4730 tx=0x7f9908076010 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:35.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.081+0000 7f990e7fc640 1 --2- 172.21.15.69:0/1467039292 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a3d2d0270 0x560a3d671370 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9900005760 tx=0x7f9900005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a7db5497c1d6dc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:35.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.081+0000 7f98f57fa640 1 -- 172.21.15.69:0/1467039292 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9900035070 con 0x560a3d2d0270 2024-04-25T11:25:35.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.081+0000 7f98f57fa640 1 -- 172.21.15.69:0/1467039292 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9900041070 con 0x560a3d2d0270 2024-04-25T11:25:35.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.081+0000 7f98f57fa640 1 -- 172.21.15.69:0/1467039292 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9900003a80 con 0x560a3d2d0270 2024-04-25T11:25:35.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.081+0000 7f991c5ae740 1 -- 172.21.15.69:0/1467039292 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a3d2d0270 msgr2=0x560a3d671370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:35.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.081+0000 7f991c5ae740 1 --2- 172.21.15.69:0/1467039292 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a3d2d0270 0x560a3d671370 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9900005760 tx=0x7f9900005e50 comp rx=0 tx=0).stop 2024-04-25T11:25:35.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.082+0000 7f991c5ae740 1 -- 172.21.15.69:0/1467039292 shutdown_connections 2024-04-25T11:25:35.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.082+0000 7f991c5ae740 1 --2- 172.21.15.69:0/1467039292 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560a3d2af590 0x560a3d6787f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:35.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.082+0000 7f991c5ae740 1 --2- 172.21.15.69:0/1467039292 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a3d2a73b0 0x560a3d6718f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:35.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.082+0000 7f991c5ae740 1 --2- 172.21.15.69:0/1467039292 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a3d2d0270 0x560a3d671370 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:35.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.082+0000 7f991c5ae740 1 -- 172.21.15.69:0/1467039292 >> 172.21.15.69:0/1467039292 conn(0x560a3d2c43c0 msgr2=0x560a3d66cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:35.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.082+0000 7f991c5ae740 1 -- 172.21.15.69:0/1467039292 shutdown_connections 2024-04-25T11:25:35.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.082+0000 7f991c5ae740 1 -- 172.21.15.69:0/1467039292 wait complete. 2024-04-25T11:25:35.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.082+0000 7f991c5ae740 1 -- start start 2024-04-25T11:25:35.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.082+0000 7f991c5ae740 1 Processor -- start 2024-04-25T11:25:35.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.082+0000 7f991c5ae740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560a3d2a73b0 0x560a3d67b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:35.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.082+0000 7f991c5ae740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a3d2af590 0x560a3d67b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:35.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.082+0000 7f991c5ae740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a3d2d0270 0x560a3d682790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:35.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.082+0000 7f991c5ae740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560a3d6791e0 con 0x560a3d2af590 2024-04-25T11:25:35.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.082+0000 7f991c5ae740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x560a3d55dfe0 con 0x560a3d2d0270 2024-04-25T11:25:35.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.082+0000 7f991c5ae740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x560a3d679360 con 0x560a3d2a73b0 2024-04-25T11:25:35.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.082+0000 7f991c5ae740 10 client.0 _ll_register_callbacks cb 0x560a3d574150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:25:35.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.082+0000 7f990effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a3d2af590 0x560a3d67b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:35.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.082+0000 7f990effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a3d2af590 0x560a3d67b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55296/0 (socket says 172.21.15.69:55296) 2024-04-25T11:25:35.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.082+0000 7f990effd640 1 -- 172.21.15.69:0/796290155 learned_addr learned my addr 172.21.15.69:0/796290155 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:35.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.082+0000 7f990e7fc640 1 --2- 172.21.15.69:0/796290155 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560a3d2a73b0 0x560a3d67b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:35.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.083+0000 7f990e7fc640 1 --2- 172.21.15.69:0/796290155 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a3d2d0270 0x560a3d682790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:35.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.083+0000 7f990effd640 1 -- 172.21.15.69:0/796290155 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560a3d2a73b0 msgr2=0x560a3d67b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:25:35.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.083+0000 7f990effd640 1 --2- 172.21.15.69:0/796290155 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560a3d2a73b0 0x560a3d67b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:35.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.083+0000 7f990effd640 1 -- 172.21.15.69:0/796290155 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a3d2d0270 msgr2=0x560a3d682790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:25:35.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.083+0000 7f990effd640 1 --2- 172.21.15.69:0/796290155 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a3d2d0270 0x560a3d682790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:35.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.083+0000 7f990effd640 1 -- 172.21.15.69:0/796290155 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a3d2e82d0 con 0x560a3d2af590 2024-04-25T11:25:35.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.083+0000 7f990effd640 1 --2- 172.21.15.69:0/796290155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a3d2af590 0x560a3d67b830 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f990400b940 tx=0x7f990400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=595648817a0c9c2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:35.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.083+0000 7f98f2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-25T11:25:35.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.083+0000 7f98f2ff5640 1 -- 172.21.15.69:0/796290155 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990401b670 con 0x560a3d2af590 2024-04-25T11:25:35.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.083+0000 7f98f2ff5640 1 -- 172.21.15.69:0/796290155 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f990401b810 con 0x560a3d2af590 2024-04-25T11:25:35.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.083+0000 7f991c5ae740 1 -- 172.21.15.69:0/796290155 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9900041070 con 0x560a3d2af590 2024-04-25T11:25:35.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.083+0000 7f98f2ff5640 1 -- 172.21.15.69:0/796290155 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9904024050 con 0x560a3d2af590 2024-04-25T11:25:35.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.083+0000 7f991c5ae740 10 client.15288 fetch_fsmap learned FSMap version 28 2024-04-25T11:25:35.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.083+0000 7f991c5ae740 1 -- 172.21.15.69:0/796290155 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560a3d3b26b0 con 0x560a3d2af590 2024-04-25T11:25:35.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.083+0000 7f98f2ff5640 1 -- 172.21.15.69:0/796290155 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9900041070 con 0x560a3d2af590 2024-04-25T11:25:35.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.084+0000 7f98f2ff5640 1 -- 172.21.15.69:0/796290155 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9904012950 con 0x560a3d2af590 2024-04-25T11:25:35.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.084+0000 7f991c5ae740 10 client.15288 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T11:25:35.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.084+0000 7f991c5ae740 10 client.15288 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T11:25:35.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.084+0000 7f991c5ae740 20 client.15288 populate_metadata read hostname 'smithi069' 2024-04-25T11:25:35.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.084+0000 7f991c5ae740 1 --2- 172.21.15.69:0/796290155 >> [v2:172.21.15.146:6838/2013283448,v1:172.21.15.146:6839/2013283448] conn(0x560a3d28ee40 0x560a3d689090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:35.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.084+0000 7f991c5ae740 4 client.15288 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:25:35.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.084+0000 7f991c5ae740 1 -- 172.21.15.69:0/796290155 --> [v2:172.21.15.146:6838/2013283448,v1:172.21.15.146:6839/2013283448] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a3d5e4350 con 0x560a3d28ee40 2024-04-25T11:25:35.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.084+0000 7f990e7fc640 1 --2- 172.21.15.69:0/796290155 >> [v2:172.21.15.146:6838/2013283448,v1:172.21.15.146:6839/2013283448] conn(0x560a3d28ee40 0x560a3d689090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:35.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.084+0000 7f98f2ff5640 10 client.15288 ms_handle_connect on v2:172.21.15.146:6838/2013283448 2024-04-25T11:25:35.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.084+0000 7f990e7fc640 1 --2- 172.21.15.69:0/796290155 >> [v2:172.21.15.146:6838/2013283448,v1:172.21.15.146:6839/2013283448] conn(0x560a3d28ee40 0x560a3d689090 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2388fa1d5dce35d server_cookie=a1d2666af13ccd76 in_seq=0 out_seq=0 2024-04-25T11:25:35.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.085+0000 7f98f2ff5640 1 -- 172.21.15.69:0/796290155 <== mds.0 v2:172.21.15.146:6838/2013283448 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560a3d5e4350 con 0x560a3d28ee40 2024-04-25T11:25:35.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.085+0000 7f98f2ff5640 10 client.15288 handle_command_reply: tid=0 2024-04-25T11:25:35.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.085+0000 7f991c5ae740 1 client.15288 shutdown 2024-04-25T11:25:35.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.085+0000 7f991c5ae740 1 -- 172.21.15.69:0/796290155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a3d2af590 msgr2=0x560a3d67b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:35.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.085+0000 7f991c5ae740 1 --2- 172.21.15.69:0/796290155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a3d2af590 0x560a3d67b830 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f990400b940 tx=0x7f990400be00 comp rx=0 tx=0).stop 2024-04-25T11:25:35.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.085+0000 7f991c5ae740 1 -- 172.21.15.69:0/796290155 shutdown_connections 2024-04-25T11:25:35.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.085+0000 7f991c5ae740 1 --2- 172.21.15.69:0/796290155 >> [v2:172.21.15.146:6838/2013283448,v1:172.21.15.146:6839/2013283448] conn(0x560a3d28ee40 0x560a3d689090 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:35.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.085+0000 7f991c5ae740 1 --2- 172.21.15.69:0/796290155 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a3d2d0270 0x560a3d682790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:35.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.085+0000 7f98f2ff5640 0 client.15288 ms_handle_reset on v2:172.21.15.146:6838/2013283448 2024-04-25T11:25:35.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.085+0000 7f991c5ae740 1 --2- 172.21.15.69:0/796290155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a3d2af590 0x560a3d67b830 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:35.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.085+0000 7f991c5ae740 1 --2- 172.21.15.69:0/796290155 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560a3d2a73b0 0x560a3d67b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:35.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.085+0000 7f991c5ae740 1 -- 172.21.15.69:0/796290155 >> 172.21.15.69:0/796290155 conn(0x560a3d2c43c0 msgr2=0x560a3d66bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:35.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.085+0000 7f991c5ae740 1 -- 172.21.15.69:0/796290155 shutdown_connections 2024-04-25T11:25:35.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.085+0000 7f991c5ae740 1 -- 172.21.15.69:0/796290155 wait complete. 2024-04-25T11:25:35.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.085+0000 7f991c5ae740 20 client.15288 trim_cache size 0 max 16384 2024-04-25T11:25:35.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.104+0000 7f991c5ae740 1 Processor -- start 2024-04-25T11:25:35.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.104+0000 7f991c5ae740 1 -- start start 2024-04-25T11:25:35.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.104+0000 7f991c5ae740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a3d28ee40 0x560a3d67a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:35.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.104+0000 7f991c5ae740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560a3d2a73b0 0x560a3d6826b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:35.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.104+0000 7f991c5ae740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a3d2af590 0x560a3d67aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:35.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.104+0000 7f991c5ae740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560a3d6873d0 con 0x560a3d28ee40 2024-04-25T11:25:35.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.104+0000 7f991c5ae740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x560a3d687aa0 con 0x560a3d2af590 2024-04-25T11:25:35.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.104+0000 7f991c5ae740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x560a3d688170 con 0x560a3d2a73b0 2024-04-25T11:25:35.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.104+0000 7f990e7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a3d28ee40 0x560a3d67a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:35.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.104+0000 7f990e7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a3d28ee40 0x560a3d67a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55312/0 (socket says 172.21.15.69:55312) 2024-04-25T11:25:35.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.104+0000 7f990e7fc640 1 -- 172.21.15.69:0/3275107547 learned_addr learned my addr 172.21.15.69:0/3275107547 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:35.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.104+0000 7f990effd640 1 --2- 172.21.15.69:0/3275107547 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560a3d2a73b0 0x560a3d6826b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:35.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.104+0000 7f990e7fc640 1 -- 172.21.15.69:0/3275107547 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560a3d2a73b0 msgr2=0x560a3d6826b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:25:35.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.104+0000 7f990e7fc640 1 --2- 172.21.15.69:0/3275107547 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560a3d2a73b0 0x560a3d6826b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:35.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.104+0000 7f990e7fc640 1 -- 172.21.15.69:0/3275107547 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a3d2af590 msgr2=0x560a3d67aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:25:35.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.104+0000 7f990e7fc640 1 --2- 172.21.15.69:0/3275107547 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a3d2af590 0x560a3d67aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:35.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.104+0000 7f990e7fc640 1 -- 172.21.15.69:0/3275107547 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a3d676170 con 0x560a3d28ee40 2024-04-25T11:25:35.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.105+0000 7f990e7fc640 1 --2- 172.21.15.69:0/3275107547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a3d28ee40 0x560a3d67a580 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f990002a930 tx=0x7f9900005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9f6ed2f9cc2ebd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:35.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.105+0000 7f98f4ff9640 1 -- 172.21.15.69:0/3275107547 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9900035070 con 0x560a3d28ee40 2024-04-25T11:25:35.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.105+0000 7f98f4ff9640 1 -- 172.21.15.69:0/3275107547 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9900041040 con 0x560a3d28ee40 2024-04-25T11:25:35.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.105+0000 7f98f4ff9640 1 -- 172.21.15.69:0/3275107547 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990004e040 con 0x560a3d28ee40 2024-04-25T11:25:35.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.105+0000 7f991c5ae740 1 -- 172.21.15.69:0/3275107547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a3d28ee40 msgr2=0x560a3d67a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:35.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.105+0000 7f991c5ae740 1 --2- 172.21.15.69:0/3275107547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a3d28ee40 0x560a3d67a580 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f990002a930 tx=0x7f9900005e50 comp rx=0 tx=0).stop 2024-04-25T11:25:35.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.105+0000 7f991c5ae740 1 -- 172.21.15.69:0/3275107547 shutdown_connections 2024-04-25T11:25:35.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.105+0000 7f991c5ae740 1 --2- 172.21.15.69:0/3275107547 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a3d2af590 0x560a3d67aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:35.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.105+0000 7f991c5ae740 1 --2- 172.21.15.69:0/3275107547 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560a3d2a73b0 0x560a3d6826b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:35.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.105+0000 7f991c5ae740 1 --2- 172.21.15.69:0/3275107547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a3d28ee40 0x560a3d67a580 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:35.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.105+0000 7f991c5ae740 1 -- 172.21.15.69:0/3275107547 >> 172.21.15.69:0/3275107547 conn(0x560a3d2c43c0 msgr2=0x560a3d679910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:35.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.106+0000 7f991c5ae740 1 -- 172.21.15.69:0/3275107547 shutdown_connections 2024-04-25T11:25:35.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.106+0000 7f991c5ae740 1 -- 172.21.15.69:0/3275107547 wait complete. 2024-04-25T11:25:35.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.106+0000 7f991c5ae740 1 -- start start 2024-04-25T11:25:35.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.106+0000 7f991c5ae740 1 Processor -- start 2024-04-25T11:25:35.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.106+0000 7f991c5ae740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a3d28ee40 0x560a3d67f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:35.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.106+0000 7f991c5ae740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a3d2a73b0 0x560a3d67fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:35.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.106+0000 7f991c5ae740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560a3d2af590 0x560a3d6eff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:35.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.106+0000 7f991c5ae740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560a3d688f10 con 0x560a3d28ee40 2024-04-25T11:25:35.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.106+0000 7f991c5ae740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x560a3d688840 con 0x560a3d2a73b0 2024-04-25T11:25:35.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.106+0000 7f991c5ae740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x560a3d6895e0 con 0x560a3d2af590 2024-04-25T11:25:35.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.106+0000 7f991c5ae740 10 client.0 _ll_register_callbacks cb 0x7f9904004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:25:35.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.106+0000 7f990e7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a3d28ee40 0x560a3d67f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:35.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.106+0000 7f990e7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a3d28ee40 0x560a3d67f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55316/0 (socket says 172.21.15.69:55316) 2024-04-25T11:25:35.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.106+0000 7f990e7fc640 1 -- 172.21.15.69:0/2257923589 learned_addr learned my addr 172.21.15.69:0/2257923589 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:35.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.106+0000 7f990e7fc640 1 -- 172.21.15.69:0/2257923589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560a3d2af590 msgr2=0x560a3d6eff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:25:35.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.106+0000 7f990e7fc640 1 --2- 172.21.15.69:0/2257923589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560a3d2af590 0x560a3d6eff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:35.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.106+0000 7f990e7fc640 1 -- 172.21.15.69:0/2257923589 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a3d2a73b0 msgr2=0x560a3d67fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:25:35.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.106+0000 7f990e7fc640 1 --2- 172.21.15.69:0/2257923589 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a3d2a73b0 0x560a3d67fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:35.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.106+0000 7f990e7fc640 1 -- 172.21.15.69:0/2257923589 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a3d680120 con 0x560a3d28ee40 2024-04-25T11:25:35.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.107+0000 7f990e7fc640 1 --2- 172.21.15.69:0/2257923589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a3d28ee40 0x560a3d67f570 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f99000057c0 tx=0x7f9900058170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76d791ee29fefc97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:35.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.107+0000 7f98f2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-25T11:25:35.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.107+0000 7f98f2ff5640 1 -- 172.21.15.69:0/2257923589 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9900035070 con 0x560a3d28ee40 2024-04-25T11:25:35.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.107+0000 7f98f2ff5640 1 -- 172.21.15.69:0/2257923589 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f9900004240 con 0x560a3d28ee40 2024-04-25T11:25:35.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.107+0000 7f991c5ae740 1 -- 172.21.15.69:0/2257923589 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9900041040 con 0x560a3d28ee40 2024-04-25T11:25:35.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.107+0000 7f98f2ff5640 1 -- 172.21.15.69:0/2257923589 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9900068050 con 0x560a3d28ee40 2024-04-25T11:25:35.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.107+0000 7f991c5ae740 10 client.15294 fetch_fsmap learned FSMap version 28 2024-04-25T11:25:35.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.107+0000 7f991c5ae740 1 -- 172.21.15.69:0/2257923589 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560a3d6f14f0 con 0x560a3d28ee40 2024-04-25T11:25:35.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.107+0000 7f98f2ff5640 1 -- 172.21.15.69:0/2257923589 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9900041040 con 0x560a3d28ee40 2024-04-25T11:25:35.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.107+0000 7f98f2ff5640 1 -- 172.21.15.69:0/2257923589 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f990003bab0 con 0x560a3d28ee40 2024-04-25T11:25:35.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.107+0000 7f991c5ae740 10 client.15294 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T11:25:35.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.107+0000 7f991c5ae740 10 client.15294 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T11:25:35.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.107+0000 7f991c5ae740 20 client.15294 populate_metadata read hostname 'smithi069' 2024-04-25T11:25:35.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.107+0000 7f991c5ae740 1 --2- 172.21.15.69:0/2257923589 >> [v2:172.21.15.146:6838/2013283448,v1:172.21.15.146:6839/2013283448] conn(0x560a3d2d0270 0x560a3d6f4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:35.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.107+0000 7f991c5ae740 4 client.15294 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:25:35.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.107+0000 7f991c5ae740 1 -- 172.21.15.69:0/2257923589 --> [v2:172.21.15.146:6838/2013283448,v1:172.21.15.146:6839/2013283448] -- command(tid 0: {"prefix": "status"}) -- 0x560a3d5de1c0 con 0x560a3d2d0270 2024-04-25T11:25:35.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.108+0000 7f990effd640 1 --2- 172.21.15.69:0/2257923589 >> [v2:172.21.15.146:6838/2013283448,v1:172.21.15.146:6839/2013283448] conn(0x560a3d2d0270 0x560a3d6f4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:35.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.108+0000 7f990effd640 1 --2- 172.21.15.69:0/2257923589 >> [v2:172.21.15.146:6838/2013283448,v1:172.21.15.146:6839/2013283448] conn(0x560a3d2d0270 0x560a3d6f4880 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c5d39d8e78d2279 server_cookie=ab6e93cca39b1ba6 in_seq=0 out_seq=0 2024-04-25T11:25:35.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.108+0000 7f98f2ff5640 10 client.15294 ms_handle_connect on v2:172.21.15.146:6838/2013283448 2024-04-25T11:25:35.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.108+0000 7f98f2ff5640 1 -- 172.21.15.69:0/2257923589 <== mds.0 v2:172.21.15.146:6838/2013283448 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x560a3d5de1c0 con 0x560a3d2d0270 2024-04-25T11:25:35.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.108+0000 7f98f2ff5640 10 client.15294 handle_command_reply: tid=0 2024-04-25T11:25:35.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.108+0000 7f991c5ae740 1 client.15294 shutdown 2024-04-25T11:25:35.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.108+0000 7f991c5ae740 1 -- 172.21.15.69:0/2257923589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a3d28ee40 msgr2=0x560a3d67f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:35.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.108+0000 7f991c5ae740 1 --2- 172.21.15.69:0/2257923589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a3d28ee40 0x560a3d67f570 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f99000057c0 tx=0x7f9900058170 comp rx=0 tx=0).stop 2024-04-25T11:25:35.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.108+0000 7f991c5ae740 1 -- 172.21.15.69:0/2257923589 shutdown_connections 2024-04-25T11:25:35.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.108+0000 7f991c5ae740 1 --2- 172.21.15.69:0/2257923589 >> [v2:172.21.15.146:6838/2013283448,v1:172.21.15.146:6839/2013283448] conn(0x560a3d2d0270 0x560a3d6f4880 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:35.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.108+0000 7f991c5ae740 1 --2- 172.21.15.69:0/2257923589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560a3d2af590 0x560a3d6eff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:35.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.108+0000 7f991c5ae740 1 --2- 172.21.15.69:0/2257923589 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a3d2a73b0 0x560a3d67fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:35.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.108+0000 7f991c5ae740 1 --2- 172.21.15.69:0/2257923589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a3d28ee40 0x560a3d67f570 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:35.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.108+0000 7f98f2ff5640 0 client.15294 ms_handle_reset on v2:172.21.15.146:6838/2013283448 2024-04-25T11:25:35.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.108+0000 7f991c5ae740 1 -- 172.21.15.69:0/2257923589 >> 172.21.15.69:0/2257923589 conn(0x560a3d2c43c0 msgr2=0x560a3d66fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:35.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.109+0000 7f991c5ae740 1 -- 172.21.15.69:0/2257923589 shutdown_connections 2024-04-25T11:25:35.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.109+0000 7f991c5ae740 1 -- 172.21.15.69:0/2257923589 wait complete. 2024-04-25T11:25:35.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.109+0000 7f991c5ae740 20 client.15294 trim_cache size 0 max 16384 2024-04-25T11:25:35.108 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-25T11:25:35.108 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "f7ed73dc-02f4-11ef-bc93-c7b262605968", 2024-04-25T11:25:35.108 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 0, 2024-04-25T11:25:35.109 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 24515, 2024-04-25T11:25:35.109 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-04-25T11:25:35.109 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-04-25T11:25:35.109 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-04-25T11:25:35.109 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 113.48983322300001, 2024-04-25T11:25:35.109 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 28, 2024-04-25T11:25:35.109 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 84, 2024-04-25T11:25:35.109 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 80, 2024-04-25T11:25:35.109 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 120.430912209 2024-04-25T11:25:35.109 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-25T11:25:35.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.109+0000 7f9915620640 1 -- 172.21.15.69:0/3866148683 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f98f80afaf0 msgr2=0x7f98f80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:35.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.109+0000 7f9915620640 1 --2- 172.21.15.69:0/3866148683 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f98f80afaf0 0x7f98f80b2000 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f99080a4730 tx=0x7f9908076010 comp rx=0 tx=0).stop 2024-04-25T11:25:35.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.109+0000 7f9915620640 1 -- 172.21.15.69:0/3866148683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9910104470 msgr2=0x7f99101a0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:35.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.109+0000 7f9915620640 1 --2- 172.21.15.69:0/3866148683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9910104470 0x7f99101a0520 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f990800cad0 tx=0x7f990800cf90 comp rx=0 tx=0).stop 2024-04-25T11:25:35.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.109+0000 7f9915620640 1 -- 172.21.15.69:0/3866148683 shutdown_connections 2024-04-25T11:25:35.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.109+0000 7f9915620640 1 --2- 172.21.15.69:0/3866148683 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f98f80afaf0 0x7f98f80b2000 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:35.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.109+0000 7f9915620640 1 --2- 172.21.15.69:0/3866148683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9910104470 0x7f99101a0520 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:35.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.109+0000 7f9915620640 1 --2- 172.21.15.69:0/3866148683 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9910103930 0x7f9910199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:35.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.109+0000 7f9915620640 1 --2- 172.21.15.69:0/3866148683 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9910102f50 0x7f9910199050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:35.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.109+0000 7f9915620640 1 -- 172.21.15.69:0/3866148683 >> 172.21.15.69:0/3866148683 conn(0x7f99100fd350 msgr2=0x7f9910104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:35.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.109+0000 7f9915620640 1 -- 172.21.15.69:0/3866148683 shutdown_connections 2024-04-25T11:25:35.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:35.109+0000 7f9915620640 1 -- 172.21.15.69:0/3866148683 wait complete. 2024-04-25T11:25:35.113 INFO:tasks.cephadm:Setup complete, yielding 2024-04-25T11:25:35.114 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T11:25:35.124 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi069.front.sepia.ceph.com 2024-04-25T11:25:35.125 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-25T11:25:35.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:34 smithi152 ceph-mon[32701]: pgmap v344: 129 pgs: 1 peering, 128 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T11:25:35.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:34 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/3494840236' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:25:35.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:34 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/793316877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:25:35.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:34 smithi146 ceph-mon[32821]: pgmap v344: 129 pgs: 1 peering, 128 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T11:25:35.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:34 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/3494840236' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:25:35.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:34 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/793316877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:25:35.360 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:25:36.192 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:35 smithi069 ceph-mon[29188]: pgmap v346: 129 pgs: 2 peering, 127 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 767 B/s wr, 13 op/s 2024-04-25T11:25:36.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:35 smithi152 ceph-mon[32701]: pgmap v346: 129 pgs: 2 peering, 127 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 767 B/s wr, 13 op/s 2024-04-25T11:25:36.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:35 smithi146 ceph-mon[32821]: pgmap v346: 129 pgs: 2 peering, 127 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 767 B/s wr, 13 op/s 2024-04-25T11:25:37.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.292+0000 7f5c62fe2700 1 -- 172.21.15.69:0/2425855896 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c5c104040 msgr2=0x7f5c5c104510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:37.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.292+0000 7f5c62fe2700 1 --2- 172.21.15.69:0/2425855896 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c5c104040 0x7f5c5c104510 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5c44009990 tx=0x7f5c44009ca0 comp rx=0 tx=0).stop 2024-04-25T11:25:37.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.292+0000 7f5c62fe2700 1 -- 172.21.15.69:0/2425855896 shutdown_connections 2024-04-25T11:25:37.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.292+0000 7f5c62fe2700 1 --2- 172.21.15.69:0/2425855896 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c5c104ba0 0x7f5c5c10b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:37.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.292+0000 7f5c62fe2700 1 --2- 172.21.15.69:0/2425855896 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c5c103660 0x7f5c5c103a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:37.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.292+0000 7f5c62fe2700 1 --2- 172.21.15.69:0/2425855896 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c5c104040 0x7f5c5c104510 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:37.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.292+0000 7f5c62fe2700 1 -- 172.21.15.69:0/2425855896 >> 172.21.15.69:0/2425855896 conn(0x7f5c5c0fded0 msgr2=0x7f5c5c100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:37.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.292+0000 7f5c62fe2700 1 -- 172.21.15.69:0/2425855896 shutdown_connections 2024-04-25T11:25:37.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.292+0000 7f5c62fe2700 1 -- 172.21.15.69:0/2425855896 wait complete. 2024-04-25T11:25:37.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.293+0000 7f5c62fe2700 1 Processor -- start 2024-04-25T11:25:37.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.293+0000 7f5c62fe2700 1 -- start start 2024-04-25T11:25:37.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.293+0000 7f5c62fe2700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c5c103660 0x7f5c5c199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:37.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.293+0000 7f5c62fe2700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c5c104040 0x7f5c5c199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:37.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.293+0000 7f5c62fe2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c5c104ba0 0x7f5c5c1a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:37.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.293+0000 7f5c62fe2700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c10ea60 con 0x7f5c5c104ba0 2024-04-25T11:25:37.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.293+0000 7f5c62fe2700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c10e8e0 con 0x7f5c5c103660 2024-04-25T11:25:37.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.293+0000 7f5c62fe2700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c10ebe0 con 0x7f5c5c104040 2024-04-25T11:25:37.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.293+0000 7f5c6157f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c5c104ba0 0x7f5c5c1a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:37.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.293+0000 7f5c6157f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c5c104ba0 0x7f5c5c1a0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55344/0 (socket says 172.21.15.69:55344) 2024-04-25T11:25:37.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.293+0000 7f5c6157f700 1 -- 172.21.15.69:0/2828937135 learned_addr learned my addr 172.21.15.69:0/2828937135 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:37.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.294+0000 7f5c5bfff700 1 --2- 172.21.15.69:0/2828937135 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c5c104040 0x7f5c5c199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:37.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.294+0000 7f5c60d7e700 1 --2- 172.21.15.69:0/2828937135 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c5c103660 0x7f5c5c199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:37.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.294+0000 7f5c6157f700 1 -- 172.21.15.69:0/2828937135 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c5c104040 msgr2=0x7f5c5c199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:37.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.294+0000 7f5c6157f700 1 --2- 172.21.15.69:0/2828937135 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c5c104040 0x7f5c5c199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:37.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.294+0000 7f5c6157f700 1 -- 172.21.15.69:0/2828937135 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c5c103660 msgr2=0x7f5c5c199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:37.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.294+0000 7f5c6157f700 1 --2- 172.21.15.69:0/2828937135 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c5c103660 0x7f5c5c199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:37.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.294+0000 7f5c6157f700 1 -- 172.21.15.69:0/2828937135 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c5c1a10e0 con 0x7f5c5c104ba0 2024-04-25T11:25:37.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.294+0000 7f5c6157f700 1 --2- 172.21.15.69:0/2828937135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c5c104ba0 0x7f5c5c1a0af0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5c5400c900 tx=0x7f5c5400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:37.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.294+0000 7f5c59ffb700 1 -- 172.21.15.69:0/2828937135 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c54007b00 con 0x7f5c5c104ba0 2024-04-25T11:25:37.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.294+0000 7f5c59ffb700 1 -- 172.21.15.69:0/2828937135 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c54004d30 con 0x7f5c5c104ba0 2024-04-25T11:25:37.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.294+0000 7f5c59ffb700 1 -- 172.21.15.69:0/2828937135 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c540055e0 con 0x7f5c5c104ba0 2024-04-25T11:25:37.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.294+0000 7f5c62fe2700 1 -- 172.21.15.69:0/2828937135 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c5c1a13d0 con 0x7f5c5c104ba0 2024-04-25T11:25:37.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.294+0000 7f5c62fe2700 1 -- 172.21.15.69:0/2828937135 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c5c1a18c0 con 0x7f5c5c104ba0 2024-04-25T11:25:37.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.296+0000 7f5c62fe2700 1 -- 172.21.15.69:0/2828937135 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c5c051190 con 0x7f5c5c104ba0 2024-04-25T11:25:37.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.297+0000 7f5c59ffb700 1 -- 172.21.15.69:0/2828937135 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f5c540047a0 con 0x7f5c5c104ba0 2024-04-25T11:25:37.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.298+0000 7f5c59ffb700 1 --2- 172.21.15.69:0/2828937135 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f5c480afc00 0x7f5c480b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:37.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.298+0000 7f5c60d7e700 1 --2- 172.21.15.69:0/2828937135 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f5c480afc00 0x7f5c480b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:37.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.298+0000 7f5c59ffb700 1 -- 172.21.15.69:0/2828937135 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f5c540debc0 con 0x7f5c5c104ba0 2024-04-25T11:25:37.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.298+0000 7f5c60d7e700 1 --2- 172.21.15.69:0/2828937135 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f5c480afc00 0x7f5c480b2110 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c06c160 tx=0x7f5c4c009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:37.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.303+0000 7f5c59ffb700 1 -- 172.21.15.69:0/2828937135 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5c54014070 con 0x7f5c5c104ba0 2024-04-25T11:25:37.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.518+0000 7f5c62fe2700 1 -- 172.21.15.69:0/2828937135 --> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f5c5c063ad0 con 0x7f5c480afc00 2024-04-25T11:25:37.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.520+0000 7f5c59ffb700 1 -- 172.21.15.69:0/2828937135 <== mgr.14152 v2:172.21.15.69:6800/4220065409 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f5c5c063ad0 con 0x7f5c480afc00 2024-04-25T11:25:37.512 INFO:teuthology.orchestra.run.smithi069.stdout:Backend: cephadm 2024-04-25T11:25:37.512 INFO:teuthology.orchestra.run.smithi069.stdout:Available: Yes 2024-04-25T11:25:37.512 INFO:teuthology.orchestra.run.smithi069.stdout:Paused: No 2024-04-25T11:25:37.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.522+0000 7f5c62fe2700 1 -- 172.21.15.69:0/2828937135 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f5c480afc00 msgr2=0x7f5c480b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:37.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.522+0000 7f5c62fe2700 1 --2- 172.21.15.69:0/2828937135 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f5c480afc00 0x7f5c480b2110 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c06c160 tx=0x7f5c4c009040 comp rx=0 tx=0).stop 2024-04-25T11:25:37.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.522+0000 7f5c62fe2700 1 -- 172.21.15.69:0/2828937135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c5c104ba0 msgr2=0x7f5c5c1a0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:37.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.522+0000 7f5c62fe2700 1 --2- 172.21.15.69:0/2828937135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c5c104ba0 0x7f5c5c1a0af0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5c5400c900 tx=0x7f5c5400ccc0 comp rx=0 tx=0).stop 2024-04-25T11:25:37.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.522+0000 7f5c62fe2700 1 -- 172.21.15.69:0/2828937135 shutdown_connections 2024-04-25T11:25:37.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.522+0000 7f5c62fe2700 1 --2- 172.21.15.69:0/2828937135 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f5c480afc00 0x7f5c480b2110 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:37.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.522+0000 7f5c62fe2700 1 --2- 172.21.15.69:0/2828937135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c5c104ba0 0x7f5c5c1a0af0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:37.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.522+0000 7f5c62fe2700 1 --2- 172.21.15.69:0/2828937135 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c5c103660 0x7f5c5c199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:37.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.522+0000 7f5c62fe2700 1 --2- 172.21.15.69:0/2828937135 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c5c104040 0x7f5c5c199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:37.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.522+0000 7f5c62fe2700 1 -- 172.21.15.69:0/2828937135 >> 172.21.15.69:0/2828937135 conn(0x7f5c5c0fded0 msgr2=0x7f5c5c107610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:37.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.522+0000 7f5c62fe2700 1 -- 172.21.15.69:0/2828937135 shutdown_connections 2024-04-25T11:25:37.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:37.522+0000 7f5c62fe2700 1 -- 172.21.15.69:0/2828937135 wait complete. 2024-04-25T11:25:38.351 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-25T11:25:38.593 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:25:39.040 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:38 smithi069 ceph-mon[29188]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:25:39.092 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:38 smithi069 ceph-mon[29188]: pgmap v347: 129 pgs: 2 peering, 127 active+clean; 644 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 895 B/s wr, 8 op/s 2024-04-25T11:25:39.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:38 smithi152 ceph-mon[32701]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:25:39.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:38 smithi152 ceph-mon[32701]: pgmap v347: 129 pgs: 2 peering, 127 active+clean; 644 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 895 B/s wr, 8 op/s 2024-04-25T11:25:39.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:38 smithi146 ceph-mon[32821]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:25:39.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:38 smithi146 ceph-mon[32821]: pgmap v347: 129 pgs: 2 peering, 127 active+clean; 644 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 895 B/s wr, 8 op/s 2024-04-25T11:25:40.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.385+0000 7f51b2f97700 1 -- 172.21.15.69:0/1014820664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51ac104210 msgr2=0x7f51ac104620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:40.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.385+0000 7f51b2f97700 1 --2- 172.21.15.69:0/1014820664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51ac104210 0x7f51ac104620 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f51a0009990 tx=0x7f51a0009ca0 comp rx=0 tx=0).stop 2024-04-25T11:25:40.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.385+0000 7f51b2f97700 1 -- 172.21.15.69:0/1014820664 shutdown_connections 2024-04-25T11:25:40.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.385+0000 7f51b2f97700 1 --2- 172.21.15.69:0/1014820664 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f51ac105ce0 0x7f51ac10c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:40.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.385+0000 7f51b2f97700 1 --2- 172.21.15.69:0/1014820664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51ac104210 0x7f51ac104620 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:40.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.385+0000 7f51b2f97700 1 --2- 172.21.15.69:0/1014820664 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51ac105340 0x7f51ac105750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:40.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.385+0000 7f51b2f97700 1 -- 172.21.15.69:0/1014820664 >> 172.21.15.69:0/1014820664 conn(0x7f51ac0ff870 msgr2=0x7f51ac101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:40.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.385+0000 7f51b2f97700 1 -- 172.21.15.69:0/1014820664 shutdown_connections 2024-04-25T11:25:40.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.385+0000 7f51b2f97700 1 -- 172.21.15.69:0/1014820664 wait complete. 2024-04-25T11:25:40.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.386+0000 7f51b2f97700 1 Processor -- start 2024-04-25T11:25:40.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.386+0000 7f51b2f97700 1 -- start start 2024-04-25T11:25:40.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.386+0000 7f51b2f97700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51ac104210 0x7f51ac19a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:40.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.386+0000 7f51b2f97700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51ac105340 0x7f51ac19ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:40.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.386+0000 7f51b2f97700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f51ac105ce0 0x7f51ac1a1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:40.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.386+0000 7f51b2f97700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f51ac10fb80 con 0x7f51ac104210 2024-04-25T11:25:40.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.386+0000 7f51b2f97700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f51ac10fa00 con 0x7f51ac105340 2024-04-25T11:25:40.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.386+0000 7f51b2f97700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f51ac10fd00 con 0x7f51ac105ce0 2024-04-25T11:25:40.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.386+0000 7f51b0d33700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51ac104210 0x7f51ac19a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:40.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.386+0000 7f51b0d33700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51ac104210 0x7f51ac19a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55372/0 (socket says 172.21.15.69:55372) 2024-04-25T11:25:40.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.386+0000 7f51b0d33700 1 -- 172.21.15.69:0/3074036485 learned_addr learned my addr 172.21.15.69:0/3074036485 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:40.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.386+0000 7f51abfff700 1 --2- 172.21.15.69:0/3074036485 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51ac105340 0x7f51ac19ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:40.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.386+0000 7f51b1534700 1 --2- 172.21.15.69:0/3074036485 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f51ac105ce0 0x7f51ac1a1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:40.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.386+0000 7f51b0d33700 1 -- 172.21.15.69:0/3074036485 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f51ac105ce0 msgr2=0x7f51ac1a1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:40.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.386+0000 7f51b0d33700 1 --2- 172.21.15.69:0/3074036485 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f51ac105ce0 0x7f51ac1a1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:40.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.386+0000 7f51b0d33700 1 -- 172.21.15.69:0/3074036485 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51ac105340 msgr2=0x7f51ac19ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:40.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.386+0000 7f51b0d33700 1 --2- 172.21.15.69:0/3074036485 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51ac105340 0x7f51ac19ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:40.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.386+0000 7f51b0d33700 1 -- 172.21.15.69:0/3074036485 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51ac1a2290 con 0x7f51ac104210 2024-04-25T11:25:40.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.387+0000 7f51b0d33700 1 --2- 172.21.15.69:0/3074036485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51ac104210 0x7f51ac19a730 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f51a0006010 tx=0x7f51a00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:40.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.387+0000 7f51a9ffb700 1 -- 172.21.15.69:0/3074036485 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a0027070 con 0x7f51ac104210 2024-04-25T11:25:40.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.387+0000 7f51a9ffb700 1 -- 172.21.15.69:0/3074036485 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f51a0004020 con 0x7f51ac104210 2024-04-25T11:25:40.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.387+0000 7f51b2f97700 1 -- 172.21.15.69:0/3074036485 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51ac1a2520 con 0x7f51ac104210 2024-04-25T11:25:40.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.387+0000 7f51a9ffb700 1 -- 172.21.15.69:0/3074036485 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a0022cb0 con 0x7f51ac104210 2024-04-25T11:25:40.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.387+0000 7f51b2f97700 1 -- 172.21.15.69:0/3074036485 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51ac1a29f0 con 0x7f51ac104210 2024-04-25T11:25:40.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.388+0000 7f51b2f97700 1 -- 172.21.15.69:0/3074036485 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51ac051190 con 0x7f51ac104210 2024-04-25T11:25:40.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.389+0000 7f51a9ffb700 1 -- 172.21.15.69:0/3074036485 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f51a002a070 con 0x7f51ac104210 2024-04-25T11:25:40.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.390+0000 7f51a9ffb700 1 --2- 172.21.15.69:0/3074036485 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f519c0afb80 0x7f519c0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:40.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.390+0000 7f51abfff700 1 --2- 172.21.15.69:0/3074036485 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f519c0afb80 0x7f519c0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:40.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.390+0000 7f51a9ffb700 1 -- 172.21.15.69:0/3074036485 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f51a00dffc0 con 0x7f51ac104210 2024-04-25T11:25:40.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.391+0000 7f51abfff700 1 --2- 172.21.15.69:0/3074036485 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f519c0afb80 0x7f519c0b2090 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f51ac19b7e0 tx=0x7f519800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:40.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.395+0000 7f51a9ffb700 1 -- 172.21.15.69:0/3074036485 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f51a0013070 con 0x7f51ac104210 2024-04-25T11:25:40.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.603+0000 7f51b2f97700 1 -- 172.21.15.69:0/3074036485 --> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f51ac063b90 con 0x7f519c0afb80 2024-04-25T11:25:40.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.616+0000 7f51a9ffb700 1 -- 172.21.15.69:0/3074036485 <== mgr.14152 v2:172.21.15.69:6800/4220065409 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4092 (secure 0 0 0) 0x7f51ac063b90 con 0x7f519c0afb80 2024-04-25T11:25:40.609 INFO:teuthology.orchestra.run.smithi069.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T11:25:40.609 INFO:teuthology.orchestra.run.smithi069.stdout:mds.a smithi069 running (2m) 19s ago 2m 86.3M - 19.0.0-3291-g994e2f20 d53c96779a29 4700ba9fb454 2024-04-25T11:25:40.609 INFO:teuthology.orchestra.run.smithi069.stdout:mds.b smithi146 running (2m) 60s ago 2m 94.2M - 19.0.0-3291-g994e2f20 d53c96779a29 26a675f1aaf4 2024-04-25T11:25:40.609 INFO:teuthology.orchestra.run.smithi069.stdout:mds.c smithi152 running (2m) 60s ago 2m 84.6M - 19.0.0-3291-g994e2f20 d53c96779a29 f812ac2e4ae3 2024-04-25T11:25:40.609 INFO:teuthology.orchestra.run.smithi069.stdout:mds.d smithi069 running (2m) 19s ago 2m 83.0M - 19.0.0-3291-g994e2f20 d53c96779a29 178baf59a50b 2024-04-25T11:25:40.609 INFO:teuthology.orchestra.run.smithi069.stdout:mds.e smithi146 running (2m) 60s ago 2m 87.0M - 19.0.0-3291-g994e2f20 d53c96779a29 6f4c14e69976 2024-04-25T11:25:40.610 INFO:teuthology.orchestra.run.smithi069.stdout:mds.f smithi152 running (2m) 60s ago 2m 82.4M - 19.0.0-3291-g994e2f20 d53c96779a29 840d6a5b4e77 2024-04-25T11:25:40.610 INFO:teuthology.orchestra.run.smithi069.stdout:mds.g smithi069 running (2m) 19s ago 2m 85.2M - 19.0.0-3291-g994e2f20 d53c96779a29 bf0ac5d14701 2024-04-25T11:25:40.610 INFO:teuthology.orchestra.run.smithi069.stdout:mds.h smithi146 running (2m) 60s ago 2m 84.7M - 19.0.0-3291-g994e2f20 d53c96779a29 e2ddf7bbd963 2024-04-25T11:25:40.610 INFO:teuthology.orchestra.run.smithi069.stdout:mds.i smithi152 running (2m) 60s ago 2m 83.9M - 19.0.0-3291-g994e2f20 d53c96779a29 9c74b6b22d60 2024-04-25T11:25:40.610 INFO:teuthology.orchestra.run.smithi069.stdout:mds.j smithi069 running (2m) 19s ago 2m 84.0M - 19.0.0-3291-g994e2f20 d53c96779a29 755177b434f5 2024-04-25T11:25:40.610 INFO:teuthology.orchestra.run.smithi069.stdout:mds.k smithi146 running (2m) 60s ago 2m 82.5M - 19.0.0-3291-g994e2f20 d53c96779a29 115cfd316cf7 2024-04-25T11:25:40.610 INFO:teuthology.orchestra.run.smithi069.stdout:mds.l smithi152 running (2m) 60s ago 2m 85.2M - 19.0.0-3291-g994e2f20 d53c96779a29 98e10171bbd1 2024-04-25T11:25:40.610 INFO:teuthology.orchestra.run.smithi069.stdout:mgr.x smithi069 *:9283,8765 running (9m) 19s ago 9m 587M - 19.0.0-3291-g994e2f20 d53c96779a29 447f3c9a8cd8 2024-04-25T11:25:40.610 INFO:teuthology.orchestra.run.smithi069.stdout:mgr.y smithi146 *:8443,8765 running (7m) 60s ago 7m 520M - 19.0.0-3291-g994e2f20 d53c96779a29 91e9d1422874 2024-04-25T11:25:40.610 INFO:teuthology.orchestra.run.smithi069.stdout:mgr.z smithi152 *:8443,8765 running (7m) 60s ago 7m 521M - 19.0.0-3291-g994e2f20 d53c96779a29 36ffa8d398c7 2024-04-25T11:25:40.610 INFO:teuthology.orchestra.run.smithi069.stdout:mon.a smithi069 running (9m) 19s ago 10m 118M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 a2cd3fa68258 2024-04-25T11:25:40.611 INFO:teuthology.orchestra.run.smithi069.stdout:mon.b smithi146 running (8m) 60s ago 8m 108M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 9c36beef881f 2024-04-25T11:25:40.611 INFO:teuthology.orchestra.run.smithi069.stdout:mon.c smithi152 running (8m) 60s ago 8m 107M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 98da677f5104 2024-04-25T11:25:40.611 INFO:teuthology.orchestra.run.smithi069.stdout:osd.0 smithi069 running (7m) 19s ago 7m 123M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 232293110a41 2024-04-25T11:25:40.611 INFO:teuthology.orchestra.run.smithi069.stdout:osd.1 smithi146 running (7m) 60s ago 7m 116M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 4a8ee29f3ed7 2024-04-25T11:25:40.611 INFO:teuthology.orchestra.run.smithi069.stdout:osd.2 smithi152 running (6m) 60s ago 6m 114M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 679e9636a1e3 2024-04-25T11:25:40.611 INFO:teuthology.orchestra.run.smithi069.stdout:osd.3 smithi069 running (6m) 19s ago 6m 114M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 01056ca5a80b 2024-04-25T11:25:40.611 INFO:teuthology.orchestra.run.smithi069.stdout:osd.4 smithi146 running (5m) 60s ago 5m 110M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 b2786c557fd4 2024-04-25T11:25:40.611 INFO:teuthology.orchestra.run.smithi069.stdout:osd.5 smithi152 running (5m) 60s ago 5m 122M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 5dabd07b1f36 2024-04-25T11:25:40.611 INFO:teuthology.orchestra.run.smithi069.stdout:osd.6 smithi069 running (5m) 19s ago 5m 118M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 a793b52289b3 2024-04-25T11:25:40.611 INFO:teuthology.orchestra.run.smithi069.stdout:osd.7 smithi146 running (4m) 60s ago 4m 111M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 40ee3dc36199 2024-04-25T11:25:40.611 INFO:teuthology.orchestra.run.smithi069.stdout:osd.8 smithi152 running (4m) 60s ago 4m 120M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 4559c80314fb 2024-04-25T11:25:40.612 INFO:teuthology.orchestra.run.smithi069.stdout:osd.9 smithi069 running (3m) 19s ago 3m 120M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 18114e4ddf16 2024-04-25T11:25:40.612 INFO:teuthology.orchestra.run.smithi069.stdout:osd.10 smithi146 running (3m) 60s ago 3m 111M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 17a6e62b0b9b 2024-04-25T11:25:40.612 INFO:teuthology.orchestra.run.smithi069.stdout:osd.11 smithi152 running (2m) 60s ago 2m 108M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 7d80ca925b06 2024-04-25T11:25:40.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.619+0000 7f51b2f97700 1 -- 172.21.15.69:0/3074036485 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f519c0afb80 msgr2=0x7f519c0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:40.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.619+0000 7f51b2f97700 1 --2- 172.21.15.69:0/3074036485 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f519c0afb80 0x7f519c0b2090 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f51ac19b7e0 tx=0x7f519800b410 comp rx=0 tx=0).stop 2024-04-25T11:25:40.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.619+0000 7f51b2f97700 1 -- 172.21.15.69:0/3074036485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51ac104210 msgr2=0x7f51ac19a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:40.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.619+0000 7f51b2f97700 1 --2- 172.21.15.69:0/3074036485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51ac104210 0x7f51ac19a730 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f51a0006010 tx=0x7f51a00047b0 comp rx=0 tx=0).stop 2024-04-25T11:25:40.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.619+0000 7f51b2f97700 1 -- 172.21.15.69:0/3074036485 shutdown_connections 2024-04-25T11:25:40.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.619+0000 7f51b2f97700 1 --2- 172.21.15.69:0/3074036485 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f519c0afb80 0x7f519c0b2090 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:40.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.619+0000 7f51b2f97700 1 --2- 172.21.15.69:0/3074036485 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f51ac105ce0 0x7f51ac1a1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:40.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.619+0000 7f51b2f97700 1 --2- 172.21.15.69:0/3074036485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51ac104210 0x7f51ac19a730 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:40.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.619+0000 7f51b2f97700 1 --2- 172.21.15.69:0/3074036485 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51ac105340 0x7f51ac19ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:40.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.619+0000 7f51b2f97700 1 -- 172.21.15.69:0/3074036485 >> 172.21.15.69:0/3074036485 conn(0x7f51ac0ff870 msgr2=0x7f51ac108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:40.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.619+0000 7f51b2f97700 1 -- 172.21.15.69:0/3074036485 shutdown_connections 2024-04-25T11:25:40.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:40.619+0000 7f51b2f97700 1 -- 172.21.15.69:0/3074036485 wait complete. 2024-04-25T11:25:41.192 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:40 smithi069 ceph-mon[29188]: pgmap v348: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 895 B/s wr, 12 op/s 2024-04-25T11:25:41.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:40 smithi152 ceph-mon[32701]: pgmap v348: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 895 B/s wr, 12 op/s 2024-04-25T11:25:41.341 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-25T11:25:41.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:40 smithi146 ceph-mon[32821]: pgmap v348: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 895 B/s wr, 12 op/s 2024-04-25T11:25:41.585 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:25:41.963 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:41 smithi069 ceph-mon[29188]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:25:42.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:41 smithi152 ceph-mon[32701]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:25:42.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:41 smithi146 ceph-mon[32821]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:25:43.064 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:42 smithi069 ceph-mon[29188]: pgmap v349: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 803 B/s wr, 12 op/s 2024-04-25T11:25:43.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:42 smithi152 ceph-mon[32701]: pgmap v349: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 803 B/s wr, 12 op/s 2024-04-25T11:25:43.249 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:42 smithi146 ceph-mon[32821]: pgmap v349: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 803 B/s wr, 12 op/s 2024-04-25T11:25:43.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.308+0000 7f329a52a700 1 -- 172.21.15.69:0/162089561 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32941057b0 msgr2=0x7f3294105bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:43.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.308+0000 7f329a52a700 1 --2- 172.21.15.69:0/162089561 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32941057b0 0x7f3294105bc0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3288009990 tx=0x7f3288009ca0 comp rx=0 tx=0).stop 2024-04-25T11:25:43.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.308+0000 7f329a52a700 1 -- 172.21.15.69:0/162089561 shutdown_connections 2024-04-25T11:25:43.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.308+0000 7f329a52a700 1 --2- 172.21.15.69:0/162089561 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3294102b50 0x7f329410c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:43.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.308+0000 7f329a52a700 1 --2- 172.21.15.69:0/162089561 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32941057b0 0x7f3294105bc0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:43.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.308+0000 7f329a52a700 1 --2- 172.21.15.69:0/162089561 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3294102200 0x7f3294102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:43.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.308+0000 7f329a52a700 1 -- 172.21.15.69:0/162089561 >> 172.21.15.69:0/162089561 conn(0x7f32940fded0 msgr2=0x7f3294100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:43.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.308+0000 7f329a52a700 1 -- 172.21.15.69:0/162089561 shutdown_connections 2024-04-25T11:25:43.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.308+0000 7f329a52a700 1 -- 172.21.15.69:0/162089561 wait complete. 2024-04-25T11:25:43.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.309+0000 7f329a52a700 1 Processor -- start 2024-04-25T11:25:43.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.309+0000 7f329a52a700 1 -- start start 2024-04-25T11:25:43.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.309+0000 7f329a52a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3294102200 0x7f329419a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:43.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.309+0000 7f329a52a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3294102b50 0x7f329419aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:43.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.309+0000 7f329a52a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32941057b0 0x7f32941a1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:43.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.309+0000 7f329a52a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f329410fb80 con 0x7f3294102200 2024-04-25T11:25:43.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.309+0000 7f329a52a700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f329410fa00 con 0x7f32941057b0 2024-04-25T11:25:43.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.309+0000 7f329a52a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f329410fd00 con 0x7f3294102b50 2024-04-25T11:25:43.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.309+0000 7f3293fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3294102200 0x7f329419a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:43.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.310+0000 7f3293fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3294102200 0x7f329419a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43296/0 (socket says 172.21.15.69:43296) 2024-04-25T11:25:43.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.310+0000 7f3293fff700 1 -- 172.21.15.69:0/3071383863 learned_addr learned my addr 172.21.15.69:0/3071383863 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:43.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.310+0000 7f3298ac7700 1 --2- 172.21.15.69:0/3071383863 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32941057b0 0x7f32941a1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:43.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.310+0000 7f32937fe700 1 --2- 172.21.15.69:0/3071383863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3294102b50 0x7f329419aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:43.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.310+0000 7f3293fff700 1 -- 172.21.15.69:0/3071383863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3294102b50 msgr2=0x7f329419aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:43.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.310+0000 7f3293fff700 1 --2- 172.21.15.69:0/3071383863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3294102b50 0x7f329419aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:43.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.310+0000 7f3293fff700 1 -- 172.21.15.69:0/3071383863 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32941057b0 msgr2=0x7f32941a1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:43.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.310+0000 7f3293fff700 1 --2- 172.21.15.69:0/3071383863 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32941057b0 0x7f32941a1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:43.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.310+0000 7f3293fff700 1 -- 172.21.15.69:0/3071383863 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32941a22c0 con 0x7f3294102200 2024-04-25T11:25:43.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.310+0000 7f3293fff700 1 --2- 172.21.15.69:0/3071383863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3294102200 0x7f329419a760 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f32880098e0 tx=0x7f3288009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:43.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.310+0000 7f32917fa700 1 -- 172.21.15.69:0/3071383863 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3288027070 con 0x7f3294102200 2024-04-25T11:25:43.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.310+0000 7f329a52a700 1 -- 172.21.15.69:0/3071383863 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32941a2550 con 0x7f3294102200 2024-04-25T11:25:43.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.310+0000 7f32917fa700 1 -- 172.21.15.69:0/3071383863 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3288004210 con 0x7f3294102200 2024-04-25T11:25:43.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.310+0000 7f32917fa700 1 -- 172.21.15.69:0/3071383863 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3288022e50 con 0x7f3294102200 2024-04-25T11:25:43.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.310+0000 7f329a52a700 1 -- 172.21.15.69:0/3071383863 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32941a2a80 con 0x7f3294102200 2024-04-25T11:25:43.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.311+0000 7f329a52a700 1 -- 172.21.15.69:0/3071383863 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3294051190 con 0x7f3294102200 2024-04-25T11:25:43.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.313+0000 7f32917fa700 1 -- 172.21.15.69:0/3071383863 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f328802a070 con 0x7f3294102200 2024-04-25T11:25:43.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.313+0000 7f32917fa700 1 --2- 172.21.15.69:0/3071383863 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f327c0afda0 0x7f327c0b22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:43.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.313+0000 7f32937fe700 1 --2- 172.21.15.69:0/3071383863 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f327c0afda0 0x7f327c0b22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:43.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.313+0000 7f32917fa700 1 -- 172.21.15.69:0/3071383863 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f32880e0350 con 0x7f3294102200 2024-04-25T11:25:43.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.314+0000 7f32937fe700 1 --2- 172.21.15.69:0/3071383863 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f327c0afda0 0x7f327c0b22b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3284005950 tx=0x7f3284005ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:43.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.317+0000 7f32917fa700 1 -- 172.21.15.69:0/3071383863 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3288013070 con 0x7f3294102200 2024-04-25T11:25:43.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.526+0000 7f329a52a700 1 -- 172.21.15.69:0/3071383863 --> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f3294063ad0 con 0x7f327c0afda0 2024-04-25T11:25:43.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.532+0000 7f32917fa700 1 -- 172.21.15.69:0/3071383863 <== mgr.14152 v2:172.21.15.69:6800/4220065409 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f3294063ad0 con 0x7f327c0afda0 2024-04-25T11:25:43.525 INFO:teuthology.orchestra.run.smithi069.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-25T11:25:43.525 INFO:teuthology.orchestra.run.smithi069.stdout:mds.all 12/12 63s ago 2m smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi146=b;smithi146=e;smithi146=h;smithi146=k;smithi152=c;smithi152=f;smithi152=i;smithi152=l;count:12 2024-04-25T11:25:43.525 INFO:teuthology.orchestra.run.smithi069.stdout:mgr 3/3 63s ago 8m smithi069=x;smithi146=y;smithi152=z;count:3 2024-04-25T11:25:43.525 INFO:teuthology.orchestra.run.smithi069.stdout:mon 3/3 63s ago 8m smithi069:172.21.15.69=a;smithi146:172.21.15.146=b;smithi152:172.21.15.152=c;count:3 2024-04-25T11:25:43.525 INFO:teuthology.orchestra.run.smithi069.stdout:osd 12 63s ago - 2024-04-25T11:25:43.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.534+0000 7f329a52a700 1 -- 172.21.15.69:0/3071383863 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f327c0afda0 msgr2=0x7f327c0b22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:43.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.534+0000 7f329a52a700 1 --2- 172.21.15.69:0/3071383863 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f327c0afda0 0x7f327c0b22b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3284005950 tx=0x7f3284005ed0 comp rx=0 tx=0).stop 2024-04-25T11:25:43.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.534+0000 7f329a52a700 1 -- 172.21.15.69:0/3071383863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3294102200 msgr2=0x7f329419a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:43.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.534+0000 7f329a52a700 1 --2- 172.21.15.69:0/3071383863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3294102200 0x7f329419a760 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f32880098e0 tx=0x7f3288009f90 comp rx=0 tx=0).stop 2024-04-25T11:25:43.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.534+0000 7f329a52a700 1 -- 172.21.15.69:0/3071383863 shutdown_connections 2024-04-25T11:25:43.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.534+0000 7f329a52a700 1 --2- 172.21.15.69:0/3071383863 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f327c0afda0 0x7f327c0b22b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:43.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.534+0000 7f329a52a700 1 --2- 172.21.15.69:0/3071383863 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32941057b0 0x7f32941a1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:43.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.534+0000 7f329a52a700 1 --2- 172.21.15.69:0/3071383863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3294102200 0x7f329419a760 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:43.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.534+0000 7f329a52a700 1 --2- 172.21.15.69:0/3071383863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3294102b50 0x7f329419aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:43.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.534+0000 7f329a52a700 1 -- 172.21.15.69:0/3071383863 >> 172.21.15.69:0/3071383863 conn(0x7f32940fded0 msgr2=0x7f32941087c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:43.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.534+0000 7f329a52a700 1 -- 172.21.15.69:0/3071383863 shutdown_connections 2024-04-25T11:25:43.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:43.534+0000 7f329a52a700 1 -- 172.21.15.69:0/3071383863 wait complete. 2024-04-25T11:25:44.253 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-25T11:25:44.492 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:25:45.192 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:44 smithi069 ceph-mon[29188]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:25:45.192 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:44 smithi069 ceph-mon[29188]: pgmap v350: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 614 B/s wr, 10 op/s 2024-04-25T11:25:45.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:44 smithi152 ceph-mon[32701]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:25:45.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:44 smithi152 ceph-mon[32701]: pgmap v350: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 614 B/s wr, 10 op/s 2024-04-25T11:25:45.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:44 smithi146 ceph-mon[32821]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:25:45.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:44 smithi146 ceph-mon[32821]: pgmap v350: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 614 B/s wr, 10 op/s 2024-04-25T11:25:46.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.211+0000 7facc7554700 1 -- 172.21.15.69:0/1805583762 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7facc0103660 msgr2=0x7facc0103a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:46.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.211+0000 7facc7554700 1 --2- 172.21.15.69:0/1805583762 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7facc0103660 0x7facc0103a70 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7facbc00b0a0 tx=0x7facbc00b3b0 comp rx=0 tx=0).stop 2024-04-25T11:25:46.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.211+0000 7facc7554700 1 -- 172.21.15.69:0/1805583762 shutdown_connections 2024-04-25T11:25:46.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.211+0000 7facc7554700 1 --2- 172.21.15.69:0/1805583762 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7facc0104ba0 0x7facc010b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:46.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.211+0000 7facc7554700 1 --2- 172.21.15.69:0/1805583762 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7facc0103660 0x7facc0103a70 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:46.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.211+0000 7facc7554700 1 --2- 172.21.15.69:0/1805583762 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facc0104040 0x7facc0104510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:46.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.211+0000 7facc7554700 1 -- 172.21.15.69:0/1805583762 >> 172.21.15.69:0/1805583762 conn(0x7facc00fded0 msgr2=0x7facc0100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:46.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.211+0000 7facc7554700 1 -- 172.21.15.69:0/1805583762 shutdown_connections 2024-04-25T11:25:46.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.211+0000 7facc7554700 1 -- 172.21.15.69:0/1805583762 wait complete. 2024-04-25T11:25:46.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.212+0000 7facc7554700 1 Processor -- start 2024-04-25T11:25:46.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.212+0000 7facc7554700 1 -- start start 2024-04-25T11:25:46.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.212+0000 7facc7554700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7facc0103660 0x7facc0199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:46.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.212+0000 7facc7554700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7facc0104040 0x7facc0199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:46.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.212+0000 7facc7554700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facc0104ba0 0x7facc01a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:46.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.212+0000 7facc7554700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7facc010ea80 con 0x7facc0104040 2024-04-25T11:25:46.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.212+0000 7facc7554700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7facc010e900 con 0x7facc0104ba0 2024-04-25T11:25:46.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.212+0000 7facc7554700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7facc010ec00 con 0x7facc0103660 2024-04-25T11:25:46.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.213+0000 7facc4aef700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7facc0104040 0x7facc0199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:46.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.213+0000 7facc52f0700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7facc0103660 0x7facc0199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:46.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.213+0000 7facc4aef700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7facc0104040 0x7facc0199c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43326/0 (socket says 172.21.15.69:43326) 2024-04-25T11:25:46.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.213+0000 7facc4aef700 1 -- 172.21.15.69:0/2081586455 learned_addr learned my addr 172.21.15.69:0/2081586455 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:46.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.213+0000 7facc5af1700 1 --2- 172.21.15.69:0/2081586455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facc0104ba0 0x7facc01a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:46.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.213+0000 7facc4aef700 1 -- 172.21.15.69:0/2081586455 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7facc0103660 msgr2=0x7facc0199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:46.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.213+0000 7facc4aef700 1 --2- 172.21.15.69:0/2081586455 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7facc0103660 0x7facc0199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:46.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.213+0000 7facc4aef700 1 -- 172.21.15.69:0/2081586455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facc0104ba0 msgr2=0x7facc01a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:46.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.213+0000 7facc4aef700 1 --2- 172.21.15.69:0/2081586455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facc0104ba0 0x7facc01a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:46.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.213+0000 7facc4aef700 1 -- 172.21.15.69:0/2081586455 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc01a1270 con 0x7facc0104040 2024-04-25T11:25:46.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.213+0000 7facc4aef700 1 --2- 172.21.15.69:0/2081586455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7facc0104040 0x7facc0199c50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7facb000b530 tx=0x7facb000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:46.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.213+0000 7facb67fc700 1 -- 172.21.15.69:0/2081586455 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb0015020 con 0x7facc0104040 2024-04-25T11:25:46.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.213+0000 7facc7554700 1 -- 172.21.15.69:0/2081586455 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facc01a1560 con 0x7facc0104040 2024-04-25T11:25:46.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.213+0000 7facb67fc700 1 -- 172.21.15.69:0/2081586455 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7facb0011cc0 con 0x7facc0104040 2024-04-25T11:25:46.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.213+0000 7facb67fc700 1 -- 172.21.15.69:0/2081586455 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb0010c80 con 0x7facc0104040 2024-04-25T11:25:46.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.213+0000 7facc7554700 1 -- 172.21.15.69:0/2081586455 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facc01a1af0 con 0x7facc0104040 2024-04-25T11:25:46.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.214+0000 7facc7554700 1 -- 172.21.15.69:0/2081586455 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facc0051190 con 0x7facc0104040 2024-04-25T11:25:46.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.216+0000 7facb67fc700 1 -- 172.21.15.69:0/2081586455 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7facb0018020 con 0x7facc0104040 2024-04-25T11:25:46.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.217+0000 7facb67fc700 1 --2- 172.21.15.69:0/2081586455 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7facac0afb80 0x7facac0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:46.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.217+0000 7facc52f0700 1 --2- 172.21.15.69:0/2081586455 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7facac0afb80 0x7facac0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:46.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.217+0000 7facb67fc700 1 -- 172.21.15.69:0/2081586455 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7facb00dec60 con 0x7facc0104040 2024-04-25T11:25:46.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.217+0000 7facc52f0700 1 --2- 172.21.15.69:0/2081586455 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7facac0afb80 0x7facac0b2090 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7facbc00b830 tx=0x7facbc007ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:46.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.220+0000 7facb67fc700 1 -- 172.21.15.69:0/2081586455 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7facb001f020 con 0x7facc0104040 2024-04-25T11:25:46.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.424+0000 7facc7554700 1 -- 172.21.15.69:0/2081586455 --> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7facc0063ad0 con 0x7facac0afb80 2024-04-25T11:25:46.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.426+0000 7facb67fc700 1 -- 172.21.15.69:0/2081586455 <== mgr.14152 v2:172.21.15.69:6800/4220065409 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7facc0063ad0 con 0x7facac0afb80 2024-04-25T11:25:46.419 INFO:teuthology.orchestra.run.smithi069.stdout:HOST ADDR LABELS STATUS 2024-04-25T11:25:46.419 INFO:teuthology.orchestra.run.smithi069.stdout:smithi069 172.21.15.69 2024-04-25T11:25:46.419 INFO:teuthology.orchestra.run.smithi069.stdout:smithi146 172.21.15.146 2024-04-25T11:25:46.419 INFO:teuthology.orchestra.run.smithi069.stdout:smithi152 172.21.15.152 2024-04-25T11:25:46.419 INFO:teuthology.orchestra.run.smithi069.stdout:3 hosts in cluster 2024-04-25T11:25:46.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.429+0000 7facc7554700 1 -- 172.21.15.69:0/2081586455 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7facac0afb80 msgr2=0x7facac0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:46.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.429+0000 7facc7554700 1 --2- 172.21.15.69:0/2081586455 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7facac0afb80 0x7facac0b2090 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7facbc00b830 tx=0x7facbc007ed0 comp rx=0 tx=0).stop 2024-04-25T11:25:46.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.429+0000 7facc7554700 1 -- 172.21.15.69:0/2081586455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7facc0104040 msgr2=0x7facc0199c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:46.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.429+0000 7facc7554700 1 --2- 172.21.15.69:0/2081586455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7facc0104040 0x7facc0199c50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7facb000b530 tx=0x7facb000b840 comp rx=0 tx=0).stop 2024-04-25T11:25:46.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.429+0000 7facc7554700 1 -- 172.21.15.69:0/2081586455 shutdown_connections 2024-04-25T11:25:46.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.429+0000 7facc7554700 1 --2- 172.21.15.69:0/2081586455 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7facac0afb80 0x7facac0b2090 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:46.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.429+0000 7facc7554700 1 --2- 172.21.15.69:0/2081586455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7facc0104ba0 0x7facc01a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:46.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.429+0000 7facc7554700 1 --2- 172.21.15.69:0/2081586455 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7facc0103660 0x7facc0199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:46.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.429+0000 7facc7554700 1 --2- 172.21.15.69:0/2081586455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7facc0104040 0x7facc0199c50 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:46.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.429+0000 7facc7554700 1 -- 172.21.15.69:0/2081586455 >> 172.21.15.69:0/2081586455 conn(0x7facc00fded0 msgr2=0x7facc0107610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:46.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.429+0000 7facc7554700 1 -- 172.21.15.69:0/2081586455 shutdown_connections 2024-04-25T11:25:46.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:46.429+0000 7facc7554700 1 -- 172.21.15.69:0/2081586455 wait complete. 2024-04-25T11:25:47.059 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:46 smithi069 ceph-mon[29188]: pgmap v351: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-25T11:25:47.085 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-25T11:25:47.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:46 smithi152 ceph-mon[32701]: pgmap v351: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-25T11:25:47.329 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:25:47.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:46 smithi146 ceph-mon[32821]: pgmap v351: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-25T11:25:48.192 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:47 smithi069 ceph-mon[29188]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:25:48.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:47 smithi152 ceph-mon[32701]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:25:48.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:47 smithi146 ceph-mon[32821]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:25:49.096 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:48 smithi069 ceph-mon[29188]: pgmap v352: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T11:25:49.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.107+0000 7f2748d39700 1 -- 172.21.15.69:0/1193498729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27441040c0 msgr2=0x7f2744104530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:49.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.107+0000 7f2748d39700 1 --2- 172.21.15.69:0/1193498729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27441040c0 0x7f2744104530 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2734009990 tx=0x7f2734009ca0 comp rx=0 tx=0).stop 2024-04-25T11:25:49.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.108+0000 7f2748d39700 1 -- 172.21.15.69:0/1193498729 shutdown_connections 2024-04-25T11:25:49.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.108+0000 7f2748d39700 1 --2- 172.21.15.69:0/1193498729 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2744104bc0 0x7f274410b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:49.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.108+0000 7f2748d39700 1 --2- 172.21.15.69:0/1193498729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27441036e0 0x7f2744103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:49.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.108+0000 7f2748d39700 1 --2- 172.21.15.69:0/1193498729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27441040c0 0x7f2744104530 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:49.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.108+0000 7f2748d39700 1 -- 172.21.15.69:0/1193498729 >> 172.21.15.69:0/1193498729 conn(0x7f27440fdef0 msgr2=0x7f27441003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:49.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.108+0000 7f2748d39700 1 -- 172.21.15.69:0/1193498729 shutdown_connections 2024-04-25T11:25:49.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.108+0000 7f2748d39700 1 -- 172.21.15.69:0/1193498729 wait complete. 2024-04-25T11:25:49.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.109+0000 7f2748d39700 1 Processor -- start 2024-04-25T11:25:49.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.109+0000 7f2748d39700 1 -- start start 2024-04-25T11:25:49.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.109+0000 7f2748d39700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27441036e0 0x7f2744102dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:49.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.109+0000 7f2748d39700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27441040c0 0x7f274419f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:49.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.109+0000 7f2748d39700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2744104bc0 0x7f27441a1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:49.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.109+0000 7f2748d39700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f274410eb50 con 0x7f27441040c0 2024-04-25T11:25:49.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.109+0000 7f2748d39700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f274410e9d0 con 0x7f2744104bc0 2024-04-25T11:25:49.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.109+0000 7f2748d39700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f274410ecd0 con 0x7f27441036e0 2024-04-25T11:25:49.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.109+0000 7f2741d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27441040c0 0x7f274419f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:49.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.109+0000 7f2741d9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27441040c0 0x7f274419f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43354/0 (socket says 172.21.15.69:43354) 2024-04-25T11:25:49.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.109+0000 7f2741d9b700 1 -- 172.21.15.69:0/331135554 learned_addr learned my addr 172.21.15.69:0/331135554 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:49.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.109+0000 7f274259c700 1 --2- 172.21.15.69:0/331135554 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27441036e0 0x7f2744102dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:49.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.109+0000 7f2742d9d700 1 --2- 172.21.15.69:0/331135554 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2744104bc0 0x7f27441a1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:49.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.109+0000 7f2741d9b700 1 -- 172.21.15.69:0/331135554 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27441036e0 msgr2=0x7f2744102dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:49.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.109+0000 7f2741d9b700 1 --2- 172.21.15.69:0/331135554 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27441036e0 0x7f2744102dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:49.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.109+0000 7f2741d9b700 1 -- 172.21.15.69:0/331135554 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2744104bc0 msgr2=0x7f27441a1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:49.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.109+0000 7f2741d9b700 1 --2- 172.21.15.69:0/331135554 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2744104bc0 0x7f27441a1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:49.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.109+0000 7f2741d9b700 1 -- 172.21.15.69:0/331135554 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27441a23b0 con 0x7f27441040c0 2024-04-25T11:25:49.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.109+0000 7f274259c700 1 --2- 172.21.15.69:0/331135554 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27441036e0 0x7f2744102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:25:49.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.110+0000 7f2741d9b700 1 --2- 172.21.15.69:0/331135554 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27441040c0 0x7f274419f5f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2734005b30 tx=0x7f2734009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:49.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.110+0000 7f27337fe700 1 -- 172.21.15.69:0/331135554 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2734027070 con 0x7f27441040c0 2024-04-25T11:25:49.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.110+0000 7f27337fe700 1 -- 172.21.15.69:0/331135554 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2734004220 con 0x7f27441040c0 2024-04-25T11:25:49.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.110+0000 7f2748d39700 1 -- 172.21.15.69:0/331135554 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27441a2640 con 0x7f27441040c0 2024-04-25T11:25:49.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.110+0000 7f27337fe700 1 -- 172.21.15.69:0/331135554 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2734022da0 con 0x7f27441040c0 2024-04-25T11:25:49.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.110+0000 7f2748d39700 1 -- 172.21.15.69:0/331135554 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27441a2ba0 con 0x7f27441040c0 2024-04-25T11:25:49.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.111+0000 7f2748d39700 1 -- 172.21.15.69:0/331135554 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2744051190 con 0x7f27441040c0 2024-04-25T11:25:49.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.112+0000 7f27337fe700 1 -- 172.21.15.69:0/331135554 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f273402a070 con 0x7f27441040c0 2024-04-25T11:25:49.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.113+0000 7f27337fe700 1 --2- 172.21.15.69:0/331135554 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f272c0afb80 0x7f272c0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:49.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.113+0000 7f274259c700 1 --2- 172.21.15.69:0/331135554 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f272c0afb80 0x7f272c0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:49.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.113+0000 7f27337fe700 1 -- 172.21.15.69:0/331135554 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f27340e07e0 con 0x7f27441040c0 2024-04-25T11:25:49.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.113+0000 7f274259c700 1 --2- 172.21.15.69:0/331135554 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f272c0afb80 0x7f272c0b2090 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f274406bf10 tx=0x7f2738008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:49.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.117+0000 7f27337fe700 1 -- 172.21.15.69:0/331135554 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2734031080 con 0x7f27441040c0 2024-04-25T11:25:49.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:48 smithi152 ceph-mon[32701]: pgmap v352: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T11:25:49.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.325+0000 7f2748d39700 1 -- 172.21.15.69:0/331135554 --> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f2744063ad0 con 0x7f272c0afb80 2024-04-25T11:25:49.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.327+0000 7f27337fe700 1 -- 172.21.15.69:0/331135554 <== mgr.14152 v2:172.21.15.69:6800/4220065409 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f2744063ad0 con 0x7f272c0afb80 2024-04-25T11:25:49.319 INFO:teuthology.orchestra.run.smithi069.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-25T11:25:49.319 INFO:teuthology.orchestra.run.smithi069.stdout:smithi069 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400Y0400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-25T11:25:49.319 INFO:teuthology.orchestra.run.smithi069.stdout:smithi146 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GB400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-25T11:25:49.319 INFO:teuthology.orchestra.run.smithi069.stdout:smithi152 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300JT400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-25T11:25:49.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.329+0000 7f2748d39700 1 -- 172.21.15.69:0/331135554 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f272c0afb80 msgr2=0x7f272c0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:49.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.329+0000 7f2748d39700 1 --2- 172.21.15.69:0/331135554 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f272c0afb80 0x7f272c0b2090 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f274406bf10 tx=0x7f2738008040 comp rx=0 tx=0).stop 2024-04-25T11:25:49.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.329+0000 7f2748d39700 1 -- 172.21.15.69:0/331135554 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27441040c0 msgr2=0x7f274419f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:49.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.329+0000 7f2748d39700 1 --2- 172.21.15.69:0/331135554 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27441040c0 0x7f274419f5f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2734005b30 tx=0x7f2734009f90 comp rx=0 tx=0).stop 2024-04-25T11:25:49.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.329+0000 7f2748d39700 1 -- 172.21.15.69:0/331135554 shutdown_connections 2024-04-25T11:25:49.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.329+0000 7f2748d39700 1 --2- 172.21.15.69:0/331135554 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f272c0afb80 0x7f272c0b2090 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:49.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.329+0000 7f2748d39700 1 --2- 172.21.15.69:0/331135554 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2744104bc0 0x7f27441a1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:49.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.329+0000 7f2748d39700 1 --2- 172.21.15.69:0/331135554 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f27441036e0 0x7f2744102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:49.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.329+0000 7f2748d39700 1 --2- 172.21.15.69:0/331135554 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27441040c0 0x7f274419f5f0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:49.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.329+0000 7f2748d39700 1 -- 172.21.15.69:0/331135554 >> 172.21.15.69:0/331135554 conn(0x7f27440fdef0 msgr2=0x7f2744107690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:49.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.329+0000 7f2748d39700 1 -- 172.21.15.69:0/331135554 shutdown_connections 2024-04-25T11:25:49.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:49.329+0000 7f2748d39700 1 -- 172.21.15.69:0/331135554 wait complete. 2024-04-25T11:25:49.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:48 smithi146 ceph-mon[32821]: pgmap v352: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T11:25:50.028 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T11:25:50.038 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi069.front.sepia.ceph.com 2024-04-25T11:25:50.038 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-25T11:25:50.277 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:25:51.192 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:50 smithi069 ceph-mon[29188]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:25:51.192 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:50 smithi069 ceph-mon[29188]: pgmap v353: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:25:51.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:50 smithi152 ceph-mon[32701]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:25:51.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:50 smithi152 ceph-mon[32701]: pgmap v353: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:25:51.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:50 smithi146 ceph-mon[32821]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:25:51.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:50 smithi146 ceph-mon[32821]: pgmap v353: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:25:52.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.064+0000 7fbea3cdf700 1 -- 172.21.15.69:0/60471614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe9c104bb0 msgr2=0x7fbe9c10b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:52.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.064+0000 7fbea3cdf700 1 --2- 172.21.15.69:0/60471614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe9c104bb0 0x7fbe9c10b460 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fbe9400b0a0 tx=0x7fbe9400b3b0 comp rx=0 tx=0).stop 2024-04-25T11:25:52.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.064+0000 7fbea3cdf700 1 -- 172.21.15.69:0/60471614 shutdown_connections 2024-04-25T11:25:52.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.064+0000 7fbea3cdf700 1 --2- 172.21.15.69:0/60471614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe9c104bb0 0x7fbe9c10b460 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:52.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.064+0000 7fbea3cdf700 1 --2- 172.21.15.69:0/60471614 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe9c1030e0 0x7fbe9c1034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:52.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.064+0000 7fbea3cdf700 1 --2- 172.21.15.69:0/60471614 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe9c104210 0x7fbe9c104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:52.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.064+0000 7fbea3cdf700 1 -- 172.21.15.69:0/60471614 >> 172.21.15.69:0/60471614 conn(0x7fbe9c0fe740 msgr2=0x7fbe9c100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:52.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.064+0000 7fbea3cdf700 1 -- 172.21.15.69:0/60471614 shutdown_connections 2024-04-25T11:25:52.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.064+0000 7fbea3cdf700 1 -- 172.21.15.69:0/60471614 wait complete. 2024-04-25T11:25:52.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.064+0000 7fbea3cdf700 1 Processor -- start 2024-04-25T11:25:52.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.064+0000 7fbea3cdf700 1 -- start start 2024-04-25T11:25:52.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.065+0000 7fbea3cdf700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe9c1030e0 0x7fbe9c06bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:52.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.065+0000 7fbea3cdf700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe9c104210 0x7fbe9c19f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:52.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.065+0000 7fbea3cdf700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe9c104bb0 0x7fbe9c1a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:52.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.065+0000 7fbea3cdf700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9c10ea80 con 0x7fbe9c104bb0 2024-04-25T11:25:52.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.065+0000 7fbea3cdf700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9c10e900 con 0x7fbe9c104210 2024-04-25T11:25:52.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.065+0000 7fbea3cdf700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9c10ec00 con 0x7fbe9c1030e0 2024-04-25T11:25:52.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.065+0000 7fbea227c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe9c104bb0 0x7fbe9c1a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:52.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.065+0000 7fbea227c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe9c104bb0 0x7fbe9c1a1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43368/0 (socket says 172.21.15.69:43368) 2024-04-25T11:25:52.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.065+0000 7fbea227c700 1 -- 172.21.15.69:0/3869012144 learned_addr learned my addr 172.21.15.69:0/3869012144 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:52.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.065+0000 7fbea1a7b700 1 --2- 172.21.15.69:0/3869012144 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe9c1030e0 0x7fbe9c06bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:52.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.065+0000 7fbea127a700 1 --2- 172.21.15.69:0/3869012144 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe9c104210 0x7fbe9c19f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:52.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.065+0000 7fbea227c700 1 -- 172.21.15.69:0/3869012144 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe9c1030e0 msgr2=0x7fbe9c06bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:52.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.065+0000 7fbea227c700 1 --2- 172.21.15.69:0/3869012144 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe9c1030e0 0x7fbe9c06bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:52.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.065+0000 7fbea227c700 1 -- 172.21.15.69:0/3869012144 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe9c104210 msgr2=0x7fbe9c19f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:52.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.065+0000 7fbea227c700 1 --2- 172.21.15.69:0/3869012144 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe9c104210 0x7fbe9c19f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:52.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.065+0000 7fbea227c700 1 -- 172.21.15.69:0/3869012144 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe9c1a2230 con 0x7fbe9c104bb0 2024-04-25T11:25:52.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.065+0000 7fbea127a700 1 --2- 172.21.15.69:0/3869012144 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe9c104210 0x7fbe9c19f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:25:52.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.065+0000 7fbea227c700 1 --2- 172.21.15.69:0/3869012144 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe9c104bb0 0x7fbe9c1a1a00 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fbe94009010 tx=0x7fbe94009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:52.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.065+0000 7fbe92ffd700 1 -- 172.21.15.69:0/3869012144 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe94021020 con 0x7fbe9c104bb0 2024-04-25T11:25:52.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.065+0000 7fbea3cdf700 1 -- 172.21.15.69:0/3869012144 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe9c199200 con 0x7fbe9c104bb0 2024-04-25T11:25:52.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.065+0000 7fbe92ffd700 1 -- 172.21.15.69:0/3869012144 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbe94007c00 con 0x7fbe9c104bb0 2024-04-25T11:25:52.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.066+0000 7fbe92ffd700 1 -- 172.21.15.69:0/3869012144 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9401b7b0 con 0x7fbe9c104bb0 2024-04-25T11:25:52.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.066+0000 7fbea3cdf700 1 -- 172.21.15.69:0/3869012144 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe9c1997b0 con 0x7fbe9c104bb0 2024-04-25T11:25:52.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.067+0000 7fbea3cdf700 1 -- 172.21.15.69:0/3869012144 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe9c051190 con 0x7fbe9c104bb0 2024-04-25T11:25:52.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.068+0000 7fbe92ffd700 1 -- 172.21.15.69:0/3869012144 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fbe94024020 con 0x7fbe9c104bb0 2024-04-25T11:25:52.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.068+0000 7fbe92ffd700 1 --2- 172.21.15.69:0/3869012144 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fbe880afb80 0x7fbe880b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:52.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.069+0000 7fbea1a7b700 1 --2- 172.21.15.69:0/3869012144 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fbe880afb80 0x7fbe880b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:52.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.069+0000 7fbe92ffd700 1 -- 172.21.15.69:0/3869012144 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7fbe9400f020 con 0x7fbe9c104bb0 2024-04-25T11:25:52.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.069+0000 7fbea1a7b700 1 --2- 172.21.15.69:0/3869012144 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fbe880afb80 0x7fbe880b2090 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c104110 tx=0x7fbe8c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:52.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.073+0000 7fbe92ffd700 1 -- 172.21.15.69:0/3869012144 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbe940a81b0 con 0x7fbe9c104bb0 2024-04-25T11:25:52.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.318+0000 7fbea3cdf700 1 -- 172.21.15.69:0/3869012144 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fbe9c02e8f0 con 0x7fbe9c104bb0 2024-04-25T11:25:52.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.318+0000 7fbe92ffd700 1 -- 172.21.15.69:0/3869012144 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+3100 (secure 0 0 0) 0x7fbe940a7ab0 con 0x7fbe9c104bb0 2024-04-25T11:25:52.311 INFO:teuthology.orchestra.run.smithi069.stdout:e28 2024-04-25T11:25:52.311 INFO:teuthology.orchestra.run.smithi069.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T11:25:52.311 INFO:teuthology.orchestra.run.smithi069.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T11:25:52.311 INFO:teuthology.orchestra.run.smithi069.stdout:legacy client fscid: 1 2024-04-25T11:25:52.311 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:25:52.312 INFO:teuthology.orchestra.run.smithi069.stdout:Filesystem 'cephfs' (1) 2024-04-25T11:25:52.312 INFO:teuthology.orchestra.run.smithi069.stdout:fs_name cephfs 2024-04-25T11:25:52.312 INFO:teuthology.orchestra.run.smithi069.stdout:epoch 28 2024-04-25T11:25:52.312 INFO:teuthology.orchestra.run.smithi069.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-25T11:25:52.312 INFO:teuthology.orchestra.run.smithi069.stdout:created 2024-04-25T11:23:40.984930+0000 2024-04-25T11:25:52.312 INFO:teuthology.orchestra.run.smithi069.stdout:modified 2024-04-25T11:24:09.061188+0000 2024-04-25T11:25:52.312 INFO:teuthology.orchestra.run.smithi069.stdout:tableserver 0 2024-04-25T11:25:52.312 INFO:teuthology.orchestra.run.smithi069.stdout:root 0 2024-04-25T11:25:52.312 INFO:teuthology.orchestra.run.smithi069.stdout:session_timeout 300 2024-04-25T11:25:52.312 INFO:teuthology.orchestra.run.smithi069.stdout:session_autoclose 300 2024-04-25T11:25:52.312 INFO:teuthology.orchestra.run.smithi069.stdout:max_file_size 1099511627776 2024-04-25T11:25:52.312 INFO:teuthology.orchestra.run.smithi069.stdout:max_xattr_size 65536 2024-04-25T11:25:52.312 INFO:teuthology.orchestra.run.smithi069.stdout:required_client_features {} 2024-04-25T11:25:52.313 INFO:teuthology.orchestra.run.smithi069.stdout:last_failure 0 2024-04-25T11:25:52.313 INFO:teuthology.orchestra.run.smithi069.stdout:last_failure_osd_epoch 0 2024-04-25T11:25:52.313 INFO:teuthology.orchestra.run.smithi069.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T11:25:52.313 INFO:teuthology.orchestra.run.smithi069.stdout:max_mds 5 2024-04-25T11:25:52.313 INFO:teuthology.orchestra.run.smithi069.stdout:in 0,1,2,3,4 2024-04-25T11:25:52.313 INFO:teuthology.orchestra.run.smithi069.stdout:up {0=24515,1=24485,2=24475,3=24458,4=14625} 2024-04-25T11:25:52.313 INFO:teuthology.orchestra.run.smithi069.stdout:failed 2024-04-25T11:25:52.313 INFO:teuthology.orchestra.run.smithi069.stdout:damaged 2024-04-25T11:25:52.313 INFO:teuthology.orchestra.run.smithi069.stdout:stopped 2024-04-25T11:25:52.313 INFO:teuthology.orchestra.run.smithi069.stdout:data_pools [3,4] 2024-04-25T11:25:52.313 INFO:teuthology.orchestra.run.smithi069.stdout:metadata_pool 2 2024-04-25T11:25:52.313 INFO:teuthology.orchestra.run.smithi069.stdout:inline_data disabled 2024-04-25T11:25:52.313 INFO:teuthology.orchestra.run.smithi069.stdout:balancer 2024-04-25T11:25:52.313 INFO:teuthology.orchestra.run.smithi069.stdout:bal_rank_mask -1 2024-04-25T11:25:52.314 INFO:teuthology.orchestra.run.smithi069.stdout:standby_count_wanted 0 2024-04-25T11:25:52.314 INFO:teuthology.orchestra.run.smithi069.stdout:qdb_cluster leader: 24515 members: 24485,24515,24475,24458,14625 2024-04-25T11:25:52.314 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.b{0:24515} state up:active seq 3 addr [v2:172.21.15.146:6838/2013283448,v1:172.21.15.146:6839/2013283448] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T11:25:52.314 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.c{0:24451} state up:standby-replay seq 1 addr [v2:172.21.15.152:6836/563274982,v1:172.21.15.152:6837/563274982] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T11:25:52.314 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.h{1:24485} state up:active seq 12 addr [v2:172.21.15.146:6836/2730317134,v1:172.21.15.146:6837/2730317134] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T11:25:52.314 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.f{1:24439} state up:standby-replay seq 1 addr [v2:172.21.15.152:6834/3188843362,v1:172.21.15.152:6835/3188843362] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T11:25:52.314 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.i{2:24475} state up:active seq 12 addr [v2:172.21.15.152:6838/3008615657,v1:172.21.15.152:6839/3008615657] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T11:25:52.314 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.k{2:24425} state up:standby-replay seq 1 addr [v2:172.21.15.146:6832/4267885323,v1:172.21.15.146:6833/4267885323] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T11:25:52.314 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.e{3:24458} state up:active seq 15 addr [v2:172.21.15.146:6834/1876533699,v1:172.21.15.146:6835/1876533699] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T11:25:52.314 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.l{3:24391} state up:standby-replay seq 1 addr [v2:172.21.15.152:6832/4197482473,v1:172.21.15.152:6833/4197482473] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T11:25:52.314 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.a{4:14625} state up:active seq 11 addr [v2:172.21.15.69:6840/2060225181,v1:172.21.15.69:6841/2060225181] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T11:25:52.314 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.g{4:14613} state up:standby-replay seq 1 addr [v2:172.21.15.69:6838/966316001,v1:172.21.15.69:6839/966316001] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T11:25:52.315 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:25:52.315 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:25:52.315 INFO:teuthology.orchestra.run.smithi069.stdout:Standby daemons: 2024-04-25T11:25:52.315 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:25:52.315 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.j{-1:14532} state up:standby seq 1 addr [v2:172.21.15.69:6834/3399424094,v1:172.21.15.69:6835/3399424094] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T11:25:52.315 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.d{-1:14550} state up:standby seq 1 addr [v2:172.21.15.69:6836/1463217771,v1:172.21.15.69:6837/1463217771] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T11:25:52.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.321+0000 7fbea3cdf700 1 -- 172.21.15.69:0/3869012144 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fbe880afb80 msgr2=0x7fbe880b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:52.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.321+0000 7fbea3cdf700 1 --2- 172.21.15.69:0/3869012144 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fbe880afb80 0x7fbe880b2090 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c104110 tx=0x7fbe8c00a380 comp rx=0 tx=0).stop 2024-04-25T11:25:52.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.321+0000 7fbea3cdf700 1 -- 172.21.15.69:0/3869012144 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe9c104bb0 msgr2=0x7fbe9c1a1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:52.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.321+0000 7fbea3cdf700 1 --2- 172.21.15.69:0/3869012144 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe9c104bb0 0x7fbe9c1a1a00 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fbe94009010 tx=0x7fbe94009bf0 comp rx=0 tx=0).stop 2024-04-25T11:25:52.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.322+0000 7fbea3cdf700 1 -- 172.21.15.69:0/3869012144 shutdown_connections 2024-04-25T11:25:52.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.322+0000 7fbea3cdf700 1 --2- 172.21.15.69:0/3869012144 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fbe880afb80 0x7fbe880b2090 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:52.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.322+0000 7fbea3cdf700 1 --2- 172.21.15.69:0/3869012144 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe9c104bb0 0x7fbe9c1a1a00 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:52.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.322+0000 7fbea3cdf700 1 --2- 172.21.15.69:0/3869012144 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe9c1030e0 0x7fbe9c06bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:52.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.322+0000 7fbea3cdf700 1 --2- 172.21.15.69:0/3869012144 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe9c104210 0x7fbe9c19f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:52.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.322+0000 7fbea3cdf700 1 -- 172.21.15.69:0/3869012144 >> 172.21.15.69:0/3869012144 conn(0x7fbe9c0fe740 msgr2=0x7fbe9c107440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:52.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.322+0000 7fbea3cdf700 1 -- 172.21.15.69:0/3869012144 shutdown_connections 2024-04-25T11:25:52.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:52.322+0000 7fbea3cdf700 1 -- 172.21.15.69:0/3869012144 wait complete. 2024-04-25T11:25:52.317 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 28 2024-04-25T11:25:53.051 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:52 smithi069 ceph-mon[29188]: pgmap v354: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:25:53.052 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:52 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/3869012144' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T11:25:53.081 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-25T11:25:53.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:52 smithi152 ceph-mon[32701]: pgmap v354: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:25:53.230 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:52 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/3869012144' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T11:25:53.320 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:25:53.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:52 smithi146 ceph-mon[32821]: pgmap v354: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:25:53.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:52 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/3869012144' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T11:25:54.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:54.985+0000 7fcf9bd36700 1 -- 172.21.15.69:0/2112730768 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf94104be0 msgr2=0x7fcf9410b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:54.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:54.985+0000 7fcf9bd36700 1 --2- 172.21.15.69:0/2112730768 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf94104be0 0x7fcf9410b670 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c00b0a0 tx=0x7fcf8c00b3b0 comp rx=0 tx=0).stop 2024-04-25T11:25:54.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:54.985+0000 7fcf9bd36700 1 -- 172.21.15.69:0/2112730768 shutdown_connections 2024-04-25T11:25:54.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:54.985+0000 7fcf9bd36700 1 --2- 172.21.15.69:0/2112730768 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf94104be0 0x7fcf9410b670 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:54.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:54.985+0000 7fcf9bd36700 1 --2- 172.21.15.69:0/2112730768 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf941036a0 0x7fcf94103ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:54.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:54.985+0000 7fcf9bd36700 1 --2- 172.21.15.69:0/2112730768 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf94104080 0x7fcf94104550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:54.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:54.985+0000 7fcf9bd36700 1 -- 172.21.15.69:0/2112730768 >> 172.21.15.69:0/2112730768 conn(0x7fcf940fdef0 msgr2=0x7fcf941003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:54.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:54.985+0000 7fcf9bd36700 1 -- 172.21.15.69:0/2112730768 shutdown_connections 2024-04-25T11:25:54.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:54.985+0000 7fcf9bd36700 1 -- 172.21.15.69:0/2112730768 wait complete. 2024-04-25T11:25:54.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:54.986+0000 7fcf9bd36700 1 Processor -- start 2024-04-25T11:25:54.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:54.986+0000 7fcf9bd36700 1 -- start start 2024-04-25T11:25:54.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:54.986+0000 7fcf9bd36700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf941036a0 0x7fcf94199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:54.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:54.986+0000 7fcf9bd36700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf94104080 0x7fcf94199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:54.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:54.986+0000 7fcf9bd36700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf94104be0 0x7fcf941a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:54.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:54.986+0000 7fcf9bd36700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9410ea50 con 0x7fcf94104080 2024-04-25T11:25:54.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:54.986+0000 7fcf9bd36700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9410e8d0 con 0x7fcf941036a0 2024-04-25T11:25:54.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:54.986+0000 7fcf9bd36700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9410ebd0 con 0x7fcf94104be0 2024-04-25T11:25:54.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:54.986+0000 7fcf992d1700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf94104080 0x7fcf94199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:54.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:54.986+0000 7fcf992d1700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf94104080 0x7fcf94199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56878/0 (socket says 172.21.15.69:56878) 2024-04-25T11:25:54.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:54.986+0000 7fcf992d1700 1 -- 172.21.15.69:0/2335443210 learned_addr learned my addr 172.21.15.69:0/2335443210 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:54.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:54.986+0000 7fcf99ad2700 1 --2- 172.21.15.69:0/2335443210 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf941036a0 0x7fcf94199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:54.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:54.987+0000 7fcf9a2d3700 1 --2- 172.21.15.69:0/2335443210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf94104be0 0x7fcf941a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:54.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:54.987+0000 7fcf992d1700 1 -- 172.21.15.69:0/2335443210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf94104be0 msgr2=0x7fcf941a0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:54.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:54.987+0000 7fcf992d1700 1 --2- 172.21.15.69:0/2335443210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf94104be0 0x7fcf941a0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:54.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:54.987+0000 7fcf992d1700 1 -- 172.21.15.69:0/2335443210 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf941036a0 msgr2=0x7fcf94199530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:54.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:54.987+0000 7fcf992d1700 1 --2- 172.21.15.69:0/2335443210 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf941036a0 0x7fcf94199530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:54.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:54.987+0000 7fcf992d1700 1 -- 172.21.15.69:0/2335443210 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf941a1000 con 0x7fcf94104080 2024-04-25T11:25:54.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:54.987+0000 7fcf992d1700 1 --2- 172.21.15.69:0/2335443210 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf94104080 0x7fcf94199a70 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fcf84007b50 tx=0x7fcf84007e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:54.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:54.987+0000 7fcf8affd700 1 -- 172.21.15.69:0/2335443210 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf84019020 con 0x7fcf94104080 2024-04-25T11:25:54.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:54.987+0000 7fcf9bd36700 1 -- 172.21.15.69:0/2335443210 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf94102360 con 0x7fcf94104080 2024-04-25T11:25:54.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:54.987+0000 7fcf8affd700 1 -- 172.21.15.69:0/2335443210 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf84004cc0 con 0x7fcf94104080 2024-04-25T11:25:54.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:54.987+0000 7fcf8affd700 1 -- 172.21.15.69:0/2335443210 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf8400e500 con 0x7fcf94104080 2024-04-25T11:25:54.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:54.987+0000 7fcf9bd36700 1 -- 172.21.15.69:0/2335443210 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf941028a0 con 0x7fcf94104080 2024-04-25T11:25:54.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:54.989+0000 7fcf9bd36700 1 -- 172.21.15.69:0/2335443210 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf94051190 con 0x7fcf94104080 2024-04-25T11:25:54.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:54.990+0000 7fcf8affd700 1 -- 172.21.15.69:0/2335443210 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fcf8400e6a0 con 0x7fcf94104080 2024-04-25T11:25:54.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:54.990+0000 7fcf8affd700 1 --2- 172.21.15.69:0/2335443210 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fcf800afb80 0x7fcf800b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:54.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:54.991+0000 7fcf99ad2700 1 --2- 172.21.15.69:0/2335443210 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fcf800afb80 0x7fcf800b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:54.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:54.991+0000 7fcf8affd700 1 -- 172.21.15.69:0/2335443210 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7fcf840e0400 con 0x7fcf94104080 2024-04-25T11:25:54.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:54.991+0000 7fcf99ad2700 1 --2- 172.21.15.69:0/2335443210 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fcf800afb80 0x7fcf800b2090 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcf9406bf30 tx=0x7fcf9000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:54.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:54.995+0000 7fcf8affd700 1 -- 172.21.15.69:0/2335443210 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcf84023020 con 0x7fcf94104080 2024-04-25T11:25:55.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:55.195+0000 7fcf9bd36700 1 -- 172.21.15.69:0/2335443210 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fcf94109ef0 con 0x7fcf94104080 2024-04-25T11:25:55.188 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:54 smithi069 ceph-mon[29188]: pgmap v355: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:25:55.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:55.196+0000 7fcf8affd700 1 -- 172.21.15.69:0/2335443210 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v84) ==== 56+0+5711 (secure 0 0 0) 0x7fcf840a8ca0 con 0x7fcf94104080 2024-04-25T11:25:55.189 INFO:teuthology.orchestra.run.smithi069.stdout:epoch 84 2024-04-25T11:25:55.189 INFO:teuthology.orchestra.run.smithi069.stdout:fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 2024-04-25T11:25:55.189 INFO:teuthology.orchestra.run.smithi069.stdout:created 2024-04-25T11:15:37.965281+0000 2024-04-25T11:25:55.189 INFO:teuthology.orchestra.run.smithi069.stdout:modified 2024-04-25T11:25:33.888184+0000 2024-04-25T11:25:55.189 INFO:teuthology.orchestra.run.smithi069.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-25T11:25:55.189 INFO:teuthology.orchestra.run.smithi069.stdout:crush_version 27 2024-04-25T11:25:55.189 INFO:teuthology.orchestra.run.smithi069.stdout:full_ratio 0.95 2024-04-25T11:25:55.189 INFO:teuthology.orchestra.run.smithi069.stdout:backfillfull_ratio 0.9 2024-04-25T11:25:55.189 INFO:teuthology.orchestra.run.smithi069.stdout:nearfull_ratio 0.85 2024-04-25T11:25:55.189 INFO:teuthology.orchestra.run.smithi069.stdout:require_min_compat_client luminous 2024-04-25T11:25:55.190 INFO:teuthology.orchestra.run.smithi069.stdout:min_compat_client luminous 2024-04-25T11:25:55.190 INFO:teuthology.orchestra.run.smithi069.stdout:require_osd_release squid 2024-04-25T11:25:55.190 INFO:teuthology.orchestra.run.smithi069.stdout:stretch_mode_enabled false 2024-04-25T11:25:55.190 INFO:teuthology.orchestra.run.smithi069.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-25T11:25:55.190 INFO:teuthology.orchestra.run.smithi069.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 74 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-25T11:25:55.190 INFO:teuthology.orchestra.run.smithi069.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 75 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-04-25T11:25:55.190 INFO:teuthology.orchestra.run.smithi069.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 80 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-04-25T11:25:55.190 INFO:teuthology.orchestra.run.smithi069.stdout:max_osd 12 2024-04-25T11:25:55.190 INFO:teuthology.orchestra.run.smithi069.stdout:osd.0 up in weight 1 up_from 8 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.69:6802/3756025235,v1:172.21.15.69:6803/3756025235] [v2:172.21.15.69:6804/3756025235,v1:172.21.15.69:6805/3756025235] exists,up a6a0d179-4998-4754-bf82-2ea3dac54e12 2024-04-25T11:25:55.190 INFO:teuthology.orchestra.run.smithi069.stdout:osd.1 up in weight 1 up_from 13 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.146:6800/3166305213,v1:172.21.15.146:6801/3166305213] [v2:172.21.15.146:6802/3166305213,v1:172.21.15.146:6803/3166305213] exists,up c0c66930-4233-493f-b2bb-99c473994b59 2024-04-25T11:25:55.190 INFO:teuthology.orchestra.run.smithi069.stdout:osd.2 up in weight 1 up_from 18 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.152:6800/1474319175,v1:172.21.15.152:6801/1474319175] [v2:172.21.15.152:6802/1474319175,v1:172.21.15.152:6803/1474319175] exists,up d4ef735b-9786-4cb9-83ef-7c3f0691a729 2024-04-25T11:25:55.190 INFO:teuthology.orchestra.run.smithi069.stdout:osd.3 up in weight 1 up_from 25 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.69:6810/1809071296,v1:172.21.15.69:6811/1809071296] [v2:172.21.15.69:6812/1809071296,v1:172.21.15.69:6813/1809071296] exists,up 0ab85fb8-5bbb-4ffc-bddf-cae7644a64ab 2024-04-25T11:25:55.190 INFO:teuthology.orchestra.run.smithi069.stdout:osd.4 up in weight 1 up_from 31 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.146:6808/1898520972,v1:172.21.15.146:6809/1898520972] [v2:172.21.15.146:6810/1898520972,v1:172.21.15.146:6811/1898520972] exists,up 273e0592-bbbc-4188-8eeb-46a3879cf632 2024-04-25T11:25:55.191 INFO:teuthology.orchestra.run.smithi069.stdout:osd.5 up in weight 1 up_from 36 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.152:6808/1996555108,v1:172.21.15.152:6809/1996555108] [v2:172.21.15.152:6810/1996555108,v1:172.21.15.152:6811/1996555108] exists,up bbeae945-9fa2-48f9-a1d6-8bf18dcaef19 2024-04-25T11:25:55.191 INFO:teuthology.orchestra.run.smithi069.stdout:osd.6 up in weight 1 up_from 42 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.69:6818/491215762,v1:172.21.15.69:6819/491215762] [v2:172.21.15.69:6820/491215762,v1:172.21.15.69:6821/491215762] exists,up dbb61253-30bc-418e-bd75-118c80bbf931 2024-04-25T11:25:55.191 INFO:teuthology.orchestra.run.smithi069.stdout:osd.7 up in weight 1 up_from 47 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.146:6816/2330294690,v1:172.21.15.146:6817/2330294690] [v2:172.21.15.146:6818/2330294690,v1:172.21.15.146:6819/2330294690] exists,up 75f23854-e283-4e3e-94cf-7b4079d83876 2024-04-25T11:25:55.191 INFO:teuthology.orchestra.run.smithi069.stdout:osd.8 up in weight 1 up_from 53 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.152:6816/950669381,v1:172.21.15.152:6817/950669381] [v2:172.21.15.152:6818/950669381,v1:172.21.15.152:6819/950669381] exists,up 1aa1060c-0c14-4abd-be56-5ebfeb78759c 2024-04-25T11:25:55.191 INFO:teuthology.orchestra.run.smithi069.stdout:osd.9 up in weight 1 up_from 59 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.69:6826/2414186758,v1:172.21.15.69:6827/2414186758] [v2:172.21.15.69:6828/2414186758,v1:172.21.15.69:6829/2414186758] exists,up 3f4bf800-4090-4138-9bb3-d001f13f450c 2024-04-25T11:25:55.191 INFO:teuthology.orchestra.run.smithi069.stdout:osd.10 up in weight 1 up_from 64 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.146:6824/707256400,v1:172.21.15.146:6825/707256400] [v2:172.21.15.146:6826/707256400,v1:172.21.15.146:6827/707256400] exists,up 19e967e5-51ce-4506-80c9-0a95441baf99 2024-04-25T11:25:55.191 INFO:teuthology.orchestra.run.smithi069.stdout:osd.11 up in weight 1 up_from 70 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.152:6824/2934336833,v1:172.21.15.152:6825/2934336833] [v2:172.21.15.152:6826/2934336833,v1:172.21.15.152:6827/2934336833] exists,up 7788c2de-6930-4808-be0f-f1fd1ae0cb4b 2024-04-25T11:25:55.191 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 2.e [2,1] 2024-04-25T11:25:55.191 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 2.10 [2,4] 2024-04-25T11:25:55.191 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 2.1a [8,4] 2024-04-25T11:25:55.191 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 2.1b [5,11] 2024-04-25T11:25:55.191 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 2.1c [8,11] 2024-04-25T11:25:55.191 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 3.1 [4,9] 2024-04-25T11:25:55.191 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 3.7 [3,9] 2024-04-25T11:25:55.192 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 3.21 [4,8,5,9] 2024-04-25T11:25:55.192 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 3.23 [4,7] 2024-04-25T11:25:55.192 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 3.28 [5,6] 2024-04-25T11:25:55.192 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 4.9 [5,8] 2024-04-25T11:25:55.192 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 4.b [9,10] 2024-04-25T11:25:55.192 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 4.13 [9,8] 2024-04-25T11:25:55.192 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 4.1f [9,8] 2024-04-25T11:25:55.192 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:0/1372727546 expires 2024-04-26T11:16:31.688159+0000 2024-04-25T11:25:55.192 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:6801/1097124842 expires 2024-04-26T11:16:31.688159+0000 2024-04-25T11:25:55.192 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:6800/1508198215 expires 2024-04-26T11:16:04.860792+0000 2024-04-25T11:25:55.192 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:0/3103465990 expires 2024-04-26T11:16:04.860792+0000 2024-04-25T11:25:55.192 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:0/3540132960 expires 2024-04-26T11:16:04.860792+0000 2024-04-25T11:25:55.192 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:0/3899912190 expires 2024-04-26T11:16:31.688159+0000 2024-04-25T11:25:55.192 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:6800/1097124842 expires 2024-04-26T11:16:31.688159+0000 2024-04-25T11:25:55.193 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:0/1199907202 expires 2024-04-26T11:16:04.860792+0000 2024-04-25T11:25:55.193 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:6801/1508198215 expires 2024-04-26T11:16:04.860792+0000 2024-04-25T11:25:55.193 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:0/2879102529 expires 2024-04-26T11:16:31.688159+0000 2024-04-25T11:25:55.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:55.198+0000 7fcf9bd36700 1 -- 172.21.15.69:0/2335443210 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fcf800afb80 msgr2=0x7fcf800b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:55.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:55.198+0000 7fcf9bd36700 1 --2- 172.21.15.69:0/2335443210 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fcf800afb80 0x7fcf800b2090 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcf9406bf30 tx=0x7fcf9000a380 comp rx=0 tx=0).stop 2024-04-25T11:25:55.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:55.198+0000 7fcf9bd36700 1 -- 172.21.15.69:0/2335443210 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf94104080 msgr2=0x7fcf94199a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:55.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:55.198+0000 7fcf9bd36700 1 --2- 172.21.15.69:0/2335443210 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf94104080 0x7fcf94199a70 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fcf84007b50 tx=0x7fcf84007e60 comp rx=0 tx=0).stop 2024-04-25T11:25:55.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:55.198+0000 7fcf9bd36700 1 -- 172.21.15.69:0/2335443210 shutdown_connections 2024-04-25T11:25:55.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:55.198+0000 7fcf9bd36700 1 --2- 172.21.15.69:0/2335443210 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fcf800afb80 0x7fcf800b2090 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:55.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:55.198+0000 7fcf9bd36700 1 --2- 172.21.15.69:0/2335443210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf94104be0 0x7fcf941a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:55.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:55.198+0000 7fcf9bd36700 1 --2- 172.21.15.69:0/2335443210 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf941036a0 0x7fcf94199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:55.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:55.198+0000 7fcf9bd36700 1 --2- 172.21.15.69:0/2335443210 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf94104080 0x7fcf94199a70 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:55.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:55.198+0000 7fcf9bd36700 1 -- 172.21.15.69:0/2335443210 >> 172.21.15.69:0/2335443210 conn(0x7fcf940fdef0 msgr2=0x7fcf94107650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:55.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:55.199+0000 7fcf9bd36700 1 -- 172.21.15.69:0/2335443210 shutdown_connections 2024-04-25T11:25:55.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:55.199+0000 7fcf9bd36700 1 -- 172.21.15.69:0/2335443210 wait complete. 2024-04-25T11:25:55.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:54 smithi152 ceph-mon[32701]: pgmap v355: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:25:55.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:54 smithi146 ceph-mon[32821]: pgmap v355: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:25:55.939 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-25T11:25:55.952 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:25:56.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.069+0000 7f170eb0d640 1 -- 172.21.15.69:0/1680619490 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1708103b00 msgr2=0x7f1708104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:56.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.069+0000 7f170eb0d640 1 --2- 172.21.15.69:0/1680619490 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1708103b00 0x7f1708104800 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f16fc009900 tx=0x7f16fc02a410 comp rx=0 tx=0).stop 2024-04-25T11:25:56.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.069+0000 7f170eb0d640 1 -- 172.21.15.69:0/1680619490 shutdown_connections 2024-04-25T11:25:56.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.069+0000 7f170eb0d640 1 --2- 172.21.15.69:0/1680619490 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1708109a40 0x7f170810be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:56.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.069+0000 7f170eb0d640 1 --2- 172.21.15.69:0/1680619490 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1708104d40 0x7f1708105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:56.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.069+0000 7f170eb0d640 1 --2- 172.21.15.69:0/1680619490 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1708103b00 0x7f1708104800 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:56.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.069+0000 7f170eb0d640 1 -- 172.21.15.69:0/1680619490 >> 172.21.15.69:0/1680619490 conn(0x7f17080fed10 msgr2=0x7f17081011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:56.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.069+0000 7f170eb0d640 1 -- 172.21.15.69:0/1680619490 shutdown_connections 2024-04-25T11:25:56.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.069+0000 7f170eb0d640 1 -- 172.21.15.69:0/1680619490 wait complete. 2024-04-25T11:25:56.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.070+0000 7f170eb0d640 1 Processor -- start 2024-04-25T11:25:56.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.070+0000 7f170eb0d640 1 -- start start 2024-04-25T11:25:56.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.070+0000 7f170eb0d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1708103b00 0x7f170819a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:56.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.070+0000 7f170eb0d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1708104d40 0x7f170819a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:56.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.070+0000 7f170eb0d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1708109a40 0x7f17081a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:56.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.070+0000 7f170eb0d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f170810f510 con 0x7f1708104d40 2024-04-25T11:25:56.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.070+0000 7f170eb0d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f170810f390 con 0x7f1708103b00 2024-04-25T11:25:56.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.070+0000 7f170eb0d640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f170810f690 con 0x7f1708109a40 2024-04-25T11:25:56.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.070+0000 7f1707fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1708104d40 0x7f170819a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:56.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.070+0000 7f1707fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1708104d40 0x7f170819a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56892/0 (socket says 172.21.15.69:56892) 2024-04-25T11:25:56.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.070+0000 7f1707fff640 1 -- 172.21.15.69:0/4164969738 learned_addr learned my addr 172.21.15.69:0/4164969738 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:56.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.070+0000 7f170c8a9640 1 --2- 172.21.15.69:0/4164969738 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1708103b00 0x7f170819a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:56.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.070+0000 7f170d0aa640 1 --2- 172.21.15.69:0/4164969738 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1708109a40 0x7f17081a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:56.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.070+0000 7f1707fff640 1 -- 172.21.15.69:0/4164969738 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1708109a40 msgr2=0x7f17081a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:56.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.070+0000 7f1707fff640 1 --2- 172.21.15.69:0/4164969738 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1708109a40 0x7f17081a1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:56.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.070+0000 7f1707fff640 1 -- 172.21.15.69:0/4164969738 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1708103b00 msgr2=0x7f170819a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:56.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.070+0000 7f1707fff640 1 --2- 172.21.15.69:0/4164969738 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1708103b00 0x7f170819a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:56.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.070+0000 7f1707fff640 1 -- 172.21.15.69:0/4164969738 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17081a1ce0 con 0x7f1708104d40 2024-04-25T11:25:56.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.071+0000 7f1707fff640 1 --2- 172.21.15.69:0/4164969738 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1708104d40 0x7f170819a6e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f16f400c890 tx=0x7f16f400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:56.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.071+0000 7f1705ffb640 1 -- 172.21.15.69:0/4164969738 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f4004220 con 0x7f1708104d40 2024-04-25T11:25:56.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.071+0000 7f1705ffb640 1 -- 172.21.15.69:0/4164969738 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f16f4007e40 con 0x7f1708104d40 2024-04-25T11:25:56.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.071+0000 7f1705ffb640 1 -- 172.21.15.69:0/4164969738 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f4004a00 con 0x7f1708104d40 2024-04-25T11:25:56.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.071+0000 7f170eb0d640 1 -- 172.21.15.69:0/4164969738 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17081a1fd0 con 0x7f1708104d40 2024-04-25T11:25:56.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.071+0000 7f170eb0d640 1 -- 172.21.15.69:0/4164969738 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17081a24b0 con 0x7f1708104d40 2024-04-25T11:25:56.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.072+0000 7f170eb0d640 1 -- 172.21.15.69:0/4164969738 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1708108850 con 0x7f1708104d40 2024-04-25T11:25:56.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.073+0000 7f1705ffb640 1 -- 172.21.15.69:0/4164969738 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f16f40079b0 con 0x7f1708104d40 2024-04-25T11:25:56.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.074+0000 7f1705ffb640 1 --2- 172.21.15.69:0/4164969738 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f16f80afaf0 0x7f16f80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:56.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.074+0000 7f170c8a9640 1 --2- 172.21.15.69:0/4164969738 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f16f80afaf0 0x7f16f80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:56.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.074+0000 7f1705ffb640 1 -- 172.21.15.69:0/4164969738 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f16f40ded90 con 0x7f1708104d40 2024-04-25T11:25:56.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.074+0000 7f170c8a9640 1 --2- 172.21.15.69:0/4164969738 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f16f80afaf0 0x7f16f80b2000 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f16fc0057c0 tx=0x7f16fc005720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:56.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.079+0000 7f1705ffb640 1 -- 172.21.15.69:0/4164969738 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f16f4014030 con 0x7f1708104d40 2024-04-25T11:25:56.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:55 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2335443210' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T11:25:56.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:55 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/2335443210' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T11:25:56.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.260+0000 7f170eb0d640 1 -- 172.21.15.69:0/4164969738 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f170802dd70 con 0x7f1708104d40 2024-04-25T11:25:56.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.261+0000 7f1705ffb640 1 -- 172.21.15.69:0/4164969738 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+24862 (secure 0 0 0) 0x7f16f40a75a0 con 0x7f1708104d40 2024-04-25T11:25:56.254 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:25:56.255 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":84,"fsid":"f7ed73dc-02f4-11ef-bc93-c7b262605968","created":"2024-04-25T11:15:37.965281+0000","modified":"2024-04-25T11:25:33.888184+0000","last_up_change":"2024-04-25T11:22:50.691869+0000","last_in_change":"2024-04-25T11:22:30.271959+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:18:57.781263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:23:38.685110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:23:39.943803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T11:23:43.650521+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6a0d179-4998-4754-bf82-2ea3dac54e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6803","nonce":3756025235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6805","nonce":3756025235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6809","nonce":3756025235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6807","nonce":3756025235}]},"public_addr":"172.21.15.69:6803/3756025235","cluster_addr":"172.21.15.69:6805/3756025235","heartbeat_back_addr":"172.21.15.69:6809/3756025235","heartbeat_front_addr":"172.21.15.69:6807/3756025235","state":["exists","up"]},{"osd":1,"uuid":"c0c66930-4233-493f-b2bb-99c473994b59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6801","nonce":3166305213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6803","nonce":3166305213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6807","nonce":3166305213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6805","nonce":3166305213}]},"public_addr":"172.21.15.146:6801/3166305213","cluster_addr":"172.21.15.146:6803/3166305213","heartbeat_back_addr":"172.21.15.146:6807/3166305213","heartbeat_front_addr":"172.21.15.146:6805/3166305213","state":["exists","up"]},{"osd":2,"uuid":"d4ef735b-9786-4cb9-83ef-7c3f0691a729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6801","nonce":1474319175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6803","nonce":1474319175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6807","nonce":1474319175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6805","nonce":1474319175}]},"public_addr":"172.21.15.152:6801/1474319175","cluster_addr":"172.21.15.152:6803/1474319175","heartbeat_back_addr":"172.21.15.152:6807/1474319175","heartbeat_front_addr":"172.21.15.152:6805/1474319175","state":["exists","up"]},{"osd":3,"uuid":"0ab85fb8-5bbb-4ffc-bddf-cae7644a64ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6811","nonce":1809071296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6813","nonce":1809071296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6817","nonce":1809071296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6815","nonce":1809071296}]},"public_addr":"172.21.15.69:6811/1809071296","cluster_addr":"172.21.15.69:6813/1809071296","heartbeat_back_addr":"172.21.15.69:6817/1809071296","heartbeat_front_addr":"172.21.15.69:6815/1809071296","state":["exists","up"]},{"osd":4,"uuid":"273e0592-bbbc-4188-8eeb-46a3879cf632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6809","nonce":1898520972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6811","nonce":1898520972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6815","nonce":1898520972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6813","nonce":1898520972}]},"public_addr":"172.21.15.146:6809/1898520972","cluster_addr":"172.21.15.146:6811/1898520972","heartbeat_back_addr":"172.21.15.146:6815/1898520972","heartbeat_front_addr":"172.21.15.146:6813/1898520972","state":["exists","up"]},{"osd":5,"uuid":"bbeae945-9fa2-48f9-a1d6-8bf18dcaef19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6809","nonce":1996555108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6811","nonce":1996555108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6815","nonce":1996555108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6813","nonce":1996555108}]},"public_addr":"172.21.15.152:6809/1996555108","cluster_addr":"172.21.15.152:6811/1996555108","heartbeat_back_addr":"172.21.15.152:6815/1996555108","heartbeat_front_addr":"172.21.15.152:6813/1996555108","state":["exists","up"]},{"osd":6,"uuid":"dbb61253-30bc-418e-bd75-118c80bbf931","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6819","nonce":491215762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6821","nonce":491215762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6825","nonce":491215762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6823","nonce":491215762}]},"public_addr":"172.21.15.69:6819/491215762","cluster_addr":"172.21.15.69:6821/491215762","heartbeat_back_addr":"172.21.15.69:6825/491215762","heartbeat_front_addr":"172.21.15.69:6823/491215762","state":["exists","up"]},{"osd":7,"uuid":"75f23854-e283-4e3e-94cf-7b4079d83876","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6817","nonce":2330294690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6819","nonce":2330294690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6823","nonce":2330294690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6821","nonce":2330294690}]},"public_addr":"172.21.15.146:6817/2330294690","cluster_addr":"172.21.15.146:6819/2330294690","heartbeat_back_addr":"172.21.15.146:6823/2330294690","heartbeat_front_addr":"172.21.15.146:6821/2330294690","state":["exists","up"]},{"osd":8,"uuid":"1aa1060c-0c14-4abd-be56-5ebfeb78759c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6817","nonce":950669381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6819","nonce":950669381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6823","nonce":950669381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6821","nonce":950669381}]},"public_addr":"172.21.15.152:6817/950669381","cluster_addr":"172.21.15.152:6819/950669381","heartbeat_back_addr":"172.21.15.152:6823/950669381","heartbeat_front_addr":"172.21.15.152:6821/950669381","state":["exists","up"]},{"osd":9,"uuid":"3f4bf800-4090-4138-9bb3-d001f13f450c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6827","nonce":2414186758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6829","nonce":2414186758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6833","nonce":2414186758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6831","nonce":2414186758}]},"public_addr":"172.21.15.69:6827/2414186758","cluster_addr":"172.21.15.69:6829/2414186758","heartbeat_back_addr":"172.21.15.69:6833/2414186758","heartbeat_front_addr":"172.21.15.69:6831/2414186758","state":["exists","up"]},{"osd":10,"uuid":"19e967e5-51ce-4506-80c9-0a95441baf99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6825","nonce":707256400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6827","nonce":707256400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6831","nonce":707256400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6829","nonce":707256400}]},"public_addr":"172.21.15.146:6825/707256400","cluster_addr":"172.21.15.146:6827/707256400","heartbeat_back_addr":"172.21.15.146:6831/707256400","heartbeat_front_addr":"172.21.15.146:6829/707256400","state":["exists","up"]},{"osd":11,"uuid":"7788c2de-6930-4808-be0f-f1fd1ae0cb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6825","nonce":2934336833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6827","nonce":2934336833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6831","nonce":2934336833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6829","nonce":2934336833}]},"public_addr":"172.21.15.152:6825/2934336833","cluster_addr":"172.21.15.152:6827/2934336833","heartbeat_back_addr":"172.21.15.152:6831/2934336833","heartbeat_front_addr":"172.21.15.152:6829/2934336833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:07.497448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:31.227494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:55.027978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:20.713437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:46.214704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:20:11.199983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:20:37.177070+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:02.951240+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:28.963680+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:56.095807+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:22:22.221744+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:22:48.929334+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1b","mappings":[{"from":5,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/1372727546":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6801/1097124842":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6800/1508198215":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3103465990":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3540132960":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3899912190":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6800/1097124842":"2024-04-26T11:16:31.688159+0000","172.21.15.69:0/1199907202":"2024-04-26T11:16:04.860792+0000","172.21.15.69:6801/1508198215":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/2879102529":"2024-04-26T11:16:31.688159+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:25:56.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.263+0000 7f170eb0d640 1 -- 172.21.15.69:0/4164969738 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f16f80afaf0 msgr2=0x7f16f80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:56.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.263+0000 7f170eb0d640 1 --2- 172.21.15.69:0/4164969738 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f16f80afaf0 0x7f16f80b2000 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f16fc0057c0 tx=0x7f16fc005720 comp rx=0 tx=0).stop 2024-04-25T11:25:56.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.263+0000 7f170eb0d640 1 -- 172.21.15.69:0/4164969738 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1708104d40 msgr2=0x7f170819a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:56.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.263+0000 7f170eb0d640 1 --2- 172.21.15.69:0/4164969738 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1708104d40 0x7f170819a6e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f16f400c890 tx=0x7f16f400cd50 comp rx=0 tx=0).stop 2024-04-25T11:25:56.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.263+0000 7f170eb0d640 1 -- 172.21.15.69:0/4164969738 shutdown_connections 2024-04-25T11:25:56.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.263+0000 7f170eb0d640 1 --2- 172.21.15.69:0/4164969738 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f16f80afaf0 0x7f16f80b2000 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:56.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.263+0000 7f170eb0d640 1 --2- 172.21.15.69:0/4164969738 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1708109a40 0x7f17081a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:56.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.263+0000 7f170eb0d640 1 --2- 172.21.15.69:0/4164969738 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1708104d40 0x7f170819a6e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:56.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.263+0000 7f170eb0d640 1 --2- 172.21.15.69:0/4164969738 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1708103b00 0x7f170819a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:56.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.263+0000 7f170eb0d640 1 -- 172.21.15.69:0/4164969738 >> 172.21.15.69:0/4164969738 conn(0x7f17080fed10 msgr2=0x7f1708105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:56.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.264+0000 7f170eb0d640 1 -- 172.21.15.69:0/4164969738 shutdown_connections 2024-04-25T11:25:56.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.264+0000 7f170eb0d640 1 -- 172.21.15.69:0/4164969738 wait complete. 2024-04-25T11:25:56.268 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:18:57.781263+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:23:38.685110+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:23:39.943803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T11:23:43.650521+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T11:25:56.268 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:25:56.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:55 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/2335443210' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T11:25:56.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.386+0000 7f5591270640 1 -- 172.21.15.69:0/2310263040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f558c1050a0 msgr2=0x7f558c1054b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:56.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.386+0000 7f5591270640 1 --2- 172.21.15.69:0/2310263040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f558c1050a0 0x7f558c1054b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f557c009900 tx=0x7f557c02a410 comp rx=0 tx=0).stop 2024-04-25T11:25:56.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.386+0000 7f5591270640 1 -- 172.21.15.69:0/2310263040 shutdown_connections 2024-04-25T11:25:56.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.386+0000 7f5591270640 1 --2- 172.21.15.69:0/2310263040 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f558c102250 0x7f558c10bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:56.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.386+0000 7f5591270640 1 --2- 172.21.15.69:0/2310263040 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f558c101750 0x7f558c101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:56.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.386+0000 7f5591270640 1 --2- 172.21.15.69:0/2310263040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f558c1050a0 0x7f558c1054b0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:56.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.386+0000 7f5591270640 1 -- 172.21.15.69:0/2310263040 >> 172.21.15.69:0/2310263040 conn(0x7f558c0fd350 msgr2=0x7f558c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:56.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.386+0000 7f5591270640 1 -- 172.21.15.69:0/2310263040 shutdown_connections 2024-04-25T11:25:56.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.386+0000 7f5591270640 1 -- 172.21.15.69:0/2310263040 wait complete. 2024-04-25T11:25:56.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.387+0000 7f5591270640 1 Processor -- start 2024-04-25T11:25:56.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.387+0000 7f5591270640 1 -- start start 2024-04-25T11:25:56.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.387+0000 7f5591270640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f558c101750 0x7f558c19a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:56.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.387+0000 7f5591270640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f558c102250 0x7f558c19a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:56.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.387+0000 7f5591270640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f558c1050a0 0x7f558c1a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:56.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.387+0000 7f5591270640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f558c10f590 con 0x7f558c102250 2024-04-25T11:25:56.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.387+0000 7f5591270640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f558c10f410 con 0x7f558c101750 2024-04-25T11:25:56.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.387+0000 7f5591270640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f558c10f710 con 0x7f558c1050a0 2024-04-25T11:25:56.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.387+0000 7f558a7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f558c102250 0x7f558c19a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:56.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.387+0000 7f558a7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f558c102250 0x7f558c19a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56922/0 (socket says 172.21.15.69:56922) 2024-04-25T11:25:56.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.388+0000 7f558a7fc640 1 -- 172.21.15.69:0/1443228516 learned_addr learned my addr 172.21.15.69:0/1443228516 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:56.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.388+0000 7f558affd640 1 --2- 172.21.15.69:0/1443228516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f558c101750 0x7f558c19a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:56.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.388+0000 7f558b7fe640 1 --2- 172.21.15.69:0/1443228516 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f558c1050a0 0x7f558c1a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:56.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.388+0000 7f558a7fc640 1 -- 172.21.15.69:0/1443228516 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f558c1050a0 msgr2=0x7f558c1a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:56.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.388+0000 7f558a7fc640 1 --2- 172.21.15.69:0/1443228516 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f558c1050a0 0x7f558c1a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:56.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.388+0000 7f558a7fc640 1 -- 172.21.15.69:0/1443228516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f558c101750 msgr2=0x7f558c19a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:56.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.388+0000 7f558a7fc640 1 --2- 172.21.15.69:0/1443228516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f558c101750 0x7f558c19a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:56.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.388+0000 7f558a7fc640 1 -- 172.21.15.69:0/1443228516 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f558c1a1df0 con 0x7f558c102250 2024-04-25T11:25:56.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.388+0000 7f558affd640 1 --2- 172.21.15.69:0/1443228516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f558c101750 0x7f558c19a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:25:56.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.388+0000 7f558a7fc640 1 --2- 172.21.15.69:0/1443228516 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f558c102250 0x7f558c19a7f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f558000b4a0 tx=0x7f558000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:56.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.388+0000 7f5573fff640 1 -- 172.21.15.69:0/1443228516 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5580014070 con 0x7f558c102250 2024-04-25T11:25:56.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.388+0000 7f5591270640 1 -- 172.21.15.69:0/1443228516 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f558c1a20e0 con 0x7f558c102250 2024-04-25T11:25:56.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.388+0000 7f5573fff640 1 -- 172.21.15.69:0/1443228516 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5580010aa0 con 0x7f558c102250 2024-04-25T11:25:56.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.388+0000 7f5573fff640 1 -- 172.21.15.69:0/1443228516 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5580017b70 con 0x7f558c102250 2024-04-25T11:25:56.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.388+0000 7f5591270640 1 -- 172.21.15.69:0/1443228516 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f558c1a26b0 con 0x7f558c102250 2024-04-25T11:25:56.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.389+0000 7f5591270640 1 -- 172.21.15.69:0/1443228516 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f558c108a30 con 0x7f558c102250 2024-04-25T11:25:56.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.390+0000 7f5573fff640 1 -- 172.21.15.69:0/1443228516 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f5580017070 con 0x7f558c102250 2024-04-25T11:25:56.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.391+0000 7f5573fff640 1 --2- 172.21.15.69:0/1443228516 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f55740afaf0 0x7f55740b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:56.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.391+0000 7f558affd640 1 --2- 172.21.15.69:0/1443228516 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f55740afaf0 0x7f55740b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:56.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.391+0000 7f5573fff640 1 -- 172.21.15.69:0/1443228516 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f55800de870 con 0x7f558c102250 2024-04-25T11:25:56.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.392+0000 7f558affd640 1 --2- 172.21.15.69:0/1443228516 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f55740afaf0 0x7f55740b2000 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f557c0057c0 tx=0x7f557c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:56.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.395+0000 7f5573fff640 1 -- 172.21.15.69:0/1443228516 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f55800a7080 con 0x7f558c102250 2024-04-25T11:25:56.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.570+0000 7f5591270640 1 -- 172.21.15.69:0/1443228516 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f558c02db50 con 0x7f558c102250 2024-04-25T11:25:56.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.571+0000 7f5573fff640 1 -- 172.21.15.69:0/1443228516 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7f55800af100 con 0x7f558c102250 2024-04-25T11:25:56.563 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-04-25T11:25:56.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.573+0000 7f5591270640 1 -- 172.21.15.69:0/1443228516 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f55740afaf0 msgr2=0x7f55740b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:56.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.573+0000 7f5591270640 1 --2- 172.21.15.69:0/1443228516 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f55740afaf0 0x7f55740b2000 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f557c0057c0 tx=0x7f557c000bc0 comp rx=0 tx=0).stop 2024-04-25T11:25:56.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.573+0000 7f5591270640 1 -- 172.21.15.69:0/1443228516 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f558c102250 msgr2=0x7f558c19a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:56.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.573+0000 7f5591270640 1 --2- 172.21.15.69:0/1443228516 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f558c102250 0x7f558c19a7f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f558000b4a0 tx=0x7f558000b960 comp rx=0 tx=0).stop 2024-04-25T11:25:56.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.573+0000 7f5591270640 1 -- 172.21.15.69:0/1443228516 shutdown_connections 2024-04-25T11:25:56.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.573+0000 7f5591270640 1 --2- 172.21.15.69:0/1443228516 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f55740afaf0 0x7f55740b2000 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:56.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.573+0000 7f5591270640 1 --2- 172.21.15.69:0/1443228516 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f558c1050a0 0x7f558c1a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:56.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.573+0000 7f5591270640 1 --2- 172.21.15.69:0/1443228516 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f558c102250 0x7f558c19a7f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:56.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.573+0000 7f5591270640 1 --2- 172.21.15.69:0/1443228516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f558c101750 0x7f558c19a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:56.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.573+0000 7f5591270640 1 -- 172.21.15.69:0/1443228516 >> 172.21.15.69:0/1443228516 conn(0x7f558c0fd350 msgr2=0x7f558c105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:56.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.573+0000 7f5591270640 1 -- 172.21.15.69:0/1443228516 shutdown_connections 2024-04-25T11:25:56.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.573+0000 7f5591270640 1 -- 172.21.15.69:0/1443228516 wait complete. 2024-04-25T11:25:56.577 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:25:56.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.698+0000 7f759ff1f640 1 -- 172.21.15.69:0/566540849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7598106f40 msgr2=0x7f7598109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:56.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.698+0000 7f759ff1f640 1 --2- 172.21.15.69:0/566540849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7598106f40 0x7f7598109360 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7588009900 tx=0x7f758802a410 comp rx=0 tx=0).stop 2024-04-25T11:25:56.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.698+0000 7f759ff1f640 1 -- 172.21.15.69:0/566540849 shutdown_connections 2024-04-25T11:25:56.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.698+0000 7f759ff1f640 1 --2- 172.21.15.69:0/566540849 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75981098a0 0x7f759810bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:56.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.698+0000 7f759ff1f640 1 --2- 172.21.15.69:0/566540849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7598106f40 0x7f7598109360 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:56.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.698+0000 7f759ff1f640 1 --2- 172.21.15.69:0/566540849 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f75981045e0 0x7f7598106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:56.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.698+0000 7f759ff1f640 1 -- 172.21.15.69:0/566540849 >> 172.21.15.69:0/566540849 conn(0x7f75980fdbe0 msgr2=0x7f7598100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:56.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.698+0000 7f759ff1f640 1 -- 172.21.15.69:0/566540849 shutdown_connections 2024-04-25T11:25:56.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.698+0000 7f759ff1f640 1 -- 172.21.15.69:0/566540849 wait complete. 2024-04-25T11:25:56.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.698+0000 7f759ff1f640 1 Processor -- start 2024-04-25T11:25:56.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.699+0000 7f759ff1f640 1 -- start start 2024-04-25T11:25:56.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.699+0000 7f759ff1f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f75981045e0 0x7f759819a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:56.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.699+0000 7f759ff1f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7598106f40 0x7f759819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:56.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.699+0000 7f759ff1f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75981098a0 0x7f75981a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:56.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.699+0000 7f759ff1f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f759810f4d0 con 0x7f75981098a0 2024-04-25T11:25:56.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.699+0000 7f759ff1f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f759810f350 con 0x7f7598106f40 2024-04-25T11:25:56.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.699+0000 7f759ff1f640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f759810f650 con 0x7f75981045e0 2024-04-25T11:25:56.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.699+0000 7f759e4bc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75981098a0 0x7f75981a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:56.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.699+0000 7f759e4bc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75981098a0 0x7f75981a16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56946/0 (socket says 172.21.15.69:56946) 2024-04-25T11:25:56.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.699+0000 7f759e4bc640 1 -- 172.21.15.69:0/2602021625 learned_addr learned my addr 172.21.15.69:0/2602021625 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:56.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.699+0000 7f759d4ba640 1 --2- 172.21.15.69:0/2602021625 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7598106f40 0x7f759819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:56.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.699+0000 7f759dcbb640 1 --2- 172.21.15.69:0/2602021625 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f75981045e0 0x7f759819a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:56.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.699+0000 7f759e4bc640 1 -- 172.21.15.69:0/2602021625 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f75981045e0 msgr2=0x7f759819a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:56.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.699+0000 7f759e4bc640 1 --2- 172.21.15.69:0/2602021625 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f75981045e0 0x7f759819a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:56.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.699+0000 7f759e4bc640 1 -- 172.21.15.69:0/2602021625 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7598106f40 msgr2=0x7f759819a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:56.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.699+0000 7f759e4bc640 1 --2- 172.21.15.69:0/2602021625 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7598106f40 0x7f759819a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:56.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.699+0000 7f759e4bc640 1 -- 172.21.15.69:0/2602021625 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75981a1d30 con 0x7f75981098a0 2024-04-25T11:25:56.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.700+0000 7f759e4bc640 1 --2- 172.21.15.69:0/2602021625 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75981098a0 0x7f75981a16c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f759000e9d0 tx=0x7f759000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:56.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.700+0000 7f758effd640 1 -- 172.21.15.69:0/2602021625 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759000cd80 con 0x7f75981098a0 2024-04-25T11:25:56.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.700+0000 7f758effd640 1 -- 172.21.15.69:0/2602021625 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7590005020 con 0x7f75981098a0 2024-04-25T11:25:56.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.700+0000 7f759ff1f640 1 -- 172.21.15.69:0/2602021625 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75981a1fc0 con 0x7f75981098a0 2024-04-25T11:25:56.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.700+0000 7f758effd640 1 -- 172.21.15.69:0/2602021625 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75900116d0 con 0x7f75981098a0 2024-04-25T11:25:56.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.700+0000 7f759ff1f640 1 -- 172.21.15.69:0/2602021625 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75981a2430 con 0x7f75981098a0 2024-04-25T11:25:56.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.701+0000 7f759ff1f640 1 -- 172.21.15.69:0/2602021625 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75980ff700 con 0x7f75981098a0 2024-04-25T11:25:56.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.703+0000 7f758effd640 1 -- 172.21.15.69:0/2602021625 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f7590004a40 con 0x7f75981098a0 2024-04-25T11:25:56.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.704+0000 7f758effd640 1 --2- 172.21.15.69:0/2602021625 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f75840afaf0 0x7f75840b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:56.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.704+0000 7f759dcbb640 1 --2- 172.21.15.69:0/2602021625 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f75840afaf0 0x7f75840b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:56.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.704+0000 7f758effd640 1 -- 172.21.15.69:0/2602021625 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f75900df380 con 0x7f75981098a0 2024-04-25T11:25:56.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.704+0000 7f759dcbb640 1 --2- 172.21.15.69:0/2602021625 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f75840afaf0 0x7f75840b2000 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f759806bcd0 tx=0x7f7594005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:56.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.707+0000 7f758effd640 1 -- 172.21.15.69:0/2602021625 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7590016030 con 0x7f75981098a0 2024-04-25T11:25:56.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.886+0000 7f759ff1f640 1 -- 172.21.15.69:0/2602021625 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f759802dce0 con 0x7f75981098a0 2024-04-25T11:25:56.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.887+0000 7f758effd640 1 -- 172.21.15.69:0/2602021625 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) ==== 104+0+11 (secure 0 0 0) 0x7f75900a75e0 con 0x7f75981098a0 2024-04-25T11:25:56.879 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-25T11:25:56.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.889+0000 7f759ff1f640 1 -- 172.21.15.69:0/2602021625 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f75840afaf0 msgr2=0x7f75840b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:56.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.889+0000 7f759ff1f640 1 --2- 172.21.15.69:0/2602021625 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f75840afaf0 0x7f75840b2000 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f759806bcd0 tx=0x7f7594005f90 comp rx=0 tx=0).stop 2024-04-25T11:25:56.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.889+0000 7f759ff1f640 1 -- 172.21.15.69:0/2602021625 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75981098a0 msgr2=0x7f75981a16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:56.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.889+0000 7f759ff1f640 1 --2- 172.21.15.69:0/2602021625 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75981098a0 0x7f75981a16c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f759000e9d0 tx=0x7f759000ee90 comp rx=0 tx=0).stop 2024-04-25T11:25:56.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.889+0000 7f759ff1f640 1 -- 172.21.15.69:0/2602021625 shutdown_connections 2024-04-25T11:25:56.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.889+0000 7f759ff1f640 1 --2- 172.21.15.69:0/2602021625 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f75840afaf0 0x7f75840b2000 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:56.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.889+0000 7f759ff1f640 1 --2- 172.21.15.69:0/2602021625 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75981098a0 0x7f75981a16c0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:56.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.889+0000 7f759ff1f640 1 --2- 172.21.15.69:0/2602021625 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7598106f40 0x7f759819a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:56.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.889+0000 7f759ff1f640 1 --2- 172.21.15.69:0/2602021625 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f75981045e0 0x7f759819a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:56.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.889+0000 7f759ff1f640 1 -- 172.21.15.69:0/2602021625 >> 172.21.15.69:0/2602021625 conn(0x7f75980fdbe0 msgr2=0x7f75981066a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:56.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.889+0000 7f759ff1f640 1 -- 172.21.15.69:0/2602021625 shutdown_connections 2024-04-25T11:25:56.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:56.889+0000 7f759ff1f640 1 -- 172.21.15.69:0/2602021625 wait complete. 2024-04-25T11:25:56.893 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:25:57.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.012+0000 7f6d21f40640 1 -- 172.21.15.69:0/3217620427 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d1c108940 msgr2=0x7f6d1c10ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:57.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.012+0000 7f6d21f40640 1 --2- 172.21.15.69:0/3217620427 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d1c108940 0x7f6d1c10ad60 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f6d14009ca0 tx=0x7f6d1402a8c0 comp rx=0 tx=0).stop 2024-04-25T11:25:57.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.012+0000 7f6d21f40640 1 -- 172.21.15.69:0/3217620427 shutdown_connections 2024-04-25T11:25:57.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.012+0000 7f6d21f40640 1 --2- 172.21.15.69:0/3217620427 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d1c108940 0x7f6d1c10ad60 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:57.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.012+0000 7f6d21f40640 1 --2- 172.21.15.69:0/3217620427 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d1c103c40 0x7f6d1c104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:57.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.012+0000 7f6d21f40640 1 --2- 172.21.15.69:0/3217620427 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d1c102a00 0x7f6d1c103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:57.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.012+0000 7f6d21f40640 1 -- 172.21.15.69:0/3217620427 >> 172.21.15.69:0/3217620427 conn(0x7f6d1c0fdbf0 msgr2=0x7f6d1c1000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:57.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.012+0000 7f6d21f40640 1 -- 172.21.15.69:0/3217620427 shutdown_connections 2024-04-25T11:25:57.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.012+0000 7f6d21f40640 1 -- 172.21.15.69:0/3217620427 wait complete. 2024-04-25T11:25:57.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.012+0000 7f6d21f40640 1 Processor -- start 2024-04-25T11:25:57.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.012+0000 7f6d21f40640 1 -- start start 2024-04-25T11:25:57.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.013+0000 7f6d21f40640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d1c102a00 0x7f6d1c199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:57.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.013+0000 7f6d21f40640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d1c103c40 0x7f6d1c199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:57.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.013+0000 7f6d21f40640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d1c108940 0x7f6d1c1a04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:57.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.013+0000 7f6d21f40640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1c10e410 con 0x7f6d1c108940 2024-04-25T11:25:57.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.013+0000 7f6d21f40640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1c10e290 con 0x7f6d1c102a00 2024-04-25T11:25:57.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.013+0000 7f6d21f40640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1c10e590 con 0x7f6d1c103c40 2024-04-25T11:25:57.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.013+0000 7f6d1bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d1c108940 0x7f6d1c1a04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:57.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.013+0000 7f6d1bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d1c108940 0x7f6d1c1a04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56966/0 (socket says 172.21.15.69:56966) 2024-04-25T11:25:57.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.013+0000 7f6d1bfff640 1 -- 172.21.15.69:0/333757903 learned_addr learned my addr 172.21.15.69:0/333757903 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:57.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.013+0000 7f6d1b7fe640 1 --2- 172.21.15.69:0/333757903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d1c102a00 0x7f6d1c199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:57.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.013+0000 7f6d13fff640 1 --2- 172.21.15.69:0/333757903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d1c103c40 0x7f6d1c199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:57.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.013+0000 7f6d1bfff640 1 -- 172.21.15.69:0/333757903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d1c103c40 msgr2=0x7f6d1c199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:57.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.013+0000 7f6d1bfff640 1 --2- 172.21.15.69:0/333757903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d1c103c40 0x7f6d1c199550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:57.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.013+0000 7f6d1bfff640 1 -- 172.21.15.69:0/333757903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d1c102a00 msgr2=0x7f6d1c199010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:57.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.013+0000 7f6d1bfff640 1 --2- 172.21.15.69:0/333757903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d1c102a00 0x7f6d1c199010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:57.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.013+0000 7f6d1bfff640 1 -- 172.21.15.69:0/333757903 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d1c1a0aa0 con 0x7f6d1c108940 2024-04-25T11:25:57.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.013+0000 7f6d1b7fe640 1 --2- 172.21.15.69:0/333757903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d1c102a00 0x7f6d1c199010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:25:57.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.013+0000 7f6d1bfff640 1 --2- 172.21.15.69:0/333757903 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d1c108940 0x7f6d1c1a04e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f6d14009dc0 tx=0x7f6d14003c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:57.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.013+0000 7f6d197fa640 1 -- 172.21.15.69:0/333757903 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d14004240 con 0x7f6d1c108940 2024-04-25T11:25:57.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.013+0000 7f6d197fa640 1 -- 172.21.15.69:0/333757903 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d140305c0 con 0x7f6d1c108940 2024-04-25T11:25:57.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.013+0000 7f6d21f40640 1 -- 172.21.15.69:0/333757903 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d1c1a0d30 con 0x7f6d1c108940 2024-04-25T11:25:57.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.013+0000 7f6d197fa640 1 -- 172.21.15.69:0/333757903 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1403c670 con 0x7f6d1c108940 2024-04-25T11:25:57.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.014+0000 7f6d21f40640 1 -- 172.21.15.69:0/333757903 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d1c1a1210 con 0x7f6d1c108940 2024-04-25T11:25:57.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.015+0000 7f6d21f40640 1 -- 172.21.15.69:0/333757903 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d1c107750 con 0x7f6d1c108940 2024-04-25T11:25:57.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.016+0000 7f6d197fa640 1 -- 172.21.15.69:0/333757903 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f6d14030a70 con 0x7f6d1c108940 2024-04-25T11:25:57.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.016+0000 7f6d197fa640 1 --2- 172.21.15.69:0/333757903 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f6d080afb70 0x7f6d080b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:57.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.016+0000 7f6d1b7fe640 1 --2- 172.21.15.69:0/333757903 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f6d080afb70 0x7f6d080b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:57.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.017+0000 7f6d197fa640 1 -- 172.21.15.69:0/333757903 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f6d141007a0 con 0x7f6d1c108940 2024-04-25T11:25:57.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.017+0000 7f6d1b7fe640 1 --2- 172.21.15.69:0/333757903 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f6d080afb70 0x7f6d080b2080 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c103600 tx=0x7f6d0c00a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:57.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.021+0000 7f6d197fa640 1 -- 172.21.15.69:0/333757903 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6d140c8fb0 con 0x7f6d1c108940 2024-04-25T11:25:57.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.199+0000 7f6d21f40640 1 -- 172.21.15.69:0/333757903 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6d1c063840 con 0x7f6d1c108940 2024-04-25T11:25:57.192 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:56 smithi069 ceph-mon[29188]: pgmap v356: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:25:57.192 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:56 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/4164969738' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:25:57.192 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:56 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/1443228516' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:25:57.192 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:56 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2602021625' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:25:57.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.199+0000 7f6d197fa640 1 -- 172.21.15.69:0/333757903 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) ==== 100+0+11 (secure 0 0 0) 0x7f6d140c88b0 con 0x7f6d1c108940 2024-04-25T11:25:57.193 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-04-25T11:25:57.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.201+0000 7f6d21f40640 1 -- 172.21.15.69:0/333757903 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f6d080afb70 msgr2=0x7f6d080b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:57.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.201+0000 7f6d21f40640 1 --2- 172.21.15.69:0/333757903 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f6d080afb70 0x7f6d080b2080 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c103600 tx=0x7f6d0c00a560 comp rx=0 tx=0).stop 2024-04-25T11:25:57.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.201+0000 7f6d21f40640 1 -- 172.21.15.69:0/333757903 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d1c108940 msgr2=0x7f6d1c1a04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:57.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.201+0000 7f6d21f40640 1 --2- 172.21.15.69:0/333757903 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d1c108940 0x7f6d1c1a04e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f6d14009dc0 tx=0x7f6d14003c90 comp rx=0 tx=0).stop 2024-04-25T11:25:57.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.201+0000 7f6d21f40640 1 -- 172.21.15.69:0/333757903 shutdown_connections 2024-04-25T11:25:57.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.201+0000 7f6d21f40640 1 --2- 172.21.15.69:0/333757903 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f6d080afb70 0x7f6d080b2080 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:57.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.201+0000 7f6d21f40640 1 --2- 172.21.15.69:0/333757903 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d1c108940 0x7f6d1c1a04e0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:57.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.201+0000 7f6d21f40640 1 --2- 172.21.15.69:0/333757903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d1c103c40 0x7f6d1c199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:57.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.201+0000 7f6d21f40640 1 --2- 172.21.15.69:0/333757903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d1c102a00 0x7f6d1c199010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:57.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.201+0000 7f6d21f40640 1 -- 172.21.15.69:0/333757903 >> 172.21.15.69:0/333757903 conn(0x7f6d1c0fdbf0 msgr2=0x7f6d1c104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:57.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.201+0000 7f6d21f40640 1 -- 172.21.15.69:0/333757903 shutdown_connections 2024-04-25T11:25:57.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.201+0000 7f6d21f40640 1 -- 172.21.15.69:0/333757903 wait complete. 2024-04-25T11:25:57.205 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T11:25:57.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:56 smithi152 ceph-mon[32701]: pgmap v356: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:25:57.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:56 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/4164969738' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:25:57.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:56 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/1443228516' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:25:57.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:56 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/2602021625' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:25:57.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.327+0000 7f020414e640 1 -- 172.21.15.69:0/1592235028 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01fc1050b0 msgr2=0x7f01fc1054c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:57.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.327+0000 7f020414e640 1 --2- 172.21.15.69:0/1592235028 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01fc1050b0 0x7f01fc1054c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f01f8009900 tx=0x7f01f802a410 comp rx=0 tx=0).stop 2024-04-25T11:25:57.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.327+0000 7f020414e640 1 -- 172.21.15.69:0/1592235028 shutdown_connections 2024-04-25T11:25:57.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.327+0000 7f020414e640 1 --2- 172.21.15.69:0/1592235028 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01fc102260 0x7f01fc10bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:57.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.327+0000 7f020414e640 1 --2- 172.21.15.69:0/1592235028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01fc101760 0x7f01fc101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:57.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.327+0000 7f020414e640 1 --2- 172.21.15.69:0/1592235028 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01fc1050b0 0x7f01fc1054c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:57.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.327+0000 7f020414e640 1 -- 172.21.15.69:0/1592235028 >> 172.21.15.69:0/1592235028 conn(0x7f01fc0fd380 msgr2=0x7f01fc0ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:57.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.328+0000 7f020414e640 1 -- 172.21.15.69:0/1592235028 shutdown_connections 2024-04-25T11:25:57.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.328+0000 7f020414e640 1 -- 172.21.15.69:0/1592235028 wait complete. 2024-04-25T11:25:57.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.328+0000 7f020414e640 1 Processor -- start 2024-04-25T11:25:57.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.328+0000 7f020414e640 1 -- start start 2024-04-25T11:25:57.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.328+0000 7f020414e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01fc101760 0x7f01fc19a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:57.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.328+0000 7f020414e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01fc102260 0x7f01fc19a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:57.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.328+0000 7f020414e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01fc1050b0 0x7f01fc1a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:57.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.328+0000 7f020414e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f01fc10f520 con 0x7f01fc101760 2024-04-25T11:25:57.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.328+0000 7f020414e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f01fc10f3a0 con 0x7f01fc102260 2024-04-25T11:25:57.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.328+0000 7f020414e640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f01fc10f6a0 con 0x7f01fc1050b0 2024-04-25T11:25:57.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.328+0000 7f0201eea640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01fc101760 0x7f01fc19a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:57.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.328+0000 7f0201eea640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01fc101760 0x7f01fc19a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56986/0 (socket says 172.21.15.69:56986) 2024-04-25T11:25:57.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.328+0000 7f0201eea640 1 -- 172.21.15.69:0/2164830860 learned_addr learned my addr 172.21.15.69:0/2164830860 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:57.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.329+0000 7f02026eb640 1 --2- 172.21.15.69:0/2164830860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01fc1050b0 0x7f01fc1a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:57.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.329+0000 7f02016e9640 1 --2- 172.21.15.69:0/2164830860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01fc102260 0x7f01fc19a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:57.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.329+0000 7f0201eea640 1 -- 172.21.15.69:0/2164830860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01fc1050b0 msgr2=0x7f01fc1a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:57.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.329+0000 7f0201eea640 1 --2- 172.21.15.69:0/2164830860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01fc1050b0 0x7f01fc1a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:57.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.329+0000 7f0201eea640 1 -- 172.21.15.69:0/2164830860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01fc102260 msgr2=0x7f01fc19a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:57.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.329+0000 7f0201eea640 1 --2- 172.21.15.69:0/2164830860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01fc102260 0x7f01fc19a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:57.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.329+0000 7f0201eea640 1 -- 172.21.15.69:0/2164830860 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01fc1a1cc0 con 0x7f01fc101760 2024-04-25T11:25:57.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.329+0000 7f02016e9640 1 --2- 172.21.15.69:0/2164830860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01fc102260 0x7f01fc19a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:25:57.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.329+0000 7f0201eea640 1 --2- 172.21.15.69:0/2164830860 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01fc101760 0x7f01fc19a180 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f01f8005760 tx=0x7f01f8003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:57.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.329+0000 7f01f2ffd640 1 -- 172.21.15.69:0/2164830860 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f803f070 con 0x7f01fc101760 2024-04-25T11:25:57.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.329+0000 7f01f2ffd640 1 -- 172.21.15.69:0/2164830860 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f01f8031da0 con 0x7f01fc101760 2024-04-25T11:25:57.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.329+0000 7f01f2ffd640 1 -- 172.21.15.69:0/2164830860 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f8035030 con 0x7f01fc101760 2024-04-25T11:25:57.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.329+0000 7f020414e640 1 -- 172.21.15.69:0/2164830860 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01fc1a1f50 con 0x7f01fc101760 2024-04-25T11:25:57.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.329+0000 7f020414e640 1 -- 172.21.15.69:0/2164830860 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01fc1a2350 con 0x7f01fc101760 2024-04-25T11:25:57.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.330+0000 7f020414e640 1 -- 172.21.15.69:0/2164830860 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01fc108a40 con 0x7f01fc101760 2024-04-25T11:25:57.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.332+0000 7f01f2ffd640 1 -- 172.21.15.69:0/2164830860 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f01f8042070 con 0x7f01fc101760 2024-04-25T11:25:57.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.332+0000 7f01f2ffd640 1 --2- 172.21.15.69:0/2164830860 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f01e80afaf0 0x7f01e80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:57.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.332+0000 7f02016e9640 1 --2- 172.21.15.69:0/2164830860 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f01e80afaf0 0x7f01e80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:57.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.332+0000 7f01f2ffd640 1 -- 172.21.15.69:0/2164830860 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f01f80fffe0 con 0x7f01fc101760 2024-04-25T11:25:57.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.333+0000 7f02016e9640 1 --2- 172.21.15.69:0/2164830860 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f01e80afaf0 0x7f01e80b2000 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f01fc19b220 tx=0x7f01ec005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:57.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.336+0000 7f01f2ffd640 1 -- 172.21.15.69:0/2164830860 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f01f80c87f0 con 0x7f01fc101760 2024-04-25T11:25:57.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:56 smithi146 ceph-mon[32821]: pgmap v356: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:25:57.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:56 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/4164969738' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:25:57.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:56 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/1443228516' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:25:57.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:56 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/2602021625' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:25:57.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.514+0000 7f020414e640 1 -- 172.21.15.69:0/2164830860 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f01fc02dce0 con 0x7f01fc101760 2024-04-25T11:25:57.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.514+0000 7f01f2ffd640 1 -- 172.21.15.69:0/2164830860 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v84) ==== 103+0+11 (secure 0 0 0) 0x7f01f80c80f0 con 0x7f01fc101760 2024-04-25T11:25:57.507 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-25T11:25:57.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.516+0000 7f020414e640 1 -- 172.21.15.69:0/2164830860 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f01e80afaf0 msgr2=0x7f01e80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:57.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.516+0000 7f020414e640 1 --2- 172.21.15.69:0/2164830860 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f01e80afaf0 0x7f01e80b2000 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f01fc19b220 tx=0x7f01ec005e70 comp rx=0 tx=0).stop 2024-04-25T11:25:57.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.516+0000 7f020414e640 1 -- 172.21.15.69:0/2164830860 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01fc101760 msgr2=0x7f01fc19a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:57.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.516+0000 7f020414e640 1 --2- 172.21.15.69:0/2164830860 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01fc101760 0x7f01fc19a180 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f01f8005760 tx=0x7f01f8003d50 comp rx=0 tx=0).stop 2024-04-25T11:25:57.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.517+0000 7f020414e640 1 -- 172.21.15.69:0/2164830860 shutdown_connections 2024-04-25T11:25:57.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.517+0000 7f020414e640 1 --2- 172.21.15.69:0/2164830860 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f01e80afaf0 0x7f01e80b2000 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:57.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.517+0000 7f020414e640 1 --2- 172.21.15.69:0/2164830860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01fc1050b0 0x7f01fc1a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:57.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.517+0000 7f020414e640 1 --2- 172.21.15.69:0/2164830860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01fc102260 0x7f01fc19a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:57.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.517+0000 7f020414e640 1 --2- 172.21.15.69:0/2164830860 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01fc101760 0x7f01fc19a180 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:57.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.517+0000 7f020414e640 1 -- 172.21.15.69:0/2164830860 >> 172.21.15.69:0/2164830860 conn(0x7f01fc0fd380 msgr2=0x7f01fc105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:57.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.517+0000 7f020414e640 1 -- 172.21.15.69:0/2164830860 shutdown_connections 2024-04-25T11:25:57.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.517+0000 7f020414e640 1 -- 172.21.15.69:0/2164830860 wait complete. 2024-04-25T11:25:57.520 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:25:57.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.635+0000 7f32d2b0d640 1 -- 172.21.15.69:0/1515734293 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32cc106f10 msgr2=0x7f32cc109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:57.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.635+0000 7f32d2b0d640 1 --2- 172.21.15.69:0/1515734293 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32cc106f10 0x7f32cc109330 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f32b8009900 tx=0x7f32b802a3f0 comp rx=0 tx=0).stop 2024-04-25T11:25:57.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.635+0000 7f32d2b0d640 1 -- 172.21.15.69:0/1515734293 shutdown_connections 2024-04-25T11:25:57.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.635+0000 7f32d2b0d640 1 --2- 172.21.15.69:0/1515734293 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f32cc109870 0x7f32cc10bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:57.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.635+0000 7f32d2b0d640 1 --2- 172.21.15.69:0/1515734293 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32cc106f10 0x7f32cc109330 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:57.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.635+0000 7f32d2b0d640 1 --2- 172.21.15.69:0/1515734293 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32cc1045b0 0x7f32cc1069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:57.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.635+0000 7f32d2b0d640 1 -- 172.21.15.69:0/1515734293 >> 172.21.15.69:0/1515734293 conn(0x7f32cc0fdbd0 msgr2=0x7f32cc100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:57.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.635+0000 7f32d2b0d640 1 -- 172.21.15.69:0/1515734293 shutdown_connections 2024-04-25T11:25:57.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.635+0000 7f32d2b0d640 1 -- 172.21.15.69:0/1515734293 wait complete. 2024-04-25T11:25:57.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.636+0000 7f32d2b0d640 1 Processor -- start 2024-04-25T11:25:57.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.636+0000 7f32d2b0d640 1 -- start start 2024-04-25T11:25:57.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.636+0000 7f32d2b0d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32cc1045b0 0x7f32cc19a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:57.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.636+0000 7f32d2b0d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32cc106f10 0x7f32cc19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:57.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.636+0000 7f32d2b0d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f32cc109870 0x7f32cc1a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:57.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.636+0000 7f32d2b0d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc10f480 con 0x7f32cc106f10 2024-04-25T11:25:57.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.636+0000 7f32d2b0d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc10f300 con 0x7f32cc1045b0 2024-04-25T11:25:57.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.636+0000 7f32d2b0d640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc10f600 con 0x7f32cc109870 2024-04-25T11:25:57.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.636+0000 7f32cbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32cc106f10 0x7f32cc19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:57.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.636+0000 7f32cbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32cc106f10 0x7f32cc19a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56998/0 (socket says 172.21.15.69:56998) 2024-04-25T11:25:57.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.636+0000 7f32cbfff640 1 -- 172.21.15.69:0/527961838 learned_addr learned my addr 172.21.15.69:0/527961838 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:57.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.636+0000 7f32d10aa640 1 --2- 172.21.15.69:0/527961838 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f32cc109870 0x7f32cc1a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:57.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.637+0000 7f32d08a9640 1 --2- 172.21.15.69:0/527961838 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32cc1045b0 0x7f32cc19a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:57.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.637+0000 7f32cbfff640 1 -- 172.21.15.69:0/527961838 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f32cc109870 msgr2=0x7f32cc1a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:57.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.637+0000 7f32cbfff640 1 --2- 172.21.15.69:0/527961838 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f32cc109870 0x7f32cc1a1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:57.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.637+0000 7f32cbfff640 1 -- 172.21.15.69:0/527961838 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32cc1045b0 msgr2=0x7f32cc19a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:57.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.637+0000 7f32cbfff640 1 --2- 172.21.15.69:0/527961838 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32cc1045b0 0x7f32cc19a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:57.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.637+0000 7f32cbfff640 1 -- 172.21.15.69:0/527961838 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32cc1a1be0 con 0x7f32cc106f10 2024-04-25T11:25:57.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.637+0000 7f32d08a9640 1 --2- 172.21.15.69:0/527961838 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32cc1045b0 0x7f32cc19a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:25:57.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.637+0000 7f32cbfff640 1 --2- 172.21.15.69:0/527961838 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32cc106f10 0x7f32cc19a690 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f32b8009230 tx=0x7f32b80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:57.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.637+0000 7f32c9ffb640 1 -- 172.21.15.69:0/527961838 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b803f030 con 0x7f32cc106f10 2024-04-25T11:25:57.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.637+0000 7f32c9ffb640 1 -- 172.21.15.69:0/527961838 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f32b8003c90 con 0x7f32cc106f10 2024-04-25T11:25:57.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.637+0000 7f32c9ffb640 1 -- 172.21.15.69:0/527961838 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b8035070 con 0x7f32cc106f10 2024-04-25T11:25:57.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.637+0000 7f32d2b0d640 1 -- 172.21.15.69:0/527961838 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32cc1a1e70 con 0x7f32cc106f10 2024-04-25T11:25:57.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.637+0000 7f32d2b0d640 1 -- 172.21.15.69:0/527961838 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32cc1a2220 con 0x7f32cc106f10 2024-04-25T11:25:57.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.638+0000 7f32d2b0d640 1 -- 172.21.15.69:0/527961838 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32cc0ff6f0 con 0x7f32cc106f10 2024-04-25T11:25:57.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.640+0000 7f32c9ffb640 1 -- 172.21.15.69:0/527961838 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f32b8042030 con 0x7f32cc106f10 2024-04-25T11:25:57.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.640+0000 7f32c9ffb640 1 --2- 172.21.15.69:0/527961838 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f32bc0afaf0 0x7f32bc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:57.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.640+0000 7f32d08a9640 1 --2- 172.21.15.69:0/527961838 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f32bc0afaf0 0x7f32bc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:57.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.640+0000 7f32c9ffb640 1 -- 172.21.15.69:0/527961838 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f32b80ff680 con 0x7f32cc106f10 2024-04-25T11:25:57.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.641+0000 7f32d08a9640 1 --2- 172.21.15.69:0/527961838 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f32bc0afaf0 0x7f32bc0b2000 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f32c00058c0 tx=0x7f32c0005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:57.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.646+0000 7f32c9ffb640 1 -- 172.21.15.69:0/527961838 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f32b80c7e90 con 0x7f32cc106f10 2024-04-25T11:25:57.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.864+0000 7f32d2b0d640 1 -- 172.21.15.69:0/527961838 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32cc02db50 con 0x7f32cc106f10 2024-04-25T11:25:57.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.864+0000 7f32c9ffb640 1 -- 172.21.15.69:0/527961838 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11944 (secure 0 0 0) 0x7f32b80c7790 con 0x7f32cc106f10 2024-04-25T11:25:57.857 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:25:57.857 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3399424094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3399424094},{"type":"v1","addr":"172.21.15.69:6835","nonce":3399424094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1463217771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1463217771},{"type":"v1","addr":"172.21.15.69:6837","nonce":1463217771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T11:23:40.984930+0000","modified":"2024-04-25T11:24:09.061188+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24485,"mds_2":24475,"mds_3":24458,"mds_4":14625},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6839/966316001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":966316001},{"type":"v1","addr":"172.21.15.69:6839","nonce":966316001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.69:6841/2060225181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":2060225181},{"type":"v1","addr":"172.21.15.69:6841","nonce":2060225181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6833/4197482473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":4197482473},{"type":"v1","addr":"172.21.15.152:6833","nonce":4197482473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/4267885323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4267885323},{"type":"v1","addr":"172.21.15.146:6833","nonce":4267885323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6835/3188843362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3188843362},{"type":"v1","addr":"172.21.15.152:6835","nonce":3188843362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6837/563274982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":563274982},{"type":"v1","addr":"172.21.15.152:6837","nonce":563274982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.146:6835/1876533699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1876533699},{"type":"v1","addr":"172.21.15.146:6835","nonce":1876533699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.152:6839/3008615657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":3008615657},{"type":"v1","addr":"172.21.15.152:6839","nonce":3008615657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.146:6837/2730317134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2730317134},{"type":"v1","addr":"172.21.15.146:6837","nonce":2730317134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6839/2013283448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":2013283448},{"type":"v1","addr":"172.21.15.146:6839","nonce":2013283448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24485,24515,24475,24458,14625]},"id":1}]} 2024-04-25T11:25:57.857 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 28 2024-04-25T11:25:57.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.866+0000 7f32d2b0d640 1 -- 172.21.15.69:0/527961838 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f32bc0afaf0 msgr2=0x7f32bc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:57.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.866+0000 7f32d2b0d640 1 --2- 172.21.15.69:0/527961838 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f32bc0afaf0 0x7f32bc0b2000 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f32c00058c0 tx=0x7f32c0005d30 comp rx=0 tx=0).stop 2024-04-25T11:25:57.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.866+0000 7f32d2b0d640 1 -- 172.21.15.69:0/527961838 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32cc106f10 msgr2=0x7f32cc19a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:57.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.866+0000 7f32d2b0d640 1 --2- 172.21.15.69:0/527961838 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32cc106f10 0x7f32cc19a690 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f32b8009230 tx=0x7f32b80042f0 comp rx=0 tx=0).stop 2024-04-25T11:25:57.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.866+0000 7f32d2b0d640 1 -- 172.21.15.69:0/527961838 shutdown_connections 2024-04-25T11:25:57.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.866+0000 7f32d2b0d640 1 --2- 172.21.15.69:0/527961838 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f32bc0afaf0 0x7f32bc0b2000 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:57.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.866+0000 7f32d2b0d640 1 --2- 172.21.15.69:0/527961838 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f32cc109870 0x7f32cc1a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:57.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.866+0000 7f32d2b0d640 1 --2- 172.21.15.69:0/527961838 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32cc106f10 0x7f32cc19a690 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:57.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.866+0000 7f32d2b0d640 1 --2- 172.21.15.69:0/527961838 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f32cc1045b0 0x7f32cc19a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:57.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.866+0000 7f32d2b0d640 1 -- 172.21.15.69:0/527961838 >> 172.21.15.69:0/527961838 conn(0x7f32cc0fdbd0 msgr2=0x7f32cc1065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:57.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.866+0000 7f32d2b0d640 1 -- 172.21.15.69:0/527961838 shutdown_connections 2024-04-25T11:25:57.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.866+0000 7f32d2b0d640 1 -- 172.21.15.69:0/527961838 wait complete. 2024-04-25T11:25:57.871 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:25:57.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.987+0000 7f0f74b26640 1 -- 172.21.15.69:0/1980224005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f70109a40 msgr2=0x7f0f7010be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:57.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.987+0000 7f0f74b26640 1 --2- 172.21.15.69:0/1980224005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f70109a40 0x7f0f7010be60 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0f68009ca0 tx=0x7f0f6802a8c0 comp rx=0 tx=0).stop 2024-04-25T11:25:57.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.987+0000 7f0f74b26640 1 -- 172.21.15.69:0/1980224005 shutdown_connections 2024-04-25T11:25:57.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.987+0000 7f0f74b26640 1 --2- 172.21.15.69:0/1980224005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f70109a40 0x7f0f7010be60 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:57.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.987+0000 7f0f74b26640 1 --2- 172.21.15.69:0/1980224005 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f70104d40 0x7f0f70105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:57.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.987+0000 7f0f74b26640 1 --2- 172.21.15.69:0/1980224005 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f70103b00 0x7f0f70104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:57.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.987+0000 7f0f74b26640 1 -- 172.21.15.69:0/1980224005 >> 172.21.15.69:0/1980224005 conn(0x7f0f700fed10 msgr2=0x7f0f701011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:57.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.987+0000 7f0f74b26640 1 -- 172.21.15.69:0/1980224005 shutdown_connections 2024-04-25T11:25:57.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.987+0000 7f0f74b26640 1 -- 172.21.15.69:0/1980224005 wait complete. 2024-04-25T11:25:57.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.988+0000 7f0f74b26640 1 Processor -- start 2024-04-25T11:25:57.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.988+0000 7f0f74b26640 1 -- start start 2024-04-25T11:25:57.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.988+0000 7f0f74b26640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f70103b00 0x7f0f7019a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:57.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.988+0000 7f0f74b26640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f70104d40 0x7f0f7019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:57.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.988+0000 7f0f74b26640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f70109a40 0x7f0f701a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:57.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.988+0000 7f0f74b26640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7010f530 con 0x7f0f70109a40 2024-04-25T11:25:57.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.988+0000 7f0f74b26640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7010f3b0 con 0x7f0f70103b00 2024-04-25T11:25:57.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.988+0000 7f0f74b26640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7010f6b0 con 0x7f0f70104d40 2024-04-25T11:25:57.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.988+0000 7f0f6ed9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f70109a40 0x7f0f701a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:57.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.988+0000 7f0f6ed9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f70109a40 0x7f0f701a15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57022/0 (socket says 172.21.15.69:57022) 2024-04-25T11:25:57.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.988+0000 7f0f6ed9d640 1 -- 172.21.15.69:0/305737777 learned_addr learned my addr 172.21.15.69:0/305737777 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:57.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.988+0000 7f0f6e59c640 1 --2- 172.21.15.69:0/305737777 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f70103b00 0x7f0f7019a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:57.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.988+0000 7f0f6ed9d640 1 -- 172.21.15.69:0/305737777 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f70104d40 msgr2=0x7f0f7019a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:57.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.988+0000 7f0f6dd9b640 1 --2- 172.21.15.69:0/305737777 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f70104d40 0x7f0f7019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:57.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.988+0000 7f0f6ed9d640 1 --2- 172.21.15.69:0/305737777 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f70104d40 0x7f0f7019a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:57.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.988+0000 7f0f6ed9d640 1 -- 172.21.15.69:0/305737777 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f70103b00 msgr2=0x7f0f7019a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:57.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.988+0000 7f0f6ed9d640 1 --2- 172.21.15.69:0/305737777 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f70103b00 0x7f0f7019a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:57.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.988+0000 7f0f6ed9d640 1 -- 172.21.15.69:0/305737777 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f701a1b90 con 0x7f0f70109a40 2024-04-25T11:25:57.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.988+0000 7f0f6e59c640 1 --2- 172.21.15.69:0/305737777 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f70103b00 0x7f0f7019a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:25:57.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.988+0000 7f0f6ed9d640 1 --2- 172.21.15.69:0/305737777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f70109a40 0x7f0f701a15d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0f68031040 tx=0x7f0f68003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:57.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.989+0000 7f0f5f7fe640 1 -- 172.21.15.69:0/305737777 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f68004240 con 0x7f0f70109a40 2024-04-25T11:25:57.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.989+0000 7f0f74b26640 1 -- 172.21.15.69:0/305737777 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f701a1e20 con 0x7f0f70109a40 2024-04-25T11:25:57.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.989+0000 7f0f5f7fe640 1 -- 172.21.15.69:0/305737777 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0f68003e50 con 0x7f0f70109a40 2024-04-25T11:25:57.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.989+0000 7f0f5f7fe640 1 -- 172.21.15.69:0/305737777 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6803c410 con 0x7f0f70109a40 2024-04-25T11:25:57.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.989+0000 7f0f74b26640 1 -- 172.21.15.69:0/305737777 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f701a22e0 con 0x7f0f70109a40 2024-04-25T11:25:57.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.990+0000 7f0f74b26640 1 -- 172.21.15.69:0/305737777 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f70108850 con 0x7f0f70109a40 2024-04-25T11:25:57.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.991+0000 7f0f5f7fe640 1 -- 172.21.15.69:0/305737777 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f0f680306e0 con 0x7f0f70109a40 2024-04-25T11:25:57.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.991+0000 7f0f5f7fe640 1 --2- 172.21.15.69:0/305737777 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f0f580afaf0 0x7f0f580b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:57.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.992+0000 7f0f6e59c640 1 --2- 172.21.15.69:0/305737777 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f0f580afaf0 0x7f0f580b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:57.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.992+0000 7f0f5f7fe640 1 -- 172.21.15.69:0/305737777 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f0f68100990 con 0x7f0f70109a40 2024-04-25T11:25:57.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.992+0000 7f0f6e59c640 1 --2- 172.21.15.69:0/305737777 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f0f580afaf0 0x7f0f580b2000 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0f640058c0 tx=0x7f0f64005ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:57.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:57.995+0000 7f0f5f7fe640 1 -- 172.21.15.69:0/305737777 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0f680c91a0 con 0x7f0f70109a40 2024-04-25T11:25:58.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.174+0000 7f0f74b26640 1 -- 172.21.15.69:0/305737777 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0f7002dd50 con 0x7f0f70109a40 2024-04-25T11:25:58.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.175+0000 7f0f5f7fe640 1 -- 172.21.15.69:0/305737777 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+24862 (secure 0 0 0) 0x7f0f680c8aa0 con 0x7f0f70109a40 2024-04-25T11:25:58.168 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:25:58.168 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":84,"fsid":"f7ed73dc-02f4-11ef-bc93-c7b262605968","created":"2024-04-25T11:15:37.965281+0000","modified":"2024-04-25T11:25:33.888184+0000","last_up_change":"2024-04-25T11:22:50.691869+0000","last_in_change":"2024-04-25T11:22:30.271959+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:18:57.781263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:23:38.685110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:23:39.943803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T11:23:43.650521+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6a0d179-4998-4754-bf82-2ea3dac54e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6803","nonce":3756025235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6805","nonce":3756025235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6809","nonce":3756025235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6807","nonce":3756025235}]},"public_addr":"172.21.15.69:6803/3756025235","cluster_addr":"172.21.15.69:6805/3756025235","heartbeat_back_addr":"172.21.15.69:6809/3756025235","heartbeat_front_addr":"172.21.15.69:6807/3756025235","state":["exists","up"]},{"osd":1,"uuid":"c0c66930-4233-493f-b2bb-99c473994b59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6801","nonce":3166305213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6803","nonce":3166305213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6807","nonce":3166305213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6805","nonce":3166305213}]},"public_addr":"172.21.15.146:6801/3166305213","cluster_addr":"172.21.15.146:6803/3166305213","heartbeat_back_addr":"172.21.15.146:6807/3166305213","heartbeat_front_addr":"172.21.15.146:6805/3166305213","state":["exists","up"]},{"osd":2,"uuid":"d4ef735b-9786-4cb9-83ef-7c3f0691a729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6801","nonce":1474319175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6803","nonce":1474319175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6807","nonce":1474319175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6805","nonce":1474319175}]},"public_addr":"172.21.15.152:6801/1474319175","cluster_addr":"172.21.15.152:6803/1474319175","heartbeat_back_addr":"172.21.15.152:6807/1474319175","heartbeat_front_addr":"172.21.15.152:6805/1474319175","state":["exists","up"]},{"osd":3,"uuid":"0ab85fb8-5bbb-4ffc-bddf-cae7644a64ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6811","nonce":1809071296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6813","nonce":1809071296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6817","nonce":1809071296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6815","nonce":1809071296}]},"public_addr":"172.21.15.69:6811/1809071296","cluster_addr":"172.21.15.69:6813/1809071296","heartbeat_back_addr":"172.21.15.69:6817/1809071296","heartbeat_front_addr":"172.21.15.69:6815/1809071296","state":["exists","up"]},{"osd":4,"uuid":"273e0592-bbbc-4188-8eeb-46a3879cf632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6809","nonce":1898520972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6811","nonce":1898520972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6815","nonce":1898520972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6813","nonce":1898520972}]},"public_addr":"172.21.15.146:6809/1898520972","cluster_addr":"172.21.15.146:6811/1898520972","heartbeat_back_addr":"172.21.15.146:6815/1898520972","heartbeat_front_addr":"172.21.15.146:6813/1898520972","state":["exists","up"]},{"osd":5,"uuid":"bbeae945-9fa2-48f9-a1d6-8bf18dcaef19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6809","nonce":1996555108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6811","nonce":1996555108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6815","nonce":1996555108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6813","nonce":1996555108}]},"public_addr":"172.21.15.152:6809/1996555108","cluster_addr":"172.21.15.152:6811/1996555108","heartbeat_back_addr":"172.21.15.152:6815/1996555108","heartbeat_front_addr":"172.21.15.152:6813/1996555108","state":["exists","up"]},{"osd":6,"uuid":"dbb61253-30bc-418e-bd75-118c80bbf931","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6819","nonce":491215762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6821","nonce":491215762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6825","nonce":491215762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6823","nonce":491215762}]},"public_addr":"172.21.15.69:6819/491215762","cluster_addr":"172.21.15.69:6821/491215762","heartbeat_back_addr":"172.21.15.69:6825/491215762","heartbeat_front_addr":"172.21.15.69:6823/491215762","state":["exists","up"]},{"osd":7,"uuid":"75f23854-e283-4e3e-94cf-7b4079d83876","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6817","nonce":2330294690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6819","nonce":2330294690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6823","nonce":2330294690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6821","nonce":2330294690}]},"public_addr":"172.21.15.146:6817/2330294690","cluster_addr":"172.21.15.146:6819/2330294690","heartbeat_back_addr":"172.21.15.146:6823/2330294690","heartbeat_front_addr":"172.21.15.146:6821/2330294690","state":["exists","up"]},{"osd":8,"uuid":"1aa1060c-0c14-4abd-be56-5ebfeb78759c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6817","nonce":950669381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6819","nonce":950669381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6823","nonce":950669381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6821","nonce":950669381}]},"public_addr":"172.21.15.152:6817/950669381","cluster_addr":"172.21.15.152:6819/950669381","heartbeat_back_addr":"172.21.15.152:6823/950669381","heartbeat_front_addr":"172.21.15.152:6821/950669381","state":["exists","up"]},{"osd":9,"uuid":"3f4bf800-4090-4138-9bb3-d001f13f450c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6827","nonce":2414186758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6829","nonce":2414186758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6833","nonce":2414186758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6831","nonce":2414186758}]},"public_addr":"172.21.15.69:6827/2414186758","cluster_addr":"172.21.15.69:6829/2414186758","heartbeat_back_addr":"172.21.15.69:6833/2414186758","heartbeat_front_addr":"172.21.15.69:6831/2414186758","state":["exists","up"]},{"osd":10,"uuid":"19e967e5-51ce-4506-80c9-0a95441baf99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6825","nonce":707256400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6827","nonce":707256400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6831","nonce":707256400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6829","nonce":707256400}]},"public_addr":"172.21.15.146:6825/707256400","cluster_addr":"172.21.15.146:6827/707256400","heartbeat_back_addr":"172.21.15.146:6831/707256400","heartbeat_front_addr":"172.21.15.146:6829/707256400","state":["exists","up"]},{"osd":11,"uuid":"7788c2de-6930-4808-be0f-f1fd1ae0cb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6825","nonce":2934336833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6827","nonce":2934336833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6831","nonce":2934336833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6829","nonce":2934336833}]},"public_addr":"172.21.15.152:6825/2934336833","cluster_addr":"172.21.15.152:6827/2934336833","heartbeat_back_addr":"172.21.15.152:6831/2934336833","heartbeat_front_addr":"172.21.15.152:6829/2934336833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:07.497448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:31.227494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:55.027978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:20.713437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:46.214704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:20:11.199983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:20:37.177070+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:02.951240+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:28.963680+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:56.095807+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:22:22.221744+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:22:48.929334+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1b","mappings":[{"from":5,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/1372727546":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6801/1097124842":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6800/1508198215":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3103465990":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3540132960":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3899912190":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6800/1097124842":"2024-04-26T11:16:31.688159+0000","172.21.15.69:0/1199907202":"2024-04-26T11:16:04.860792+0000","172.21.15.69:6801/1508198215":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/2879102529":"2024-04-26T11:16:31.688159+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:25:58.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.177+0000 7f0f74b26640 1 -- 172.21.15.69:0/305737777 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f0f580afaf0 msgr2=0x7f0f580b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:58.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.177+0000 7f0f74b26640 1 --2- 172.21.15.69:0/305737777 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f0f580afaf0 0x7f0f580b2000 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0f640058c0 tx=0x7f0f64005ec0 comp rx=0 tx=0).stop 2024-04-25T11:25:58.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.177+0000 7f0f74b26640 1 -- 172.21.15.69:0/305737777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f70109a40 msgr2=0x7f0f701a15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:58.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.177+0000 7f0f74b26640 1 --2- 172.21.15.69:0/305737777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f70109a40 0x7f0f701a15d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0f68031040 tx=0x7f0f68003ce0 comp rx=0 tx=0).stop 2024-04-25T11:25:58.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.177+0000 7f0f74b26640 1 -- 172.21.15.69:0/305737777 shutdown_connections 2024-04-25T11:25:58.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.177+0000 7f0f74b26640 1 --2- 172.21.15.69:0/305737777 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f0f580afaf0 0x7f0f580b2000 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:58.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.177+0000 7f0f74b26640 1 --2- 172.21.15.69:0/305737777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f70109a40 0x7f0f701a15d0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:58.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.177+0000 7f0f74b26640 1 --2- 172.21.15.69:0/305737777 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f70104d40 0x7f0f7019a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:58.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.177+0000 7f0f74b26640 1 --2- 172.21.15.69:0/305737777 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f70103b00 0x7f0f7019a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:58.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.177+0000 7f0f74b26640 1 -- 172.21.15.69:0/305737777 >> 172.21.15.69:0/305737777 conn(0x7f0f700fed10 msgr2=0x7f0f70105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:58.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.177+0000 7f0f74b26640 1 -- 172.21.15.69:0/305737777 shutdown_connections 2024-04-25T11:25:58.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.177+0000 7f0f74b26640 1 -- 172.21.15.69:0/305737777 wait complete. 2024-04-25T11:25:58.180 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:57 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/333757903' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:25:58.180 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:57 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2164830860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T11:25:58.180 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:57 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/527961838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:25:58.181 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:18:57.781263+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:23:38.685110+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:23:39.943803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T11:23:43.650521+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T11:25:58.182 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:25:58.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:57 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/333757903' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:25:58.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:57 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/2164830860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T11:25:58.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:57 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/527961838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:25:58.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.301+0000 7f0382405640 1 -- 172.21.15.69:0/3571159097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f037c103af0 msgr2=0x7f037c1047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:58.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.301+0000 7f0382405640 1 --2- 172.21.15.69:0/3571159097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f037c103af0 0x7f037c1047f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f036c009900 tx=0x7f036c02a410 comp rx=0 tx=0).stop 2024-04-25T11:25:58.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.301+0000 7f0382405640 1 -- 172.21.15.69:0/3571159097 shutdown_connections 2024-04-25T11:25:58.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.301+0000 7f0382405640 1 --2- 172.21.15.69:0/3571159097 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f037c109a30 0x7f037c10be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:58.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.301+0000 7f0382405640 1 --2- 172.21.15.69:0/3571159097 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f037c104d30 0x7f037c105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:58.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.301+0000 7f0382405640 1 --2- 172.21.15.69:0/3571159097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f037c103af0 0x7f037c1047f0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:58.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.301+0000 7f0382405640 1 -- 172.21.15.69:0/3571159097 >> 172.21.15.69:0/3571159097 conn(0x7f037c0fece0 msgr2=0x7f037c101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:58.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.301+0000 7f0382405640 1 -- 172.21.15.69:0/3571159097 shutdown_connections 2024-04-25T11:25:58.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.301+0000 7f0382405640 1 -- 172.21.15.69:0/3571159097 wait complete. 2024-04-25T11:25:58.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.301+0000 7f0382405640 1 Processor -- start 2024-04-25T11:25:58.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.301+0000 7f0382405640 1 -- start start 2024-04-25T11:25:58.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.302+0000 7f0382405640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f037c103af0 0x7f037c19a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:58.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.302+0000 7f0382405640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f037c104d30 0x7f037c19a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:58.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.302+0000 7f0382405640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f037c109a30 0x7f037c1a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:58.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.302+0000 7f0382405640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f037c10f4a0 con 0x7f037c103af0 2024-04-25T11:25:58.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.302+0000 7f0382405640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f037c10f320 con 0x7f037c109a30 2024-04-25T11:25:58.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.302+0000 7f0382405640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f037c10f620 con 0x7f037c104d30 2024-04-25T11:25:58.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.302+0000 7f037bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f037c103af0 0x7f037c19a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:58.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.302+0000 7f037bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f037c103af0 0x7f037c19a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57034/0 (socket says 172.21.15.69:57034) 2024-04-25T11:25:58.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.302+0000 7f037bfff640 1 -- 172.21.15.69:0/3885166571 learned_addr learned my addr 172.21.15.69:0/3885166571 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:58.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.302+0000 7f037b7fe640 1 --2- 172.21.15.69:0/3885166571 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f037c104d30 0x7f037c19a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:58.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.302+0000 7f03809a2640 1 --2- 172.21.15.69:0/3885166571 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f037c109a30 0x7f037c1a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:58.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.302+0000 7f037bfff640 1 -- 172.21.15.69:0/3885166571 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f037c104d30 msgr2=0x7f037c19a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:58.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.302+0000 7f037bfff640 1 --2- 172.21.15.69:0/3885166571 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f037c104d30 0x7f037c19a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:58.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.302+0000 7f037bfff640 1 -- 172.21.15.69:0/3885166571 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f037c109a30 msgr2=0x7f037c1a1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:58.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.302+0000 7f037bfff640 1 --2- 172.21.15.69:0/3885166571 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f037c109a30 0x7f037c1a1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:58.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.302+0000 7f037bfff640 1 -- 172.21.15.69:0/3885166571 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f037c1a1c20 con 0x7f037c103af0 2024-04-25T11:25:58.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.302+0000 7f037b7fe640 1 --2- 172.21.15.69:0/3885166571 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f037c104d30 0x7f037c19a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:25:58.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.302+0000 7f03809a2640 1 --2- 172.21.15.69:0/3885166571 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f037c109a30 0x7f037c1a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:25:58.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.302+0000 7f037bfff640 1 --2- 172.21.15.69:0/3885166571 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f037c103af0 0x7f037c19a200 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f036c005760 tx=0x7f036c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:58.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.302+0000 7f03797fa640 1 -- 172.21.15.69:0/3885166571 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c03f030 con 0x7f037c103af0 2024-04-25T11:25:58.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.302+0000 7f03797fa640 1 -- 172.21.15.69:0/3885166571 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f036c003a80 con 0x7f037c103af0 2024-04-25T11:25:58.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.302+0000 7f0382405640 1 -- 172.21.15.69:0/3885166571 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f037c1a1eb0 con 0x7f037c103af0 2024-04-25T11:25:58.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.302+0000 7f03797fa640 1 -- 172.21.15.69:0/3885166571 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c035070 con 0x7f037c103af0 2024-04-25T11:25:58.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.303+0000 7f0382405640 1 -- 172.21.15.69:0/3885166571 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f037c1a22b0 con 0x7f037c103af0 2024-04-25T11:25:58.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.303+0000 7f0382405640 1 -- 172.21.15.69:0/3885166571 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f037c108840 con 0x7f037c103af0 2024-04-25T11:25:58.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.305+0000 7f03797fa640 1 -- 172.21.15.69:0/3885166571 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f036c042030 con 0x7f037c103af0 2024-04-25T11:25:58.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.305+0000 7f03797fa640 1 --2- 172.21.15.69:0/3885166571 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f03640afaf0 0x7f03640b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:58.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.305+0000 7f037b7fe640 1 --2- 172.21.15.69:0/3885166571 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f03640afaf0 0x7f03640b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:58.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.305+0000 7f03797fa640 1 -- 172.21.15.69:0/3885166571 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f036c0ffbc0 con 0x7f037c103af0 2024-04-25T11:25:58.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.306+0000 7f037b7fe640 1 --2- 172.21.15.69:0/3885166571 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f03640afaf0 0x7f03640b2000 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0370005fd0 tx=0x7f0370005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:58.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.309+0000 7f03797fa640 1 -- 172.21.15.69:0/3885166571 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f036c0c83d0 con 0x7f037c103af0 2024-04-25T11:25:58.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:57 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/333757903' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:25:58.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:57 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/2164830860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T11:25:58.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:57 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/527961838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:25:58.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.486+0000 7f0382405640 1 -- 172.21.15.69:0/3885166571 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f037c02dce0 con 0x7f037c103af0 2024-04-25T11:25:58.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.487+0000 7f03797fa640 1 -- 172.21.15.69:0/3885166571 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7f036c0c7cd0 con 0x7f037c103af0 2024-04-25T11:25:58.479 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-04-25T11:25:58.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.488+0000 7f0382405640 1 -- 172.21.15.69:0/3885166571 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f03640afaf0 msgr2=0x7f03640b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:58.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.489+0000 7f0382405640 1 --2- 172.21.15.69:0/3885166571 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f03640afaf0 0x7f03640b2000 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0370005fd0 tx=0x7f0370005850 comp rx=0 tx=0).stop 2024-04-25T11:25:58.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.489+0000 7f0382405640 1 -- 172.21.15.69:0/3885166571 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f037c103af0 msgr2=0x7f037c19a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:58.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.489+0000 7f0382405640 1 --2- 172.21.15.69:0/3885166571 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f037c103af0 0x7f037c19a200 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f036c005760 tx=0x7f036c004270 comp rx=0 tx=0).stop 2024-04-25T11:25:58.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.489+0000 7f0382405640 1 -- 172.21.15.69:0/3885166571 shutdown_connections 2024-04-25T11:25:58.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.489+0000 7f0382405640 1 --2- 172.21.15.69:0/3885166571 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f03640afaf0 0x7f03640b2000 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:58.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.489+0000 7f0382405640 1 --2- 172.21.15.69:0/3885166571 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f037c109a30 0x7f037c1a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:58.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.489+0000 7f0382405640 1 --2- 172.21.15.69:0/3885166571 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f037c104d30 0x7f037c19a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:58.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.489+0000 7f0382405640 1 --2- 172.21.15.69:0/3885166571 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f037c103af0 0x7f037c19a200 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:58.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.489+0000 7f0382405640 1 -- 172.21.15.69:0/3885166571 >> 172.21.15.69:0/3885166571 conn(0x7f037c0fece0 msgr2=0x7f037c105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:58.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.489+0000 7f0382405640 1 -- 172.21.15.69:0/3885166571 shutdown_connections 2024-04-25T11:25:58.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.489+0000 7f0382405640 1 -- 172.21.15.69:0/3885166571 wait complete. 2024-04-25T11:25:58.492 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:25:58.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.611+0000 7efff4005640 1 -- 172.21.15.69:0/4126627142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7effec109a30 msgr2=0x7effec10be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:58.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.611+0000 7efff4005640 1 --2- 172.21.15.69:0/4126627142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7effec109a30 0x7effec10be50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7effe4009ca0 tx=0x7effe402a8c0 comp rx=0 tx=0).stop 2024-04-25T11:25:58.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.611+0000 7efff4005640 1 -- 172.21.15.69:0/4126627142 shutdown_connections 2024-04-25T11:25:58.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.611+0000 7efff4005640 1 --2- 172.21.15.69:0/4126627142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7effec109a30 0x7effec10be50 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:58.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.611+0000 7efff4005640 1 --2- 172.21.15.69:0/4126627142 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7effec104d30 0x7effec105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:58.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.611+0000 7efff4005640 1 --2- 172.21.15.69:0/4126627142 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effec103af0 0x7effec1047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:58.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.611+0000 7efff4005640 1 -- 172.21.15.69:0/4126627142 >> 172.21.15.69:0/4126627142 conn(0x7effec0fece0 msgr2=0x7effec101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:58.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.611+0000 7efff4005640 1 -- 172.21.15.69:0/4126627142 shutdown_connections 2024-04-25T11:25:58.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.611+0000 7efff4005640 1 -- 172.21.15.69:0/4126627142 wait complete. 2024-04-25T11:25:58.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.612+0000 7efff4005640 1 Processor -- start 2024-04-25T11:25:58.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.612+0000 7efff4005640 1 -- start start 2024-04-25T11:25:58.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.612+0000 7efff4005640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7effec103af0 0x7effec19a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:58.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.612+0000 7efff4005640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7effec104d30 0x7effec19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:58.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.612+0000 7efff4005640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effec109a30 0x7effec1a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:58.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.612+0000 7efff4005640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7effec10f520 con 0x7effec103af0 2024-04-25T11:25:58.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.612+0000 7efff4005640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7effec10f3a0 con 0x7effec104d30 2024-04-25T11:25:58.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.612+0000 7efff4005640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7effec10f6a0 con 0x7effec109a30 2024-04-25T11:25:58.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.612+0000 7efff1da1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7effec103af0 0x7effec19a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:58.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.612+0000 7efff1da1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7effec103af0 0x7effec19a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57054/0 (socket says 172.21.15.69:57054) 2024-04-25T11:25:58.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.612+0000 7efff1da1640 1 -- 172.21.15.69:0/4124117534 learned_addr learned my addr 172.21.15.69:0/4124117534 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:58.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.612+0000 7efff25a2640 1 --2- 172.21.15.69:0/4124117534 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effec109a30 0x7effec1a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:58.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.612+0000 7efff15a0640 1 --2- 172.21.15.69:0/4124117534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7effec104d30 0x7effec19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:58.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.613+0000 7efff1da1640 1 -- 172.21.15.69:0/4124117534 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effec109a30 msgr2=0x7effec1a15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:58.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.613+0000 7efff1da1640 1 --2- 172.21.15.69:0/4124117534 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effec109a30 0x7effec1a15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:58.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.613+0000 7efff1da1640 1 -- 172.21.15.69:0/4124117534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7effec104d30 msgr2=0x7effec19a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:58.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.613+0000 7efff1da1640 1 --2- 172.21.15.69:0/4124117534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7effec104d30 0x7effec19a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:58.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.613+0000 7efff1da1640 1 -- 172.21.15.69:0/4124117534 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effec1a1b60 con 0x7effec103af0 2024-04-25T11:25:58.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.613+0000 7efff15a0640 1 --2- 172.21.15.69:0/4124117534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7effec104d30 0x7effec19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:25:58.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.613+0000 7efff1da1640 1 --2- 172.21.15.69:0/4124117534 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7effec103af0 0x7effec19a0d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7effe8005d00 tx=0x7effe800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:58.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.613+0000 7effe2ffd640 1 -- 172.21.15.69:0/4124117534 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe8017070 con 0x7effec103af0 2024-04-25T11:25:58.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.613+0000 7effe2ffd640 1 -- 172.21.15.69:0/4124117534 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7effe8009c30 con 0x7effec103af0 2024-04-25T11:25:58.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.613+0000 7efff4005640 1 -- 172.21.15.69:0/4124117534 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effec1a1e50 con 0x7effec103af0 2024-04-25T11:25:58.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.613+0000 7effe2ffd640 1 -- 172.21.15.69:0/4124117534 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe800f620 con 0x7effec103af0 2024-04-25T11:25:58.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.613+0000 7efff4005640 1 -- 172.21.15.69:0/4124117534 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effec1a2260 con 0x7effec103af0 2024-04-25T11:25:58.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.614+0000 7efff4005640 1 -- 172.21.15.69:0/4124117534 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effec06a160 con 0x7effec103af0 2024-04-25T11:25:58.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.615+0000 7effe2ffd640 1 -- 172.21.15.69:0/4124117534 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7effe801a070 con 0x7effec103af0 2024-04-25T11:25:58.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.616+0000 7effe2ffd640 1 --2- 172.21.15.69:0/4124117534 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7effd80afaf0 0x7effd80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:58.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.616+0000 7effe2ffd640 1 -- 172.21.15.69:0/4124117534 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7effe80de9e0 con 0x7effec103af0 2024-04-25T11:25:58.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.616+0000 7efff15a0640 1 --2- 172.21.15.69:0/4124117534 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7effd80afaf0 0x7effd80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:58.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.616+0000 7efff15a0640 1 --2- 172.21.15.69:0/4124117534 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7effd80afaf0 0x7effd80b2000 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7effdc005fd0 tx=0x7effdc005f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:58.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.619+0000 7effe2ffd640 1 -- 172.21.15.69:0/4124117534 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7effe8009de0 con 0x7effec103af0 2024-04-25T11:25:58.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.795+0000 7efff4005640 1 -- 172.21.15.69:0/4124117534 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7effec02dc10 con 0x7effec103af0 2024-04-25T11:25:58.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.795+0000 7effe2ffd640 1 -- 172.21.15.69:0/4124117534 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) ==== 104+0+11 (secure 0 0 0) 0x7effe80a71f0 con 0x7effec103af0 2024-04-25T11:25:58.788 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-25T11:25:58.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.798+0000 7efff4005640 1 -- 172.21.15.69:0/4124117534 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7effd80afaf0 msgr2=0x7effd80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:58.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.798+0000 7efff4005640 1 --2- 172.21.15.69:0/4124117534 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7effd80afaf0 0x7effd80b2000 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7effdc005fd0 tx=0x7effdc005f40 comp rx=0 tx=0).stop 2024-04-25T11:25:58.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.798+0000 7efff4005640 1 -- 172.21.15.69:0/4124117534 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7effec103af0 msgr2=0x7effec19a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:58.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.798+0000 7efff4005640 1 --2- 172.21.15.69:0/4124117534 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7effec103af0 0x7effec19a0d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7effe8005d00 tx=0x7effe800dde0 comp rx=0 tx=0).stop 2024-04-25T11:25:58.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.798+0000 7efff4005640 1 -- 172.21.15.69:0/4124117534 shutdown_connections 2024-04-25T11:25:58.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.798+0000 7efff4005640 1 --2- 172.21.15.69:0/4124117534 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7effd80afaf0 0x7effd80b2000 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:58.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.798+0000 7efff4005640 1 --2- 172.21.15.69:0/4124117534 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effec109a30 0x7effec1a15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:58.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.798+0000 7efff4005640 1 --2- 172.21.15.69:0/4124117534 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7effec104d30 0x7effec19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:58.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.798+0000 7efff4005640 1 --2- 172.21.15.69:0/4124117534 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7effec103af0 0x7effec19a0d0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:58.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.798+0000 7efff4005640 1 -- 172.21.15.69:0/4124117534 >> 172.21.15.69:0/4124117534 conn(0x7effec0fece0 msgr2=0x7effec105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:58.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.798+0000 7efff4005640 1 -- 172.21.15.69:0/4124117534 shutdown_connections 2024-04-25T11:25:58.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.798+0000 7efff4005640 1 -- 172.21.15.69:0/4124117534 wait complete. 2024-04-25T11:25:58.801 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:25:58.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.919+0000 7f10cdf38640 1 -- 172.21.15.69:0/831999381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f10c8109a30 msgr2=0x7f10c810be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:58.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.919+0000 7f10cdf38640 1 --2- 172.21.15.69:0/831999381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f10c8109a30 0x7f10c810be50 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f10c0009900 tx=0x7f10c002a410 comp rx=0 tx=0).stop 2024-04-25T11:25:58.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.919+0000 7f10cdf38640 1 -- 172.21.15.69:0/831999381 shutdown_connections 2024-04-25T11:25:58.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.919+0000 7f10cdf38640 1 --2- 172.21.15.69:0/831999381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f10c8109a30 0x7f10c810be50 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:58.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.919+0000 7f10cdf38640 1 --2- 172.21.15.69:0/831999381 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f10c8104d30 0x7f10c8105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:58.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.919+0000 7f10cdf38640 1 --2- 172.21.15.69:0/831999381 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10c8103af0 0x7f10c81047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:58.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.919+0000 7f10cdf38640 1 -- 172.21.15.69:0/831999381 >> 172.21.15.69:0/831999381 conn(0x7f10c80fece0 msgr2=0x7f10c8101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:58.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.919+0000 7f10cdf38640 1 -- 172.21.15.69:0/831999381 shutdown_connections 2024-04-25T11:25:58.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.919+0000 7f10cdf38640 1 -- 172.21.15.69:0/831999381 wait complete. 2024-04-25T11:25:58.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.920+0000 7f10cdf38640 1 Processor -- start 2024-04-25T11:25:58.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.920+0000 7f10cdf38640 1 -- start start 2024-04-25T11:25:58.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.920+0000 7f10cdf38640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f10c8103af0 0x7f10c819a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:58.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.920+0000 7f10cdf38640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f10c8104d30 0x7f10c819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:58.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.920+0000 7f10cdf38640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10c8109a30 0x7f10c81a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:58.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.920+0000 7f10cdf38640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f10c810f4d0 con 0x7f10c8104d30 2024-04-25T11:25:58.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.920+0000 7f10cdf38640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f10c810f350 con 0x7f10c8109a30 2024-04-25T11:25:58.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.920+0000 7f10cdf38640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f10c810f650 con 0x7f10c8103af0 2024-04-25T11:25:58.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.920+0000 7f10c6ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f10c8104d30 0x7f10c819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:58.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.920+0000 7f10c6ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f10c8104d30 0x7f10c819a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57074/0 (socket says 172.21.15.69:57074) 2024-04-25T11:25:58.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.920+0000 7f10c6ffd640 1 -- 172.21.15.69:0/1269093869 learned_addr learned my addr 172.21.15.69:0/1269093869 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:58.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.920+0000 7f10c77fe640 1 --2- 172.21.15.69:0/1269093869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f10c8103af0 0x7f10c819a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:58.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.920+0000 7f10c7fff640 1 --2- 172.21.15.69:0/1269093869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10c8109a30 0x7f10c81a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:58.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.920+0000 7f10c6ffd640 1 -- 172.21.15.69:0/1269093869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f10c8103af0 msgr2=0x7f10c819a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:58.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.920+0000 7f10c6ffd640 1 --2- 172.21.15.69:0/1269093869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f10c8103af0 0x7f10c819a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:58.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.920+0000 7f10c6ffd640 1 -- 172.21.15.69:0/1269093869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10c8109a30 msgr2=0x7f10c81a15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:58.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.920+0000 7f10c6ffd640 1 --2- 172.21.15.69:0/1269093869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10c8109a30 0x7f10c81a15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:58.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.920+0000 7f10c6ffd640 1 -- 172.21.15.69:0/1269093869 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10c81a1b60 con 0x7f10c8104d30 2024-04-25T11:25:58.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.920+0000 7f10c7fff640 1 --2- 172.21.15.69:0/1269093869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10c8109a30 0x7f10c81a15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:25:58.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.921+0000 7f10c6ffd640 1 --2- 172.21.15.69:0/1269093869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f10c8104d30 0x7f10c819a610 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f10bc00d770 tx=0x7f10bc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:58.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.921+0000 7f10c4ff9640 1 -- 172.21.15.69:0/1269093869 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10bc017070 con 0x7f10c8104d30 2024-04-25T11:25:58.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.921+0000 7f10c4ff9640 1 -- 172.21.15.69:0/1269093869 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f10bc0054b0 con 0x7f10c8104d30 2024-04-25T11:25:58.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.921+0000 7f10cdf38640 1 -- 172.21.15.69:0/1269093869 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10c81a1e50 con 0x7f10c8104d30 2024-04-25T11:25:58.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.921+0000 7f10c4ff9640 1 -- 172.21.15.69:0/1269093869 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10bc004c60 con 0x7f10c8104d30 2024-04-25T11:25:58.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.921+0000 7f10cdf38640 1 -- 172.21.15.69:0/1269093869 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10c81a22c0 con 0x7f10c8104d30 2024-04-25T11:25:58.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.922+0000 7f10cdf38640 1 -- 172.21.15.69:0/1269093869 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10c8108840 con 0x7f10c8104d30 2024-04-25T11:25:58.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.923+0000 7f10c4ff9640 1 -- 172.21.15.69:0/1269093869 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f10bc004e00 con 0x7f10c8104d30 2024-04-25T11:25:58.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.924+0000 7f10c4ff9640 1 --2- 172.21.15.69:0/1269093869 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f10b00afaf0 0x7f10b00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:58.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.924+0000 7f10c77fe640 1 --2- 172.21.15.69:0/1269093869 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f10b00afaf0 0x7f10b00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:58.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.924+0000 7f10c4ff9640 1 -- 172.21.15.69:0/1269093869 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f10bc0deeb0 con 0x7f10c8104d30 2024-04-25T11:25:58.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.924+0000 7f10c77fe640 1 --2- 172.21.15.69:0/1269093869 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f10b00afaf0 0x7f10b00b2000 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f10c81046f0 tx=0x7f10b8007600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:58.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:58.928+0000 7f10c4ff9640 1 -- 172.21.15.69:0/1269093869 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f10bc0a76c0 con 0x7f10c8104d30 2024-04-25T11:25:59.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.107+0000 7f10cdf38640 1 -- 172.21.15.69:0/1269093869 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f10c802dce0 con 0x7f10c8104d30 2024-04-25T11:25:59.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.107+0000 7f10c4ff9640 1 -- 172.21.15.69:0/1269093869 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) ==== 100+0+11 (secure 0 0 0) 0x7f10bc0a6fc0 con 0x7f10c8104d30 2024-04-25T11:25:59.100 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-04-25T11:25:59.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.109+0000 7f10cdf38640 1 -- 172.21.15.69:0/1269093869 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f10b00afaf0 msgr2=0x7f10b00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:59.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.109+0000 7f10cdf38640 1 --2- 172.21.15.69:0/1269093869 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f10b00afaf0 0x7f10b00b2000 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f10c81046f0 tx=0x7f10b8007600 comp rx=0 tx=0).stop 2024-04-25T11:25:59.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.109+0000 7f10cdf38640 1 -- 172.21.15.69:0/1269093869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f10c8104d30 msgr2=0x7f10c819a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:59.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.109+0000 7f10cdf38640 1 --2- 172.21.15.69:0/1269093869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f10c8104d30 0x7f10c819a610 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f10bc00d770 tx=0x7f10bc00dc30 comp rx=0 tx=0).stop 2024-04-25T11:25:59.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.109+0000 7f10cdf38640 1 -- 172.21.15.69:0/1269093869 shutdown_connections 2024-04-25T11:25:59.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.109+0000 7f10cdf38640 1 --2- 172.21.15.69:0/1269093869 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f10b00afaf0 0x7f10b00b2000 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:59.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.109+0000 7f10cdf38640 1 --2- 172.21.15.69:0/1269093869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10c8109a30 0x7f10c81a15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:59.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.109+0000 7f10cdf38640 1 --2- 172.21.15.69:0/1269093869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f10c8104d30 0x7f10c819a610 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:59.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.109+0000 7f10cdf38640 1 --2- 172.21.15.69:0/1269093869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f10c8103af0 0x7f10c819a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:59.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.109+0000 7f10cdf38640 1 -- 172.21.15.69:0/1269093869 >> 172.21.15.69:0/1269093869 conn(0x7f10c80fece0 msgr2=0x7f10c8105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:59.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.110+0000 7f10cdf38640 1 -- 172.21.15.69:0/1269093869 shutdown_connections 2024-04-25T11:25:59.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.110+0000 7f10cdf38640 1 -- 172.21.15.69:0/1269093869 wait complete. 2024-04-25T11:25:59.112 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:58 smithi069 ceph-mon[29188]: pgmap v357: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:25:59.112 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:58 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/305737777' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:25:59.112 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:58 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/3885166571' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:25:59.112 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:58 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/4124117534' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:25:59.113 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T11:25:59.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.232+0000 7f470c9d6640 1 -- 172.21.15.69:0/426568666 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4708108a30 msgr2=0x7f470810ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:59.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.232+0000 7f470c9d6640 1 --2- 172.21.15.69:0/426568666 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4708108a30 0x7f470810ae50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4700009ca0 tx=0x7f470002a8c0 comp rx=0 tx=0).stop 2024-04-25T11:25:59.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.232+0000 7f470c9d6640 1 -- 172.21.15.69:0/426568666 shutdown_connections 2024-04-25T11:25:59.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.232+0000 7f470c9d6640 1 --2- 172.21.15.69:0/426568666 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4708108a30 0x7f470810ae50 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:59.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.232+0000 7f470c9d6640 1 --2- 172.21.15.69:0/426568666 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47081043e0 0x7f4708102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:59.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.232+0000 7f470c9d6640 1 --2- 172.21.15.69:0/426568666 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4708103f00 0x7f47081027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:59.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.232+0000 7f470c9d6640 1 -- 172.21.15.69:0/426568666 >> 172.21.15.69:0/426568666 conn(0x7f47080fd380 msgr2=0x7f47080ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:59.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.232+0000 7f470c9d6640 1 -- 172.21.15.69:0/426568666 shutdown_connections 2024-04-25T11:25:59.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.232+0000 7f470c9d6640 1 -- 172.21.15.69:0/426568666 wait complete. 2024-04-25T11:25:59.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.233+0000 7f470c9d6640 1 Processor -- start 2024-04-25T11:25:59.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.233+0000 7f470c9d6640 1 -- start start 2024-04-25T11:25:59.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.233+0000 7f470c9d6640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4708103f00 0x7f4708198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:59.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.233+0000 7f470c9d6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4708108a30 0x7f4708199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:59.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.233+0000 7f470c9d6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f470819e0c0 0x7f47081a04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:59.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.233+0000 7f470c9d6640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f470810e440 con 0x7f470819e0c0 2024-04-25T11:25:59.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.233+0000 7f470c9d6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f470810e2c0 con 0x7f4708108a30 2024-04-25T11:25:59.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.233+0000 7f470c9d6640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f470810e5c0 con 0x7f4708103f00 2024-04-25T11:25:59.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.233+0000 7f4706d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f470819e0c0 0x7f47081a04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:59.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.233+0000 7f4706d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f470819e0c0 0x7f47081a04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57088/0 (socket says 172.21.15.69:57088) 2024-04-25T11:25:59.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.233+0000 7f4706d9d640 1 -- 172.21.15.69:0/1048758707 learned_addr learned my addr 172.21.15.69:0/1048758707 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:59.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.233+0000 7f4705d9b640 1 --2- 172.21.15.69:0/1048758707 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4708108a30 0x7f4708199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:59.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.234+0000 7f470659c640 1 --2- 172.21.15.69:0/1048758707 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4708103f00 0x7f4708198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:59.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.234+0000 7f4706d9d640 1 -- 172.21.15.69:0/1048758707 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4708103f00 msgr2=0x7f4708198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:59.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.234+0000 7f4706d9d640 1 --2- 172.21.15.69:0/1048758707 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4708103f00 0x7f4708198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:59.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.234+0000 7f4706d9d640 1 -- 172.21.15.69:0/1048758707 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4708108a30 msgr2=0x7f4708199140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:59.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.234+0000 7f4706d9d640 1 --2- 172.21.15.69:0/1048758707 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4708108a30 0x7f4708199140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:59.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.234+0000 7f4706d9d640 1 -- 172.21.15.69:0/1048758707 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47081a0b50 con 0x7f470819e0c0 2024-04-25T11:25:59.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.234+0000 7f4706d9d640 1 --2- 172.21.15.69:0/1048758707 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f470819e0c0 0x7f47081a04e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4700031040 tx=0x7f4700003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:59.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.234+0000 7f46f77fe640 1 -- 172.21.15.69:0/1048758707 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4700004040 con 0x7f470819e0c0 2024-04-25T11:25:59.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.234+0000 7f46f77fe640 1 -- 172.21.15.69:0/1048758707 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4700004670 con 0x7f470819e0c0 2024-04-25T11:25:59.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.234+0000 7f470c9d6640 1 -- 172.21.15.69:0/1048758707 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47081a0de0 con 0x7f470819e0c0 2024-04-25T11:25:59.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.234+0000 7f46f77fe640 1 -- 172.21.15.69:0/1048758707 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4700030d50 con 0x7f470819e0c0 2024-04-25T11:25:59.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.234+0000 7f470c9d6640 1 -- 172.21.15.69:0/1048758707 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47081a12a0 con 0x7f470819e0c0 2024-04-25T11:25:59.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.235+0000 7f470c9d6640 1 -- 172.21.15.69:0/1048758707 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4708107130 con 0x7f470819e0c0 2024-04-25T11:25:59.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:58 smithi152 ceph-mon[32701]: pgmap v357: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:25:59.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:58 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/305737777' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:25:59.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:58 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/3885166571' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:25:59.231 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:58 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/4124117534' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:25:59.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.237+0000 7f46f77fe640 1 -- 172.21.15.69:0/1048758707 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f47000305c0 con 0x7f470819e0c0 2024-04-25T11:25:59.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.237+0000 7f46f77fe640 1 --2- 172.21.15.69:0/1048758707 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f46f00afb70 0x7f46f00b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:59.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.237+0000 7f470659c640 1 --2- 172.21.15.69:0/1048758707 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f46f00afb70 0x7f46f00b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:59.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.237+0000 7f46f77fe640 1 -- 172.21.15.69:0/1048758707 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f47001006a0 con 0x7f470819e0c0 2024-04-25T11:25:59.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.238+0000 7f470659c640 1 --2- 172.21.15.69:0/1048758707 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f46f00afb70 0x7f46f00b2080 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f46f8005fd0 tx=0x7f46f8005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:59.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.241+0000 7f46f77fe640 1 -- 172.21.15.69:0/1048758707 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f47000c8eb0 con 0x7f470819e0c0 2024-04-25T11:25:59.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:58 smithi146 ceph-mon[32821]: pgmap v357: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:25:59.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:58 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/305737777' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:25:59.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:58 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/3885166571' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:25:59.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:58 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/4124117534' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:25:59.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.417+0000 7f470c9d6640 1 -- 172.21.15.69:0/1048758707 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f470802dce0 con 0x7f470819e0c0 2024-04-25T11:25:59.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.417+0000 7f46f77fe640 1 -- 172.21.15.69:0/1048758707 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v84) ==== 103+0+11 (secure 0 0 0) 0x7f47000c87b0 con 0x7f470819e0c0 2024-04-25T11:25:59.410 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-25T11:25:59.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.419+0000 7f470c9d6640 1 -- 172.21.15.69:0/1048758707 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f46f00afb70 msgr2=0x7f46f00b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:59.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.419+0000 7f470c9d6640 1 --2- 172.21.15.69:0/1048758707 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f46f00afb70 0x7f46f00b2080 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f46f8005fd0 tx=0x7f46f8005e20 comp rx=0 tx=0).stop 2024-04-25T11:25:59.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.419+0000 7f470c9d6640 1 -- 172.21.15.69:0/1048758707 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f470819e0c0 msgr2=0x7f47081a04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:59.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.419+0000 7f470c9d6640 1 --2- 172.21.15.69:0/1048758707 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f470819e0c0 0x7f47081a04e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4700031040 tx=0x7f4700003ba0 comp rx=0 tx=0).stop 2024-04-25T11:25:59.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.419+0000 7f470c9d6640 1 -- 172.21.15.69:0/1048758707 shutdown_connections 2024-04-25T11:25:59.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.419+0000 7f470c9d6640 1 --2- 172.21.15.69:0/1048758707 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f46f00afb70 0x7f46f00b2080 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:59.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.419+0000 7f470c9d6640 1 --2- 172.21.15.69:0/1048758707 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f470819e0c0 0x7f47081a04e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:59.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.419+0000 7f470c9d6640 1 --2- 172.21.15.69:0/1048758707 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4708108a30 0x7f4708199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:59.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.419+0000 7f470c9d6640 1 --2- 172.21.15.69:0/1048758707 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4708103f00 0x7f4708198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:59.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.419+0000 7f470c9d6640 1 -- 172.21.15.69:0/1048758707 >> 172.21.15.69:0/1048758707 conn(0x7f47080fd380 msgr2=0x7f47080ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:59.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.420+0000 7f470c9d6640 1 -- 172.21.15.69:0/1048758707 shutdown_connections 2024-04-25T11:25:59.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.420+0000 7f470c9d6640 1 -- 172.21.15.69:0/1048758707 wait complete. 2024-04-25T11:25:59.423 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:25:59.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.538+0000 7fb48136c640 1 -- 172.21.15.69:0/1266187105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb47c1044d0 msgr2=0x7fb47c10ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:59.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.538+0000 7fb48136c640 1 --2- 172.21.15.69:0/1266187105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb47c1044d0 0x7fb47c10ac20 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb474009ca0 tx=0x7fb47402a8c0 comp rx=0 tx=0).stop 2024-04-25T11:25:59.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.538+0000 7fb48136c640 1 -- 172.21.15.69:0/1266187105 shutdown_connections 2024-04-25T11:25:59.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.538+0000 7fb48136c640 1 --2- 172.21.15.69:0/1266187105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb47c1044d0 0x7fb47c10ac20 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:59.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.538+0000 7fb48136c640 1 --2- 172.21.15.69:0/1266187105 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb47c1039b0 0x7fb47c103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:59.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.538+0000 7fb48136c640 1 --2- 172.21.15.69:0/1266187105 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb47c102fd0 0x7fb47c1033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:59.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.538+0000 7fb48136c640 1 -- 172.21.15.69:0/1266187105 >> 172.21.15.69:0/1266187105 conn(0x7fb47c0fd3b0 msgr2=0x7fb47c0ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:59.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.538+0000 7fb48136c640 1 -- 172.21.15.69:0/1266187105 shutdown_connections 2024-04-25T11:25:59.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.538+0000 7fb48136c640 1 -- 172.21.15.69:0/1266187105 wait complete. 2024-04-25T11:25:59.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.539+0000 7fb48136c640 1 Processor -- start 2024-04-25T11:25:59.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.539+0000 7fb48136c640 1 -- start start 2024-04-25T11:25:59.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.539+0000 7fb48136c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb47c102fd0 0x7fb47c102290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:59.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.539+0000 7fb48136c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb47c1039b0 0x7fb47c19ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:59.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.539+0000 7fb48136c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb47c1044d0 0x7fb47c1a1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:59.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.539+0000 7fb48136c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb47c10e410 con 0x7fb47c1039b0 2024-04-25T11:25:59.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.539+0000 7fb48136c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb47c10e290 con 0x7fb47c1044d0 2024-04-25T11:25:59.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.539+0000 7fb48136c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb47c10e590 con 0x7fb47c102fd0 2024-04-25T11:25:59.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.539+0000 7fb47a7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb47c1039b0 0x7fb47c19ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:59.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.539+0000 7fb47a7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb47c1039b0 0x7fb47c19ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57102/0 (socket says 172.21.15.69:57102) 2024-04-25T11:25:59.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.539+0000 7fb47a7fc640 1 -- 172.21.15.69:0/3601579288 learned_addr learned my addr 172.21.15.69:0/3601579288 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:59.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.539+0000 7fb47b7fe640 1 --2- 172.21.15.69:0/3601579288 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb47c1044d0 0x7fb47c1a1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:59.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.539+0000 7fb47affd640 1 --2- 172.21.15.69:0/3601579288 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb47c102fd0 0x7fb47c102290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:59.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.539+0000 7fb47a7fc640 1 -- 172.21.15.69:0/3601579288 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb47c102fd0 msgr2=0x7fb47c102290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:59.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.539+0000 7fb47a7fc640 1 --2- 172.21.15.69:0/3601579288 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb47c102fd0 0x7fb47c102290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:59.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.539+0000 7fb47a7fc640 1 -- 172.21.15.69:0/3601579288 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb47c1044d0 msgr2=0x7fb47c1a1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:59.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.539+0000 7fb47a7fc640 1 --2- 172.21.15.69:0/3601579288 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb47c1044d0 0x7fb47c1a1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:59.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.540+0000 7fb47a7fc640 1 -- 172.21.15.69:0/3601579288 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb47c1a1b70 con 0x7fb47c1039b0 2024-04-25T11:25:59.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.540+0000 7fb47b7fe640 1 --2- 172.21.15.69:0/3601579288 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb47c1044d0 0x7fb47c1a1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:25:59.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.540+0000 7fb47a7fc640 1 --2- 172.21.15.69:0/3601579288 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb47c1039b0 0x7fb47c19ede0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb47000c890 tx=0x7fb47000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:59.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.540+0000 7fb463fff640 1 -- 172.21.15.69:0/3601579288 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb470004220 con 0x7fb47c1039b0 2024-04-25T11:25:59.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.540+0000 7fb463fff640 1 -- 172.21.15.69:0/3601579288 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb470007e40 con 0x7fb47c1039b0 2024-04-25T11:25:59.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.540+0000 7fb48136c640 1 -- 172.21.15.69:0/3601579288 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb47c1a1e60 con 0x7fb47c1039b0 2024-04-25T11:25:59.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.540+0000 7fb463fff640 1 -- 172.21.15.69:0/3601579288 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb470004a00 con 0x7fb47c1039b0 2024-04-25T11:25:59.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.540+0000 7fb48136c640 1 -- 172.21.15.69:0/3601579288 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb47c1a22d0 con 0x7fb47c1039b0 2024-04-25T11:25:59.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.541+0000 7fb48136c640 1 -- 172.21.15.69:0/3601579288 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb47c109980 con 0x7fb47c1039b0 2024-04-25T11:25:59.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.543+0000 7fb463fff640 1 -- 172.21.15.69:0/3601579288 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fb4700079b0 con 0x7fb47c1039b0 2024-04-25T11:25:59.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.544+0000 7fb463fff640 1 --2- 172.21.15.69:0/3601579288 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fb4640afaf0 0x7fb4640b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:59.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.544+0000 7fb47affd640 1 --2- 172.21.15.69:0/3601579288 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fb4640afaf0 0x7fb4640b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:59.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.544+0000 7fb463fff640 1 -- 172.21.15.69:0/3601579288 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7fb4700df210 con 0x7fb47c1039b0 2024-04-25T11:25:59.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.544+0000 7fb47affd640 1 --2- 172.21.15.69:0/3601579288 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fb4640afaf0 0x7fb4640b2000 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb46c005fd0 tx=0x7fb46c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:59.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.547+0000 7fb463fff640 1 -- 172.21.15.69:0/3601579288 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb470014030 con 0x7fb47c1039b0 2024-04-25T11:25:59.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.760+0000 7fb48136c640 1 -- 172.21.15.69:0/3601579288 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb47c02db50 con 0x7fb47c1039b0 2024-04-25T11:25:59.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.761+0000 7fb463fff640 1 -- 172.21.15.69:0/3601579288 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11944 (secure 0 0 0) 0x7fb4700a79e0 con 0x7fb47c1039b0 2024-04-25T11:25:59.754 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:25:59.754 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3399424094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3399424094},{"type":"v1","addr":"172.21.15.69:6835","nonce":3399424094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1463217771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1463217771},{"type":"v1","addr":"172.21.15.69:6837","nonce":1463217771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T11:23:40.984930+0000","modified":"2024-04-25T11:24:09.061188+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24485,"mds_2":24475,"mds_3":24458,"mds_4":14625},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6839/966316001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":966316001},{"type":"v1","addr":"172.21.15.69:6839","nonce":966316001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.69:6841/2060225181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":2060225181},{"type":"v1","addr":"172.21.15.69:6841","nonce":2060225181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6833/4197482473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":4197482473},{"type":"v1","addr":"172.21.15.152:6833","nonce":4197482473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/4267885323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4267885323},{"type":"v1","addr":"172.21.15.146:6833","nonce":4267885323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6835/3188843362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3188843362},{"type":"v1","addr":"172.21.15.152:6835","nonce":3188843362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6837/563274982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":563274982},{"type":"v1","addr":"172.21.15.152:6837","nonce":563274982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.146:6835/1876533699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1876533699},{"type":"v1","addr":"172.21.15.146:6835","nonce":1876533699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.152:6839/3008615657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":3008615657},{"type":"v1","addr":"172.21.15.152:6839","nonce":3008615657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.146:6837/2730317134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2730317134},{"type":"v1","addr":"172.21.15.146:6837","nonce":2730317134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6839/2013283448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":2013283448},{"type":"v1","addr":"172.21.15.146:6839","nonce":2013283448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[24485,24515,24475,24458,14625]},"id":1}]} 2024-04-25T11:25:59.754 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 28 2024-04-25T11:25:59.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.763+0000 7fb48136c640 1 -- 172.21.15.69:0/3601579288 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fb4640afaf0 msgr2=0x7fb4640b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:59.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.763+0000 7fb48136c640 1 --2- 172.21.15.69:0/3601579288 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fb4640afaf0 0x7fb4640b2000 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb46c005fd0 tx=0x7fb46c005850 comp rx=0 tx=0).stop 2024-04-25T11:25:59.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.763+0000 7fb48136c640 1 -- 172.21.15.69:0/3601579288 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb47c1039b0 msgr2=0x7fb47c19ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:59.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.763+0000 7fb48136c640 1 --2- 172.21.15.69:0/3601579288 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb47c1039b0 0x7fb47c19ede0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb47000c890 tx=0x7fb47000cd50 comp rx=0 tx=0).stop 2024-04-25T11:25:59.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.763+0000 7fb48136c640 1 -- 172.21.15.69:0/3601579288 shutdown_connections 2024-04-25T11:25:59.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.763+0000 7fb48136c640 1 --2- 172.21.15.69:0/3601579288 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fb4640afaf0 0x7fb4640b2000 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:59.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.763+0000 7fb48136c640 1 --2- 172.21.15.69:0/3601579288 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb47c1044d0 0x7fb47c1a1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:59.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.763+0000 7fb48136c640 1 --2- 172.21.15.69:0/3601579288 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb47c1039b0 0x7fb47c19ede0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:59.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.763+0000 7fb48136c640 1 --2- 172.21.15.69:0/3601579288 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb47c102fd0 0x7fb47c102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:59.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.763+0000 7fb48136c640 1 -- 172.21.15.69:0/3601579288 >> 172.21.15.69:0/3601579288 conn(0x7fb47c0fd3b0 msgr2=0x7fb47c106b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:59.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.763+0000 7fb48136c640 1 -- 172.21.15.69:0/3601579288 shutdown_connections 2024-04-25T11:25:59.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.763+0000 7fb48136c640 1 -- 172.21.15.69:0/3601579288 wait complete. 2024-04-25T11:25:59.768 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:25:59.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.884+0000 7f1ac9fc5640 1 -- 172.21.15.69:0/3855169394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ac4102a00 msgr2=0x7f1ac4103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:59.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.884+0000 7f1ac9fc5640 1 --2- 172.21.15.69:0/3855169394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ac4102a00 0x7f1ac4103700 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4009900 tx=0x7f1ab402a410 comp rx=0 tx=0).stop 2024-04-25T11:25:59.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.884+0000 7f1ac9fc5640 1 -- 172.21.15.69:0/3855169394 shutdown_connections 2024-04-25T11:25:59.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.884+0000 7f1ac9fc5640 1 --2- 172.21.15.69:0/3855169394 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ac4108940 0x7f1ac410ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:59.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.884+0000 7f1ac9fc5640 1 --2- 172.21.15.69:0/3855169394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ac4103c40 0x7f1ac4104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:59.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.884+0000 7f1ac9fc5640 1 --2- 172.21.15.69:0/3855169394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ac4102a00 0x7f1ac4103700 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:59.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.884+0000 7f1ac9fc5640 1 -- 172.21.15.69:0/3855169394 >> 172.21.15.69:0/3855169394 conn(0x7f1ac40fdbf0 msgr2=0x7f1ac41000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:25:59.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.884+0000 7f1ac9fc5640 1 -- 172.21.15.69:0/3855169394 shutdown_connections 2024-04-25T11:25:59.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.884+0000 7f1ac9fc5640 1 -- 172.21.15.69:0/3855169394 wait complete. 2024-04-25T11:25:59.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.885+0000 7f1ac9fc5640 1 Processor -- start 2024-04-25T11:25:59.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.885+0000 7f1ac9fc5640 1 -- start start 2024-04-25T11:25:59.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.885+0000 7f1ac9fc5640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ac4102a00 0x7f1ac4075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:59.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.885+0000 7f1ac9fc5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ac4103c40 0x7f1ac419eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:59.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.885+0000 7f1ac9fc5640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ac4108940 0x7f1ac41a1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:59.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.885+0000 7f1ac9fc5640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac410e390 con 0x7f1ac4103c40 2024-04-25T11:25:59.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.885+0000 7f1ac9fc5640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac410e210 con 0x7f1ac4102a00 2024-04-25T11:25:59.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.885+0000 7f1ac9fc5640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac410e510 con 0x7f1ac4108940 2024-04-25T11:25:59.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.885+0000 7f1ac2ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ac4103c40 0x7f1ac419eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:59.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.885+0000 7f1ac2ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ac4103c40 0x7f1ac419eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57120/0 (socket says 172.21.15.69:57120) 2024-04-25T11:25:59.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.885+0000 7f1ac2ffd640 1 -- 172.21.15.69:0/5448446 learned_addr learned my addr 172.21.15.69:0/5448446 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:25:59.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.885+0000 7f1ac37fe640 1 --2- 172.21.15.69:0/5448446 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ac4102a00 0x7f1ac4075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:59.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.885+0000 7f1ac3fff640 1 --2- 172.21.15.69:0/5448446 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ac4108940 0x7f1ac41a1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:59.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.886+0000 7f1ac2ffd640 1 -- 172.21.15.69:0/5448446 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ac4108940 msgr2=0x7f1ac41a1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:59.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.886+0000 7f1ac2ffd640 1 --2- 172.21.15.69:0/5448446 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ac4108940 0x7f1ac41a1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:59.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.886+0000 7f1ac2ffd640 1 -- 172.21.15.69:0/5448446 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ac4102a00 msgr2=0x7f1ac4075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:25:59.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.886+0000 7f1ac2ffd640 1 --2- 172.21.15.69:0/5448446 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ac4102a00 0x7f1ac4075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:25:59.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.886+0000 7f1ac2ffd640 1 -- 172.21.15.69:0/5448446 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac41a1bc0 con 0x7f1ac4103c40 2024-04-25T11:25:59.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.886+0000 7f1ac2ffd640 1 --2- 172.21.15.69:0/5448446 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ac4103c40 0x7f1ac419eee0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1ab800c920 tx=0x7f1ab800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:59.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.886+0000 7f1ac0ff9640 1 -- 172.21.15.69:0/5448446 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8007c70 con 0x7f1ac4103c40 2024-04-25T11:25:59.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.886+0000 7f1ac0ff9640 1 -- 172.21.15.69:0/5448446 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ab8007e10 con 0x7f1ac4103c40 2024-04-25T11:25:59.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.886+0000 7f1ac9fc5640 1 -- 172.21.15.69:0/5448446 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ac41a1eb0 con 0x7f1ac4103c40 2024-04-25T11:25:59.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.886+0000 7f1ac0ff9640 1 -- 172.21.15.69:0/5448446 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8018710 con 0x7f1ac4103c40 2024-04-25T11:25:59.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.886+0000 7f1ac9fc5640 1 -- 172.21.15.69:0/5448446 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ac41a2310 con 0x7f1ac4103c40 2024-04-25T11:25:59.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.887+0000 7f1ac9fc5640 1 -- 172.21.15.69:0/5448446 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ac4107750 con 0x7f1ac4103c40 2024-04-25T11:25:59.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.889+0000 7f1ac0ff9640 1 -- 172.21.15.69:0/5448446 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f1ab800f450 con 0x7f1ac4103c40 2024-04-25T11:25:59.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.889+0000 7f1ac0ff9640 1 --2- 172.21.15.69:0/5448446 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f1aac0afaf0 0x7f1aac0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:25:59.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.890+0000 7f1ac37fe640 1 --2- 172.21.15.69:0/5448446 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f1aac0afaf0 0x7f1aac0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:25:59.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.890+0000 7f1ac0ff9640 1 -- 172.21.15.69:0/5448446 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f1ab80df340 con 0x7f1ac4103c40 2024-04-25T11:25:59.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.890+0000 7f1ac37fe640 1 --2- 172.21.15.69:0/5448446 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f1aac0afaf0 0x7f1aac0b2000 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4005760 tx=0x7f1ab4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:25:59.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:25:59.894+0000 7f1ac0ff9640 1 -- 172.21.15.69:0/5448446 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1ab8014030 con 0x7f1ac4103c40 2024-04-25T11:26:00.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.072+0000 7f1ac9fc5640 1 -- 172.21.15.69:0/5448446 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1ac402dd20 con 0x7f1ac4103c40 2024-04-25T11:26:00.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.074+0000 7f1ac0ff9640 1 -- 172.21.15.69:0/5448446 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+24862 (secure 0 0 0) 0x7f1ab80a7be0 con 0x7f1ac4103c40 2024-04-25T11:26:00.067 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:26:00.067 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":84,"fsid":"f7ed73dc-02f4-11ef-bc93-c7b262605968","created":"2024-04-25T11:15:37.965281+0000","modified":"2024-04-25T11:25:33.888184+0000","last_up_change":"2024-04-25T11:22:50.691869+0000","last_in_change":"2024-04-25T11:22:30.271959+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:18:57.781263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:23:38.685110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:23:39.943803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T11:23:43.650521+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6a0d179-4998-4754-bf82-2ea3dac54e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6803","nonce":3756025235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6805","nonce":3756025235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6809","nonce":3756025235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6807","nonce":3756025235}]},"public_addr":"172.21.15.69:6803/3756025235","cluster_addr":"172.21.15.69:6805/3756025235","heartbeat_back_addr":"172.21.15.69:6809/3756025235","heartbeat_front_addr":"172.21.15.69:6807/3756025235","state":["exists","up"]},{"osd":1,"uuid":"c0c66930-4233-493f-b2bb-99c473994b59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6801","nonce":3166305213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6803","nonce":3166305213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6807","nonce":3166305213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6805","nonce":3166305213}]},"public_addr":"172.21.15.146:6801/3166305213","cluster_addr":"172.21.15.146:6803/3166305213","heartbeat_back_addr":"172.21.15.146:6807/3166305213","heartbeat_front_addr":"172.21.15.146:6805/3166305213","state":["exists","up"]},{"osd":2,"uuid":"d4ef735b-9786-4cb9-83ef-7c3f0691a729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6801","nonce":1474319175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6803","nonce":1474319175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6807","nonce":1474319175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6805","nonce":1474319175}]},"public_addr":"172.21.15.152:6801/1474319175","cluster_addr":"172.21.15.152:6803/1474319175","heartbeat_back_addr":"172.21.15.152:6807/1474319175","heartbeat_front_addr":"172.21.15.152:6805/1474319175","state":["exists","up"]},{"osd":3,"uuid":"0ab85fb8-5bbb-4ffc-bddf-cae7644a64ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6811","nonce":1809071296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6813","nonce":1809071296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6817","nonce":1809071296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6815","nonce":1809071296}]},"public_addr":"172.21.15.69:6811/1809071296","cluster_addr":"172.21.15.69:6813/1809071296","heartbeat_back_addr":"172.21.15.69:6817/1809071296","heartbeat_front_addr":"172.21.15.69:6815/1809071296","state":["exists","up"]},{"osd":4,"uuid":"273e0592-bbbc-4188-8eeb-46a3879cf632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6809","nonce":1898520972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6811","nonce":1898520972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6815","nonce":1898520972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6813","nonce":1898520972}]},"public_addr":"172.21.15.146:6809/1898520972","cluster_addr":"172.21.15.146:6811/1898520972","heartbeat_back_addr":"172.21.15.146:6815/1898520972","heartbeat_front_addr":"172.21.15.146:6813/1898520972","state":["exists","up"]},{"osd":5,"uuid":"bbeae945-9fa2-48f9-a1d6-8bf18dcaef19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6809","nonce":1996555108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6811","nonce":1996555108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6815","nonce":1996555108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6813","nonce":1996555108}]},"public_addr":"172.21.15.152:6809/1996555108","cluster_addr":"172.21.15.152:6811/1996555108","heartbeat_back_addr":"172.21.15.152:6815/1996555108","heartbeat_front_addr":"172.21.15.152:6813/1996555108","state":["exists","up"]},{"osd":6,"uuid":"dbb61253-30bc-418e-bd75-118c80bbf931","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6819","nonce":491215762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6821","nonce":491215762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6825","nonce":491215762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6823","nonce":491215762}]},"public_addr":"172.21.15.69:6819/491215762","cluster_addr":"172.21.15.69:6821/491215762","heartbeat_back_addr":"172.21.15.69:6825/491215762","heartbeat_front_addr":"172.21.15.69:6823/491215762","state":["exists","up"]},{"osd":7,"uuid":"75f23854-e283-4e3e-94cf-7b4079d83876","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6817","nonce":2330294690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6819","nonce":2330294690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6823","nonce":2330294690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6821","nonce":2330294690}]},"public_addr":"172.21.15.146:6817/2330294690","cluster_addr":"172.21.15.146:6819/2330294690","heartbeat_back_addr":"172.21.15.146:6823/2330294690","heartbeat_front_addr":"172.21.15.146:6821/2330294690","state":["exists","up"]},{"osd":8,"uuid":"1aa1060c-0c14-4abd-be56-5ebfeb78759c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6817","nonce":950669381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6819","nonce":950669381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6823","nonce":950669381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6821","nonce":950669381}]},"public_addr":"172.21.15.152:6817/950669381","cluster_addr":"172.21.15.152:6819/950669381","heartbeat_back_addr":"172.21.15.152:6823/950669381","heartbeat_front_addr":"172.21.15.152:6821/950669381","state":["exists","up"]},{"osd":9,"uuid":"3f4bf800-4090-4138-9bb3-d001f13f450c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6827","nonce":2414186758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6829","nonce":2414186758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6833","nonce":2414186758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6831","nonce":2414186758}]},"public_addr":"172.21.15.69:6827/2414186758","cluster_addr":"172.21.15.69:6829/2414186758","heartbeat_back_addr":"172.21.15.69:6833/2414186758","heartbeat_front_addr":"172.21.15.69:6831/2414186758","state":["exists","up"]},{"osd":10,"uuid":"19e967e5-51ce-4506-80c9-0a95441baf99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6825","nonce":707256400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6827","nonce":707256400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6831","nonce":707256400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6829","nonce":707256400}]},"public_addr":"172.21.15.146:6825/707256400","cluster_addr":"172.21.15.146:6827/707256400","heartbeat_back_addr":"172.21.15.146:6831/707256400","heartbeat_front_addr":"172.21.15.146:6829/707256400","state":["exists","up"]},{"osd":11,"uuid":"7788c2de-6930-4808-be0f-f1fd1ae0cb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6825","nonce":2934336833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6827","nonce":2934336833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6831","nonce":2934336833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6829","nonce":2934336833}]},"public_addr":"172.21.15.152:6825/2934336833","cluster_addr":"172.21.15.152:6827/2934336833","heartbeat_back_addr":"172.21.15.152:6831/2934336833","heartbeat_front_addr":"172.21.15.152:6829/2934336833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:07.497448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:31.227494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:55.027978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:20.713437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:46.214704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:20:11.199983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:20:37.177070+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:02.951240+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:28.963680+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:56.095807+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:22:22.221744+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:22:48.929334+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1b","mappings":[{"from":5,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/1372727546":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6801/1097124842":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6800/1508198215":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3103465990":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3540132960":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3899912190":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6800/1097124842":"2024-04-26T11:16:31.688159+0000","172.21.15.69:0/1199907202":"2024-04-26T11:16:04.860792+0000","172.21.15.69:6801/1508198215":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/2879102529":"2024-04-26T11:16:31.688159+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:26:00.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.076+0000 7f1ac9fc5640 1 -- 172.21.15.69:0/5448446 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f1aac0afaf0 msgr2=0x7f1aac0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:00.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.076+0000 7f1ac9fc5640 1 --2- 172.21.15.69:0/5448446 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f1aac0afaf0 0x7f1aac0b2000 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4005760 tx=0x7f1ab4000bc0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.076+0000 7f1ac9fc5640 1 -- 172.21.15.69:0/5448446 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ac4103c40 msgr2=0x7f1ac419eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:00.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.076+0000 7f1ac9fc5640 1 --2- 172.21.15.69:0/5448446 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ac4103c40 0x7f1ac419eee0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1ab800c920 tx=0x7f1ab800cde0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.076+0000 7f1ac9fc5640 1 -- 172.21.15.69:0/5448446 shutdown_connections 2024-04-25T11:26:00.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.076+0000 7f1ac9fc5640 1 --2- 172.21.15.69:0/5448446 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f1aac0afaf0 0x7f1aac0b2000 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.076+0000 7f1ac9fc5640 1 --2- 172.21.15.69:0/5448446 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ac4108940 0x7f1ac41a1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.076+0000 7f1ac9fc5640 1 --2- 172.21.15.69:0/5448446 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ac4103c40 0x7f1ac419eee0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.076+0000 7f1ac9fc5640 1 --2- 172.21.15.69:0/5448446 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ac4102a00 0x7f1ac4075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.076+0000 7f1ac9fc5640 1 -- 172.21.15.69:0/5448446 >> 172.21.15.69:0/5448446 conn(0x7f1ac40fdbf0 msgr2=0x7f1ac4104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:26:00.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.076+0000 7f1ac9fc5640 1 -- 172.21.15.69:0/5448446 shutdown_connections 2024-04-25T11:26:00.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.076+0000 7f1ac9fc5640 1 -- 172.21.15.69:0/5448446 wait complete. 2024-04-25T11:26:00.079 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:59 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/1269093869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:26:00.079 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:59 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/1048758707' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T11:26:00.080 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:59 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/3601579288' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:26:00.080 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:25:59 smithi069 ceph-mon[29188]: pgmap v358: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:26:00.082 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T11:23:40.984930+0000', 'modified': '2024-04-25T11:24:09.061188+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24515, 'mds_1': 24485, 'mds_2': 24475, 'mds_3': 24458, 'mds_4': 14625}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14613': {'gid': 14613, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.69:6839/966316001', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6838', 'nonce': 966316001}, {'type': 'v1', 'addr': '172.21.15.69:6839', 'nonce': 966316001}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14625': {'gid': 14625, 'name': 'a', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.69:6841/2060225181', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6840', 'nonce': 2060225181}, {'type': 'v1', 'addr': '172.21.15.69:6841', 'nonce': 2060225181}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24391': {'gid': 24391, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.152:6833/4197482473', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6832', 'nonce': 4197482473}, {'type': 'v1', 'addr': '172.21.15.152:6833', 'nonce': 4197482473}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24425': {'gid': 24425, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.146:6833/4267885323', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6832', 'nonce': 4267885323}, {'type': 'v1', 'addr': '172.21.15.146:6833', 'nonce': 4267885323}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24439': {'gid': 24439, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.152:6835/3188843362', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6834', 'nonce': 3188843362}, {'type': 'v1', 'addr': '172.21.15.152:6835', 'nonce': 3188843362}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.152:6837/563274982', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6836', 'nonce': 563274982}, {'type': 'v1', 'addr': '172.21.15.152:6837', 'nonce': 563274982}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24458': {'gid': 24458, 'name': 'e', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.146:6835/1876533699', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6834', 'nonce': 1876533699}, {'type': 'v1', 'addr': '172.21.15.146:6835', 'nonce': 1876533699}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24475': {'gid': 24475, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.152:6839/3008615657', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6838', 'nonce': 3008615657}, {'type': 'v1', 'addr': '172.21.15.152:6839', 'nonce': 3008615657}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24485': {'gid': 24485, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.146:6837/2730317134', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6836', 'nonce': 2730317134}, {'type': 'v1', 'addr': '172.21.15.146:6837', 'nonce': 2730317134}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24515': {'gid': 24515, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.146:6839/2013283448', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6838', 'nonce': 2013283448}, {'type': 'v1', 'addr': '172.21.15.146:6839', 'nonce': 2013283448}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24515, 'qdb_cluster': [24485, 24515, 24475, 24458, 14625]} 2024-04-25T11:26:00.082 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-25T11:26:00.083 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T11:26:00.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.199+0000 7f41587b2640 1 -- 172.21.15.69:0/3952158894 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4150108940 msgr2=0x7f415010ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:00.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.199+0000 7f41587b2640 1 --2- 172.21.15.69:0/3952158894 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4150108940 0x7f415010ad60 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4140009900 tx=0x7f414002a410 comp rx=0 tx=0).stop 2024-04-25T11:26:00.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.200+0000 7f41587b2640 1 -- 172.21.15.69:0/3952158894 shutdown_connections 2024-04-25T11:26:00.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.200+0000 7f41587b2640 1 --2- 172.21.15.69:0/3952158894 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4150108940 0x7f415010ad60 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.200+0000 7f41587b2640 1 --2- 172.21.15.69:0/3952158894 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4150103c40 0x7f4150104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.200+0000 7f41587b2640 1 --2- 172.21.15.69:0/3952158894 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4150102a00 0x7f4150103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.200+0000 7f41587b2640 1 -- 172.21.15.69:0/3952158894 >> 172.21.15.69:0/3952158894 conn(0x7f41500fdbf0 msgr2=0x7f41501000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:26:00.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.200+0000 7f41587b2640 1 -- 172.21.15.69:0/3952158894 shutdown_connections 2024-04-25T11:26:00.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.200+0000 7f41587b2640 1 -- 172.21.15.69:0/3952158894 wait complete. 2024-04-25T11:26:00.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.200+0000 7f41587b2640 1 Processor -- start 2024-04-25T11:26:00.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.200+0000 7f41587b2640 1 -- start start 2024-04-25T11:26:00.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.200+0000 7f41587b2640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4150102a00 0x7f415019a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.200+0000 7f41587b2640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4150103c40 0x7f415019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.200+0000 7f41587b2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4150108940 0x7f41501a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.200+0000 7f41587b2640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f415010e3e0 con 0x7f4150108940 2024-04-25T11:26:00.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.200+0000 7f41587b2640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f415010e260 con 0x7f4150102a00 2024-04-25T11:26:00.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.200+0000 7f41587b2640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f415010e560 con 0x7f4150103c40 2024-04-25T11:26:00.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.201+0000 7f4156d4f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4150108940 0x7f41501a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.201+0000 7f4156d4f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4150108940 0x7f41501a15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57134/0 (socket says 172.21.15.69:57134) 2024-04-25T11:26:00.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.201+0000 7f4156d4f640 1 -- 172.21.15.69:0/906224348 learned_addr learned my addr 172.21.15.69:0/906224348 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:26:00.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.201+0000 7f4155d4d640 1 --2- 172.21.15.69:0/906224348 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4150103c40 0x7f415019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.201+0000 7f4156d4f640 1 -- 172.21.15.69:0/906224348 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4150103c40 msgr2=0x7f415019a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:00.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.201+0000 7f4156d4f640 1 --2- 172.21.15.69:0/906224348 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4150103c40 0x7f415019a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.201+0000 7f415654e640 1 --2- 172.21.15.69:0/906224348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4150102a00 0x7f415019a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.201+0000 7f4156d4f640 1 -- 172.21.15.69:0/906224348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4150102a00 msgr2=0x7f415019a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:00.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.201+0000 7f4156d4f640 1 --2- 172.21.15.69:0/906224348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4150102a00 0x7f415019a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.201+0000 7f4156d4f640 1 -- 172.21.15.69:0/906224348 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41501a1c40 con 0x7f4150108940 2024-04-25T11:26:00.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.201+0000 7f415654e640 1 --2- 172.21.15.69:0/906224348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4150102a00 0x7f415019a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:26:00.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.201+0000 7f4156d4f640 1 --2- 172.21.15.69:0/906224348 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4150108940 0x7f41501a15d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4140005ab0 tx=0x7f41400042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:26:00.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.201+0000 7f414f7fe640 1 -- 172.21.15.69:0/906224348 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414003f030 con 0x7f4150108940 2024-04-25T11:26:00.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.201+0000 7f414f7fe640 1 -- 172.21.15.69:0/906224348 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4140003c90 con 0x7f4150108940 2024-04-25T11:26:00.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.201+0000 7f41587b2640 1 -- 172.21.15.69:0/906224348 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41501a1ed0 con 0x7f4150108940 2024-04-25T11:26:00.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.201+0000 7f414f7fe640 1 -- 172.21.15.69:0/906224348 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4140035070 con 0x7f4150108940 2024-04-25T11:26:00.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.201+0000 7f41587b2640 1 -- 172.21.15.69:0/906224348 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41501a2330 con 0x7f4150108940 2024-04-25T11:26:00.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.204+0000 7f415f668740 1 Processor -- start 2024-04-25T11:26:00.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.204+0000 7f414f7fe640 1 -- 172.21.15.69:0/906224348 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f4140042030 con 0x7f4150108940 2024-04-25T11:26:00.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.204+0000 7f415f668740 1 -- start start 2024-04-25T11:26:00.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.204+0000 7f415f668740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa7e4a2270 0x55aa7e843370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.204+0000 7f415f668740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa7e4793b0 0x55aa7e8438f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.204+0000 7f415f668740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e481590 0x55aa7e84a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.204+0000 7f415f668740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55aa7e7a8740 con 0x55aa7e481590 2024-04-25T11:26:00.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.204+0000 7f415f668740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55aa7e7443a0 con 0x55aa7e4a2270 2024-04-25T11:26:00.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.204+0000 7f415f668740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55aa7e7a4700 con 0x55aa7e4793b0 2024-04-25T11:26:00.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.204+0000 7f4155d4d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e481590 0x55aa7e84a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.204+0000 7f4155d4d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e481590 0x55aa7e84a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57142/0 (socket says 172.21.15.69:57142) 2024-04-25T11:26:00.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.204+0000 7f4155d4d640 1 -- 172.21.15.69:0/3039796695 learned_addr learned my addr 172.21.15.69:0/3039796695 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:26:00.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.204+0000 7f415654e640 1 --2- 172.21.15.69:0/3039796695 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa7e4793b0 0x55aa7e8438f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.205+0000 7f4155d4d640 1 --2- 172.21.15.69:0/3039796695 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa7e4a2270 0x55aa7e843370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.205+0000 7f414f7fe640 1 --2- 172.21.15.69:0/906224348 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f41380afb70 0x7f41380b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.205+0000 7f4155d4d640 1 -- 172.21.15.69:0/3039796695 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa7e4793b0 msgr2=0x55aa7e8438f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:26:00.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.205+0000 7f4155d4d640 1 --2- 172.21.15.69:0/3039796695 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa7e4793b0 0x55aa7e8438f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.205+0000 7f4155d4d640 1 -- 172.21.15.69:0/3039796695 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa7e4a2270 msgr2=0x55aa7e843370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:26:00.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.205+0000 7f4155d4d640 1 --2- 172.21.15.69:0/3039796695 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa7e4a2270 0x55aa7e843370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.205+0000 7f4155d4d640 1 -- 172.21.15.69:0/3039796695 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa7e760750 con 0x55aa7e481590 2024-04-25T11:26:00.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.205+0000 7f4156d4f640 1 --2- 172.21.15.69:0/906224348 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f41380afb70 0x7f41380b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.205+0000 7f4155d4d640 1 --2- 172.21.15.69:0/3039796695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e481590 0x55aa7e84a7f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f414400bfc0 tx=0x7f414400b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82432dc407d8d7d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:26:00.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.205+0000 7f414f7fe640 1 -- 172.21.15.69:0/906224348 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f41400ff670 con 0x7f4150108940 2024-04-25T11:26:00.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.205+0000 7f414cff9640 1 -- 172.21.15.69:0/3039796695 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4144019030 con 0x55aa7e481590 2024-04-25T11:26:00.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.205+0000 7f414cff9640 1 -- 172.21.15.69:0/3039796695 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4144011950 con 0x55aa7e481590 2024-04-25T11:26:00.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.205+0000 7f414cff9640 1 -- 172.21.15.69:0/3039796695 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414401d6e0 con 0x55aa7e481590 2024-04-25T11:26:00.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.206+0000 7f415f668740 1 -- 172.21.15.69:0/3039796695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e481590 msgr2=0x55aa7e84a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:00.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.206+0000 7f4156d4f640 1 --2- 172.21.15.69:0/906224348 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f41380afb70 0x7f41380b2080 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f41400b9eb0 tx=0x7f41400f5ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:26:00.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.206+0000 7f415f668740 1 --2- 172.21.15.69:0/3039796695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e481590 0x55aa7e84a7f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f414400bfc0 tx=0x7f414400b670 comp rx=0 tx=0).stop 2024-04-25T11:26:00.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.206+0000 7f415f668740 1 -- 172.21.15.69:0/3039796695 shutdown_connections 2024-04-25T11:26:00.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.206+0000 7f415f668740 1 --2- 172.21.15.69:0/3039796695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e481590 0x55aa7e84a7f0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.206+0000 7f415f668740 1 --2- 172.21.15.69:0/3039796695 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa7e4793b0 0x55aa7e8438f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.206+0000 7f415f668740 1 --2- 172.21.15.69:0/3039796695 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa7e4a2270 0x55aa7e843370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.206+0000 7f415f668740 1 -- 172.21.15.69:0/3039796695 >> 172.21.15.69:0/3039796695 conn(0x55aa7e4963c0 msgr2=0x55aa7e83ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:26:00.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.206+0000 7f415f668740 1 -- 172.21.15.69:0/3039796695 shutdown_connections 2024-04-25T11:26:00.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.206+0000 7f415f668740 1 -- 172.21.15.69:0/3039796695 wait complete. 2024-04-25T11:26:00.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.206+0000 7f415f668740 1 -- start start 2024-04-25T11:26:00.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.207+0000 7f415f668740 1 Processor -- start 2024-04-25T11:26:00.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.207+0000 7f415f668740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa7e4793b0 0x55aa7e84d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.207+0000 7f415f668740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e481590 0x55aa7e84d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.207+0000 7f415f668740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa7e4a2270 0x55aa7e8547c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.207+0000 7f415f668740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55aa7e84b240 con 0x55aa7e481590 2024-04-25T11:26:00.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.207+0000 7f415f668740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55aa7e72ffe0 con 0x55aa7e4a2270 2024-04-25T11:26:00.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.207+0000 7f415f668740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55aa7e84b3c0 con 0x55aa7e4793b0 2024-04-25T11:26:00.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.207+0000 7f415f668740 10 client.0 _ll_register_callbacks cb 0x55aa7e746150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:26:00.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.207+0000 7f415654e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e481590 0x55aa7e84d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.207+0000 7f415654e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e481590 0x55aa7e84d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57152/0 (socket says 172.21.15.69:57152) 2024-04-25T11:26:00.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.207+0000 7f415654e640 1 -- 172.21.15.69:0/1763542790 learned_addr learned my addr 172.21.15.69:0/1763542790 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:26:00.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.207+0000 7f415654e640 1 -- 172.21.15.69:0/1763542790 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa7e4793b0 msgr2=0x55aa7e84d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:26:00.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.207+0000 7f415654e640 1 --2- 172.21.15.69:0/1763542790 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa7e4793b0 0x55aa7e84d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.207+0000 7f415654e640 1 -- 172.21.15.69:0/1763542790 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa7e4a2270 msgr2=0x55aa7e8547c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:26:00.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.207+0000 7f415654e640 1 --2- 172.21.15.69:0/1763542790 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa7e4a2270 0x55aa7e8547c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.207+0000 7f415654e640 1 -- 172.21.15.69:0/1763542790 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa7e4ba2d0 con 0x55aa7e481590 2024-04-25T11:26:00.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.207+0000 7f415654e640 1 --2- 172.21.15.69:0/1763542790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e481590 0x55aa7e84d860 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f41480126d0 tx=0x7f4148012b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44e01259922a679e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:26:00.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.207+0000 7f4135ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-25T11:26:00.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.207+0000 7f4135ffb640 1 -- 172.21.15.69:0/1763542790 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4148010dc0 con 0x55aa7e481590 2024-04-25T11:26:00.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.208+0000 7f4135ffb640 1 -- 172.21.15.69:0/1763542790 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f414800f3c0 con 0x55aa7e481590 2024-04-25T11:26:00.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.208+0000 7f415f668740 1 -- 172.21.15.69:0/1763542790 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4144011950 con 0x55aa7e481590 2024-04-25T11:26:00.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.208+0000 7f4135ffb640 1 -- 172.21.15.69:0/1763542790 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f414800f5c0 con 0x55aa7e481590 2024-04-25T11:26:00.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.208+0000 7f415f668740 10 client.15426 fetch_fsmap learned FSMap version 28 2024-04-25T11:26:00.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.208+0000 7f415f668740 1 -- 172.21.15.69:0/1763542790 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55aa7e5846b0 con 0x55aa7e481590 2024-04-25T11:26:00.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.208+0000 7f4135ffb640 1 -- 172.21.15.69:0/1763542790 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4144011950 con 0x55aa7e481590 2024-04-25T11:26:00.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.208+0000 7f4135ffb640 1 -- 172.21.15.69:0/1763542790 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f414801b820 con 0x55aa7e481590 2024-04-25T11:26:00.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.208+0000 7f415f668740 10 client.15426 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T11:26:00.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.208+0000 7f415f668740 10 client.15426 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T11:26:00.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.208+0000 7f415f668740 20 client.15426 populate_metadata read hostname 'smithi069' 2024-04-25T11:26:00.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.208+0000 7f415f668740 1 --2- 172.21.15.69:0/1763542790 >> [v2:172.21.15.69:6840/2060225181,v1:172.21.15.69:6841/2060225181] conn(0x55aa7e460e40 0x55aa7e85b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.208+0000 7f415f668740 4 client.15426 mds_command: new command op to 14625 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:26:00.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.208+0000 7f415f668740 1 -- 172.21.15.69:0/1763542790 --> [v2:172.21.15.69:6840/2060225181,v1:172.21.15.69:6841/2060225181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa7e7b6350 con 0x55aa7e460e40 2024-04-25T11:26:00.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.208+0000 7f4155d4d640 1 --2- 172.21.15.69:0/1763542790 >> [v2:172.21.15.69:6840/2060225181,v1:172.21.15.69:6841/2060225181] conn(0x55aa7e460e40 0x55aa7e85b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.209+0000 7f4135ffb640 10 client.15426 ms_handle_connect on v2:172.21.15.69:6840/2060225181 2024-04-25T11:26:00.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.209+0000 7f4155d4d640 1 --2- 172.21.15.69:0/1763542790 >> [v2:172.21.15.69:6840/2060225181,v1:172.21.15.69:6841/2060225181] conn(0x55aa7e460e40 0x55aa7e85b140 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f9bda65cbcf78031 server_cookie=880a414d9263dffd in_seq=0 out_seq=0 2024-04-25T11:26:00.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.209+0000 7f4135ffb640 1 -- 172.21.15.69:0/1763542790 <== mds.4 v2:172.21.15.69:6840/2060225181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55aa7e7b6350 con 0x55aa7e460e40 2024-04-25T11:26:00.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.209+0000 7f4135ffb640 10 client.15426 handle_command_reply: tid=0 2024-04-25T11:26:00.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.209+0000 7f415f668740 1 client.15426 shutdown 2024-04-25T11:26:00.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.209+0000 7f415f668740 1 -- 172.21.15.69:0/1763542790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e481590 msgr2=0x55aa7e84d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:00.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.209+0000 7f415f668740 1 --2- 172.21.15.69:0/1763542790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e481590 0x55aa7e84d860 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f41480126d0 tx=0x7f4148012b90 comp rx=0 tx=0).stop 2024-04-25T11:26:00.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.209+0000 7f415f668740 1 -- 172.21.15.69:0/1763542790 shutdown_connections 2024-04-25T11:26:00.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.209+0000 7f415f668740 1 --2- 172.21.15.69:0/1763542790 >> [v2:172.21.15.69:6840/2060225181,v1:172.21.15.69:6841/2060225181] conn(0x55aa7e460e40 0x55aa7e85b140 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.209+0000 7f415f668740 1 --2- 172.21.15.69:0/1763542790 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa7e4a2270 0x55aa7e8547c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.209+0000 7f415f668740 1 --2- 172.21.15.69:0/1763542790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e481590 0x55aa7e84d860 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.209+0000 7f415f668740 1 --2- 172.21.15.69:0/1763542790 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa7e4793b0 0x55aa7e84d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.209+0000 7f415f668740 1 -- 172.21.15.69:0/1763542790 >> 172.21.15.69:0/1763542790 conn(0x55aa7e4963c0 msgr2=0x55aa7e83dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:26:00.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.210+0000 7f4135ffb640 0 client.15426 ms_handle_reset on v2:172.21.15.69:6840/2060225181 2024-04-25T11:26:00.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.210+0000 7f415f668740 1 -- 172.21.15.69:0/1763542790 shutdown_connections 2024-04-25T11:26:00.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.210+0000 7f415f668740 1 -- 172.21.15.69:0/1763542790 wait complete. 2024-04-25T11:26:00.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.210+0000 7f415f668740 20 client.15426 trim_cache size 0 max 16384 2024-04-25T11:26:00.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.228+0000 7f415f668740 1 Processor -- start 2024-04-25T11:26:00.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.228+0000 7f415f668740 1 -- start start 2024-04-25T11:26:00.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.228+0000 7f415f668740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e4793b0 0x55aa7e84b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.228+0000 7f415f668740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa7e481590 0x55aa7e8546e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.228+0000 7f415f668740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa7e4a2270 0x55aa7e83e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.228+0000 7f415f668740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55aa7e859480 con 0x55aa7e4793b0 2024-04-25T11:26:00.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.228+0000 7f415f668740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55aa7e859b50 con 0x55aa7e481590 2024-04-25T11:26:00.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.228+0000 7f415f668740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55aa7e85a220 con 0x55aa7e4a2270 2024-04-25T11:26:00.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.229+0000 7f4155d4d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e4793b0 0x55aa7e84b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.229+0000 7f4155d4d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e4793b0 0x55aa7e84b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57166/0 (socket says 172.21.15.69:57166) 2024-04-25T11:26:00.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.229+0000 7f4155d4d640 1 -- 172.21.15.69:0/1141317082 learned_addr learned my addr 172.21.15.69:0/1141317082 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:26:00.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.229+0000 7f4155d4d640 1 -- 172.21.15.69:0/1141317082 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa7e4a2270 msgr2=0x55aa7e83e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:26:00.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.229+0000 7f4155d4d640 1 --2- 172.21.15.69:0/1141317082 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa7e4a2270 0x55aa7e83e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.229+0000 7f4155d4d640 1 -- 172.21.15.69:0/1141317082 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa7e481590 msgr2=0x55aa7e8546e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:26:00.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.229+0000 7f4155d4d640 1 --2- 172.21.15.69:0/1141317082 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa7e481590 0x55aa7e8546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.229+0000 7f4155d4d640 1 -- 172.21.15.69:0/1141317082 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa7e848130 con 0x55aa7e4793b0 2024-04-25T11:26:00.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.229+0000 7f4155d4d640 1 --2- 172.21.15.69:0/1141317082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e4793b0 0x55aa7e84b120 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f41440058c0 tx=0x7f414400bd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed7ba130e979bae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:26:00.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.229+0000 7f4137fff640 1 -- 172.21.15.69:0/1141317082 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414402c070 con 0x55aa7e4793b0 2024-04-25T11:26:00.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.229+0000 7f4137fff640 1 -- 172.21.15.69:0/1141317082 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4144024070 con 0x55aa7e4793b0 2024-04-25T11:26:00.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.229+0000 7f4137fff640 1 -- 172.21.15.69:0/1141317082 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4144019040 con 0x55aa7e4793b0 2024-04-25T11:26:00.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.230+0000 7f415f668740 1 -- 172.21.15.69:0/1141317082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e4793b0 msgr2=0x55aa7e84b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:00.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.230+0000 7f415f668740 1 --2- 172.21.15.69:0/1141317082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e4793b0 0x55aa7e84b120 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f41440058c0 tx=0x7f414400bd20 comp rx=0 tx=0).stop 2024-04-25T11:26:00.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.230+0000 7f415f668740 1 -- 172.21.15.69:0/1141317082 shutdown_connections 2024-04-25T11:26:00.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.230+0000 7f415f668740 1 --2- 172.21.15.69:0/1141317082 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa7e4a2270 0x55aa7e83e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.230+0000 7f415f668740 1 --2- 172.21.15.69:0/1141317082 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa7e481590 0x55aa7e8546e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.230+0000 7f415f668740 1 --2- 172.21.15.69:0/1141317082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e4793b0 0x55aa7e84b120 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.230+0000 7f415f668740 1 -- 172.21.15.69:0/1141317082 >> 172.21.15.69:0/1141317082 conn(0x55aa7e4963c0 msgr2=0x55aa7e84b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:26:00.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.230+0000 7f415f668740 1 -- 172.21.15.69:0/1141317082 shutdown_connections 2024-04-25T11:26:00.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.230+0000 7f415f668740 1 -- 172.21.15.69:0/1141317082 wait complete. 2024-04-25T11:26:00.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.230+0000 7f415f668740 1 -- start start 2024-04-25T11:26:00.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.230+0000 7f415f668740 1 Processor -- start 2024-04-25T11:26:00.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.230+0000 7f415f668740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa7e4793b0 0x55aa7e84f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.230+0000 7f415f668740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e481590 0x55aa7e84fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.230+0000 7f415f668740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa7e4a2270 0x55aa7e852cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.231+0000 7f415f668740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55aa7e85afc0 con 0x55aa7e481590 2024-04-25T11:26:00.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.231+0000 7f415f668740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55aa7e85a8f0 con 0x55aa7e4793b0 2024-04-25T11:26:00.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.231+0000 7f415f668740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55aa7e85b690 con 0x55aa7e4a2270 2024-04-25T11:26:00.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.231+0000 7f415f668740 10 client.0 _ll_register_callbacks cb 0x7f4148009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:26:00.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.231+0000 7f415654e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e481590 0x55aa7e84fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.231+0000 7f415654e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e481590 0x55aa7e84fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57170/0 (socket says 172.21.15.69:57170) 2024-04-25T11:26:00.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.231+0000 7f415654e640 1 -- 172.21.15.69:0/3503149350 learned_addr learned my addr 172.21.15.69:0/3503149350 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:26:00.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.231+0000 7f4155d4d640 1 --2- 172.21.15.69:0/3503149350 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa7e4793b0 0x55aa7e84f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.231+0000 7f4155d4d640 1 --2- 172.21.15.69:0/3503149350 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa7e4a2270 0x55aa7e852cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.231+0000 7f415654e640 1 -- 172.21.15.69:0/3503149350 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa7e4a2270 msgr2=0x55aa7e852cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:26:00.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.231+0000 7f415654e640 1 --2- 172.21.15.69:0/3503149350 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa7e4a2270 0x55aa7e852cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.231+0000 7f415654e640 1 -- 172.21.15.69:0/3503149350 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa7e4793b0 msgr2=0x55aa7e84f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:26:00.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.231+0000 7f415654e640 1 --2- 172.21.15.69:0/3503149350 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa7e4793b0 0x55aa7e84f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.231+0000 7f415654e640 1 -- 172.21.15.69:0/3503149350 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa7e853290 con 0x55aa7e481590 2024-04-25T11:26:00.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.231+0000 7f4155d4d640 1 --2- 172.21.15.69:0/3503149350 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa7e4a2270 0x55aa7e852cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:26:00.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.231+0000 7f415654e640 1 --2- 172.21.15.69:0/3503149350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e481590 0x55aa7e84fd90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f414801bc40 tx=0x7f41480107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce92b8d94b041fc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:26:00.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.231+0000 7f4135ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-25T11:26:00.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.231+0000 7f4135ffb640 1 -- 172.21.15.69:0/3503149350 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414801b680 con 0x55aa7e481590 2024-04-25T11:26:00.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.231+0000 7f4135ffb640 1 -- 172.21.15.69:0/3503149350 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f414801b820 con 0x55aa7e481590 2024-04-25T11:26:00.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.231+0000 7f415f668740 1 -- 172.21.15.69:0/3503149350 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4144024070 con 0x55aa7e481590 2024-04-25T11:26:00.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.231+0000 7f4135ffb640 1 -- 172.21.15.69:0/3503149350 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f414801ba20 con 0x55aa7e481590 2024-04-25T11:26:00.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.231+0000 7f415f668740 10 client.15432 fetch_fsmap learned FSMap version 28 2024-04-25T11:26:00.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.231+0000 7f415f668740 1 -- 172.21.15.69:0/3503149350 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55aa7e854570 con 0x55aa7e481590 2024-04-25T11:26:00.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.232+0000 7f4135ffb640 1 -- 172.21.15.69:0/3503149350 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4144024070 con 0x55aa7e481590 2024-04-25T11:26:00.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.232+0000 7f4135ffb640 1 -- 172.21.15.69:0/3503149350 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f41480197b0 con 0x55aa7e481590 2024-04-25T11:26:00.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.232+0000 7f415f668740 10 client.15432 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T11:26:00.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.232+0000 7f415f668740 10 client.15432 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T11:26:00.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.232+0000 7f415f668740 20 client.15432 populate_metadata read hostname 'smithi069' 2024-04-25T11:26:00.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.232+0000 7f415f668740 1 --2- 172.21.15.69:0/3503149350 >> [v2:172.21.15.69:6840/2060225181,v1:172.21.15.69:6841/2060225181] conn(0x55aa7e436280 0x55aa7e8c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.232+0000 7f415f668740 4 client.15432 mds_command: new command op to 14625 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:26:00.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.232+0000 7f415f668740 1 -- 172.21.15.69:0/3503149350 --> [v2:172.21.15.69:6840/2060225181,v1:172.21.15.69:6841/2060225181] -- command(tid 0: {"prefix": "status"}) -- 0x55aa7e7b01c0 con 0x55aa7e436280 2024-04-25T11:26:00.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.232+0000 7f4155d4d640 1 --2- 172.21.15.69:0/3503149350 >> [v2:172.21.15.69:6840/2060225181,v1:172.21.15.69:6841/2060225181] conn(0x55aa7e436280 0x55aa7e8c71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.232+0000 7f4135ffb640 10 client.15432 ms_handle_connect on v2:172.21.15.69:6840/2060225181 2024-04-25T11:26:00.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.232+0000 7f4155d4d640 1 --2- 172.21.15.69:0/3503149350 >> [v2:172.21.15.69:6840/2060225181,v1:172.21.15.69:6841/2060225181] conn(0x55aa7e436280 0x55aa7e8c71e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1e5407524409b61c server_cookie=ac45b0fef5ba649c in_seq=0 out_seq=0 2024-04-25T11:26:00.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.232+0000 7f4135ffb640 1 -- 172.21.15.69:0/3503149350 <== mds.4 v2:172.21.15.69:6840/2060225181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55aa7e7b01c0 con 0x55aa7e436280 2024-04-25T11:26:00.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.232+0000 7f4135ffb640 10 client.15432 handle_command_reply: tid=0 2024-04-25T11:26:00.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.232+0000 7f415f668740 1 client.15432 shutdown 2024-04-25T11:26:00.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.233+0000 7f415f668740 1 -- 172.21.15.69:0/3503149350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e481590 msgr2=0x55aa7e84fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:00.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.233+0000 7f415f668740 1 --2- 172.21.15.69:0/3503149350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e481590 0x55aa7e84fd90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f414801bc40 tx=0x7f41480107a0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.233+0000 7f415f668740 1 -- 172.21.15.69:0/3503149350 shutdown_connections 2024-04-25T11:26:00.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.233+0000 7f415f668740 1 --2- 172.21.15.69:0/3503149350 >> [v2:172.21.15.69:6840/2060225181,v1:172.21.15.69:6841/2060225181] conn(0x55aa7e436280 0x55aa7e8c71e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.233+0000 7f415f668740 1 --2- 172.21.15.69:0/3503149350 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa7e4a2270 0x55aa7e852cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.233+0000 7f415f668740 1 --2- 172.21.15.69:0/3503149350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aa7e481590 0x55aa7e84fd90 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.233+0000 7f415f668740 1 --2- 172.21.15.69:0/3503149350 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55aa7e4793b0 0x55aa7e84f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.233+0000 7f415f668740 1 -- 172.21.15.69:0/3503149350 >> 172.21.15.69:0/3503149350 conn(0x55aa7e4963c0 msgr2=0x55aa7e83e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:26:00.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.233+0000 7f4135ffb640 0 client.15432 ms_handle_reset on v2:172.21.15.69:6840/2060225181 2024-04-25T11:26:00.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.233+0000 7f415f668740 1 -- 172.21.15.69:0/3503149350 shutdown_connections 2024-04-25T11:26:00.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.233+0000 7f415f668740 1 -- 172.21.15.69:0/3503149350 wait complete. 2024-04-25T11:26:00.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.233+0000 7f415f668740 20 client.15432 trim_cache size 0 max 16384 2024-04-25T11:26:00.247 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-25T11:26:00.248 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "f7ed73dc-02f4-11ef-bc93-c7b262605968", 2024-04-25T11:26:00.248 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 4, 2024-04-25T11:26:00.248 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 14625, 2024-04-25T11:26:00.248 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-04-25T11:26:00.248 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-04-25T11:26:00.248 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-04-25T11:26:00.249 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 112.178018657, 2024-04-25T11:26:00.249 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 28, 2024-04-25T11:26:00.249 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 84, 2024-04-25T11:26:00.249 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 80, 2024-04-25T11:26:00.249 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 148.448728816 2024-04-25T11:26:00.250 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-25T11:26:00.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.233+0000 7f41587b2640 1 -- 172.21.15.69:0/906224348 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f41380afb70 msgr2=0x7f41380b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:00.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.233+0000 7f41587b2640 1 --2- 172.21.15.69:0/906224348 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f41380afb70 0x7f41380b2080 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f41400b9eb0 tx=0x7f41400f5ed0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.233+0000 7f41587b2640 1 -- 172.21.15.69:0/906224348 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4150108940 msgr2=0x7f41501a15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:00.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.233+0000 7f41587b2640 1 --2- 172.21.15.69:0/906224348 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4150108940 0x7f41501a15d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4140005ab0 tx=0x7f41400042f0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.233+0000 7f41587b2640 1 -- 172.21.15.69:0/906224348 shutdown_connections 2024-04-25T11:26:00.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.233+0000 7f41587b2640 1 --2- 172.21.15.69:0/906224348 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f41380afb70 0x7f41380b2080 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.233+0000 7f41587b2640 1 --2- 172.21.15.69:0/906224348 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4150108940 0x7f41501a15d0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.233+0000 7f41587b2640 1 --2- 172.21.15.69:0/906224348 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4150103c40 0x7f415019a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.233+0000 7f41587b2640 1 --2- 172.21.15.69:0/906224348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4150102a00 0x7f415019a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.233+0000 7f41587b2640 1 -- 172.21.15.69:0/906224348 >> 172.21.15.69:0/906224348 conn(0x7f41500fdbf0 msgr2=0x7f4150109e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:26:00.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.233+0000 7f41587b2640 1 -- 172.21.15.69:0/906224348 shutdown_connections 2024-04-25T11:26:00.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.233+0000 7f41587b2640 1 -- 172.21.15.69:0/906224348 wait complete. 2024-04-25T11:26:00.253 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-04-25T11:26:00.254 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:59 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/1269093869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:26:00.254 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:59 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/1048758707' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T11:26:00.254 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:59 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/3601579288' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:26:00.254 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:25:59 smithi152 ceph-mon[32701]: pgmap v358: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:26:00.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:59 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/1269093869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:26:00.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:59 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/1048758707' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T11:26:00.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:59 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/3601579288' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:26:00.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:25:59 smithi146 ceph-mon[32821]: pgmap v358: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:26:00.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.370+0000 7f0cd3b57640 1 -- 172.21.15.69:0/3901167576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ccc109a40 msgr2=0x7f0ccc10be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:00.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.370+0000 7f0cd3b57640 1 --2- 172.21.15.69:0/3901167576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ccc109a40 0x7f0ccc10be60 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4009ca0 tx=0x7f0cc402a8c0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.370+0000 7f0cd3b57640 1 -- 172.21.15.69:0/3901167576 shutdown_connections 2024-04-25T11:26:00.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.370+0000 7f0cd3b57640 1 --2- 172.21.15.69:0/3901167576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ccc109a40 0x7f0ccc10be60 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.370+0000 7f0cd3b57640 1 --2- 172.21.15.69:0/3901167576 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ccc104d40 0x7f0ccc105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.370+0000 7f0cd3b57640 1 --2- 172.21.15.69:0/3901167576 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ccc103b00 0x7f0ccc104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.370+0000 7f0cd3b57640 1 -- 172.21.15.69:0/3901167576 >> 172.21.15.69:0/3901167576 conn(0x7f0ccc0fecf0 msgr2=0x7f0ccc1011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:26:00.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.370+0000 7f0cd3b57640 1 -- 172.21.15.69:0/3901167576 shutdown_connections 2024-04-25T11:26:00.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.370+0000 7f0cd3b57640 1 -- 172.21.15.69:0/3901167576 wait complete. 2024-04-25T11:26:00.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.370+0000 7f0cd3b57640 1 Processor -- start 2024-04-25T11:26:00.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.371+0000 7f0cd3b57640 1 -- start start 2024-04-25T11:26:00.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.371+0000 7f0cd3b57640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ccc103b00 0x7f0ccc19a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.371+0000 7f0cd3b57640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ccc104d40 0x7f0ccc19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.371+0000 7f0cd3b57640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ccc109a40 0x7f0ccc1a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.371+0000 7f0cd3b57640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc10f510 con 0x7f0ccc109a40 2024-04-25T11:26:00.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.371+0000 7f0cd3b57640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc10f390 con 0x7f0ccc104d40 2024-04-25T11:26:00.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.371+0000 7f0cd3b57640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc10f690 con 0x7f0ccc103b00 2024-04-25T11:26:00.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.371+0000 7f0cd20f4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ccc109a40 0x7f0ccc1a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.371+0000 7f0cd20f4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ccc109a40 0x7f0ccc1a1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57192/0 (socket says 172.21.15.69:57192) 2024-04-25T11:26:00.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.371+0000 7f0cd20f4640 1 -- 172.21.15.69:0/4055819729 learned_addr learned my addr 172.21.15.69:0/4055819729 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:26:00.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.371+0000 7f0cd18f3640 1 --2- 172.21.15.69:0/4055819729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ccc103b00 0x7f0ccc19a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.371+0000 7f0cd10f2640 1 --2- 172.21.15.69:0/4055819729 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ccc104d40 0x7f0ccc19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.371+0000 7f0cd20f4640 1 -- 172.21.15.69:0/4055819729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ccc103b00 msgr2=0x7f0ccc19a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:00.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.371+0000 7f0cd20f4640 1 --2- 172.21.15.69:0/4055819729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ccc103b00 0x7f0ccc19a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.371+0000 7f0cd20f4640 1 -- 172.21.15.69:0/4055819729 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ccc104d40 msgr2=0x7f0ccc19a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:00.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.371+0000 7f0cd20f4640 1 --2- 172.21.15.69:0/4055819729 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ccc104d40 0x7f0ccc19a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.371+0000 7f0cd20f4640 1 -- 172.21.15.69:0/4055819729 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ccc1a1b40 con 0x7f0ccc109a40 2024-04-25T11:26:00.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.371+0000 7f0cd20f4640 1 --2- 172.21.15.69:0/4055819729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ccc109a40 0x7f0ccc1a1580 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4005e30 tx=0x7f0cc4003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:26:00.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.371+0000 7f0cc2ffd640 1 -- 172.21.15.69:0/4055819729 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc4032e90 con 0x7f0ccc109a40 2024-04-25T11:26:00.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.372+0000 7f0cd3b57640 1 -- 172.21.15.69:0/4055819729 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ccc1a1dd0 con 0x7f0ccc109a40 2024-04-25T11:26:00.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.372+0000 7f0cc2ffd640 1 -- 172.21.15.69:0/4055819729 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0cc40044d0 con 0x7f0ccc109a40 2024-04-25T11:26:00.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.372+0000 7f0cc2ffd640 1 -- 172.21.15.69:0/4055819729 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc4030ce0 con 0x7f0ccc109a40 2024-04-25T11:26:00.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.372+0000 7f0cd3b57640 1 -- 172.21.15.69:0/4055819729 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ccc1a2240 con 0x7f0ccc109a40 2024-04-25T11:26:00.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.374+0000 7f0cdaa0a740 1 Processor -- start 2024-04-25T11:26:00.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.374+0000 7f0cc2ffd640 1 -- 172.21.15.69:0/4055819729 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f0cc4004040 con 0x7f0ccc109a40 2024-04-25T11:26:00.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.375+0000 7f0cdaa0a740 1 -- start start 2024-04-25T11:26:00.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.375+0000 7f0cdaa0a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc64260270 0x55fc64601370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.375+0000 7f0cdaa0a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fc642373b0 0x55fc646018f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.375+0000 7f0cdaa0a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fc6423f590 0x55fc646087f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.375+0000 7f0cdaa0a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55fc64566740 con 0x55fc64260270 2024-04-25T11:26:00.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.375+0000 7f0cdaa0a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55fc645023a0 con 0x55fc6423f590 2024-04-25T11:26:00.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.375+0000 7f0cdaa0a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55fc64562700 con 0x55fc642373b0 2024-04-25T11:26:00.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.375+0000 7f0cd10f2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc64260270 0x55fc64601370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.375+0000 7f0cd10f2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc64260270 0x55fc64601370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57208/0 (socket says 172.21.15.69:57208) 2024-04-25T11:26:00.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.375+0000 7f0cd10f2640 1 -- 172.21.15.69:0/2968775506 learned_addr learned my addr 172.21.15.69:0/2968775506 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:26:00.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.375+0000 7f0cc2ffd640 1 --2- 172.21.15.69:0/4055819729 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f0cb80afaf0 0x7f0cb80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.375+0000 7f0cd10f2640 1 --2- 172.21.15.69:0/2968775506 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fc6423f590 0x55fc646087f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.375+0000 7f0cd18f3640 1 --2- 172.21.15.69:0/2968775506 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fc642373b0 0x55fc646018f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.375+0000 7f0cd20f4640 1 --2- 172.21.15.69:0/4055819729 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f0cb80afaf0 0x7f0cb80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.375+0000 7f0cd10f2640 1 -- 172.21.15.69:0/2968775506 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fc642373b0 msgr2=0x55fc646018f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:26:00.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.375+0000 7f0cd10f2640 1 --2- 172.21.15.69:0/2968775506 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fc642373b0 0x55fc646018f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.375+0000 7f0cd10f2640 1 -- 172.21.15.69:0/2968775506 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fc6423f590 msgr2=0x55fc646087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:26:00.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.375+0000 7f0cd10f2640 1 --2- 172.21.15.69:0/2968775506 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fc6423f590 0x55fc646087f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.375+0000 7f0cd10f2640 1 -- 172.21.15.69:0/2968775506 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc6451e750 con 0x55fc64260270 2024-04-25T11:26:00.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.375+0000 7f0cc2ffd640 1 -- 172.21.15.69:0/4055819729 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f0cc41005a0 con 0x7f0ccc109a40 2024-04-25T11:26:00.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.376+0000 7f0cd10f2640 1 --2- 172.21.15.69:0/2968775506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc64260270 0x55fc64601370 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0cc80148d0 tx=0x7f0cc800b530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f314782fbd6f195a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:26:00.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.376+0000 7f0cb7fff640 1 -- 172.21.15.69:0/2968775506 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc8005220 con 0x55fc64260270 2024-04-25T11:26:00.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.376+0000 7f0cb7fff640 1 -- 172.21.15.69:0/2968775506 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0cc8014e50 con 0x55fc64260270 2024-04-25T11:26:00.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.376+0000 7f0cb7fff640 1 -- 172.21.15.69:0/2968775506 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc8005650 con 0x55fc64260270 2024-04-25T11:26:00.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.377+0000 7f0cdaa0a740 1 -- 172.21.15.69:0/2968775506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc64260270 msgr2=0x55fc64601370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:00.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.377+0000 7f0cdaa0a740 1 --2- 172.21.15.69:0/2968775506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc64260270 0x55fc64601370 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0cc80148d0 tx=0x7f0cc800b530 comp rx=0 tx=0).stop 2024-04-25T11:26:00.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.377+0000 7f0cd20f4640 1 --2- 172.21.15.69:0/4055819729 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f0cb80afaf0 0x7f0cb80b2000 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0cc40c8640 tx=0x7f0cc408eb20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:26:00.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.377+0000 7f0cdaa0a740 1 -- 172.21.15.69:0/2968775506 shutdown_connections 2024-04-25T11:26:00.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.377+0000 7f0cdaa0a740 1 --2- 172.21.15.69:0/2968775506 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fc6423f590 0x55fc646087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.377+0000 7f0cdaa0a740 1 --2- 172.21.15.69:0/2968775506 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fc642373b0 0x55fc646018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.377+0000 7f0cdaa0a740 1 --2- 172.21.15.69:0/2968775506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc64260270 0x55fc64601370 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.377+0000 7f0cdaa0a740 1 -- 172.21.15.69:0/2968775506 >> 172.21.15.69:0/2968775506 conn(0x55fc642543c0 msgr2=0x55fc645fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:26:00.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.377+0000 7f0cdaa0a740 1 -- 172.21.15.69:0/2968775506 shutdown_connections 2024-04-25T11:26:00.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.377+0000 7f0cdaa0a740 1 -- 172.21.15.69:0/2968775506 wait complete. 2024-04-25T11:26:00.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.377+0000 7f0cdaa0a740 1 -- start start 2024-04-25T11:26:00.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.377+0000 7f0cdaa0a740 1 Processor -- start 2024-04-25T11:26:00.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.378+0000 7f0cdaa0a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fc642373b0 0x55fc6460b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.378+0000 7f0cdaa0a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc6423f590 0x55fc6460b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.378+0000 7f0cdaa0a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fc64260270 0x55fc646126b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.378+0000 7f0cdaa0a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55fc646091e0 con 0x55fc6423f590 2024-04-25T11:26:00.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.378+0000 7f0cdaa0a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55fc644edfe0 con 0x55fc642373b0 2024-04-25T11:26:00.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.378+0000 7f0cdaa0a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55fc64609360 con 0x55fc64260270 2024-04-25T11:26:00.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.378+0000 7f0cdaa0a740 10 client.0 _ll_register_callbacks cb 0x55fc64504150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:26:00.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.378+0000 7f0cd18f3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc6423f590 0x55fc6460b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.378+0000 7f0cd18f3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc6423f590 0x55fc6460b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57218/0 (socket says 172.21.15.69:57218) 2024-04-25T11:26:00.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.378+0000 7f0cd18f3640 1 -- 172.21.15.69:0/482415369 learned_addr learned my addr 172.21.15.69:0/482415369 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:26:00.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.378+0000 7f0cd10f2640 1 --2- 172.21.15.69:0/482415369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fc642373b0 0x55fc6460b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.378+0000 7f0cd10f2640 1 --2- 172.21.15.69:0/482415369 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fc64260270 0x55fc646126b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.378+0000 7f0cd18f3640 1 -- 172.21.15.69:0/482415369 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fc64260270 msgr2=0x55fc646126b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:26:00.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.378+0000 7f0cd18f3640 1 --2- 172.21.15.69:0/482415369 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fc64260270 0x55fc646126b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.378+0000 7f0cd18f3640 1 -- 172.21.15.69:0/482415369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fc642373b0 msgr2=0x55fc6460b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:26:00.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.378+0000 7f0cd18f3640 1 --2- 172.21.15.69:0/482415369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fc642373b0 0x55fc6460b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.378+0000 7f0cd18f3640 1 -- 172.21.15.69:0/482415369 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc642782d0 con 0x55fc6423f590 2024-04-25T11:26:00.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.378+0000 7f0cd18f3640 1 --2- 172.21.15.69:0/482415369 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc6423f590 0x55fc6460b750 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc009f90 tx=0x7f0cbc012880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=599391d9e84c197f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:26:00.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.378+0000 7f0cb57fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-25T11:26:00.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.378+0000 7f0cb57fa640 1 -- 172.21.15.69:0/482415369 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc01f060 con 0x55fc6423f590 2024-04-25T11:26:00.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.378+0000 7f0cb57fa640 1 -- 172.21.15.69:0/482415369 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f0cbc004460 con 0x55fc6423f590 2024-04-25T11:26:00.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.378+0000 7f0cdaa0a740 1 -- 172.21.15.69:0/482415369 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cc8014e50 con 0x55fc6423f590 2024-04-25T11:26:00.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.378+0000 7f0cb57fa640 1 -- 172.21.15.69:0/482415369 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0cbc026050 con 0x55fc6423f590 2024-04-25T11:26:00.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.378+0000 7f0cdaa0a740 10 client.15444 fetch_fsmap learned FSMap version 28 2024-04-25T11:26:00.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.378+0000 7f0cdaa0a740 1 -- 172.21.15.69:0/482415369 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fc643426b0 con 0x55fc6423f590 2024-04-25T11:26:00.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.379+0000 7f0cb57fa640 1 -- 172.21.15.69:0/482415369 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0cc8014e50 con 0x55fc6423f590 2024-04-25T11:26:00.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.379+0000 7f0cb57fa640 1 -- 172.21.15.69:0/482415369 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0cbc019030 con 0x55fc6423f590 2024-04-25T11:26:00.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.379+0000 7f0cdaa0a740 10 client.15444 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T11:26:00.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.379+0000 7f0cdaa0a740 10 client.15444 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-25T11:26:00.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.379+0000 7f0cdaa0a740 20 client.15444 populate_metadata read hostname 'smithi069' 2024-04-25T11:26:00.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.379+0000 7f0cdaa0a740 1 --2- 172.21.15.69:0/482415369 >> [v2:172.21.15.146:6834/1876533699,v1:172.21.15.146:6835/1876533699] conn(0x55fc6421ee40 0x55fc64618fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.379+0000 7f0cdaa0a740 4 client.15444 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:26:00.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.379+0000 7f0cdaa0a740 1 -- 172.21.15.69:0/482415369 --> [v2:172.21.15.146:6834/1876533699,v1:172.21.15.146:6835/1876533699] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc64574350 con 0x55fc6421ee40 2024-04-25T11:26:00.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.379+0000 7f0cd10f2640 1 --2- 172.21.15.69:0/482415369 >> [v2:172.21.15.146:6834/1876533699,v1:172.21.15.146:6835/1876533699] conn(0x55fc6421ee40 0x55fc64618fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.380+0000 7f0cb57fa640 10 client.15444 ms_handle_connect on v2:172.21.15.146:6834/1876533699 2024-04-25T11:26:00.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.380+0000 7f0cd10f2640 1 --2- 172.21.15.69:0/482415369 >> [v2:172.21.15.146:6834/1876533699,v1:172.21.15.146:6835/1876533699] conn(0x55fc6421ee40 0x55fc64618fa0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c30881003a7ba28b server_cookie=d0383bfff6f8be12 in_seq=0 out_seq=0 2024-04-25T11:26:00.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.380+0000 7f0cb57fa640 1 -- 172.21.15.69:0/482415369 <== mds.3 v2:172.21.15.146:6834/1876533699 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fc64574350 con 0x55fc6421ee40 2024-04-25T11:26:00.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.380+0000 7f0cb57fa640 10 client.15444 handle_command_reply: tid=0 2024-04-25T11:26:00.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.380+0000 7f0cdaa0a740 1 client.15444 shutdown 2024-04-25T11:26:00.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.381+0000 7f0cdaa0a740 1 -- 172.21.15.69:0/482415369 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc6423f590 msgr2=0x55fc6460b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:00.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.381+0000 7f0cdaa0a740 1 --2- 172.21.15.69:0/482415369 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc6423f590 0x55fc6460b750 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc009f90 tx=0x7f0cbc012880 comp rx=0 tx=0).stop 2024-04-25T11:26:00.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.381+0000 7f0cdaa0a740 1 -- 172.21.15.69:0/482415369 shutdown_connections 2024-04-25T11:26:00.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.381+0000 7f0cdaa0a740 1 --2- 172.21.15.69:0/482415369 >> [v2:172.21.15.146:6834/1876533699,v1:172.21.15.146:6835/1876533699] conn(0x55fc6421ee40 0x55fc64618fa0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.381+0000 7f0cdaa0a740 1 --2- 172.21.15.69:0/482415369 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fc64260270 0x55fc646126b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.381+0000 7f0cdaa0a740 1 --2- 172.21.15.69:0/482415369 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc6423f590 0x55fc6460b750 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.381+0000 7f0cb57fa640 0 client.15444 ms_handle_reset on v2:172.21.15.146:6834/1876533699 2024-04-25T11:26:00.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.381+0000 7f0cdaa0a740 1 --2- 172.21.15.69:0/482415369 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fc642373b0 0x55fc6460b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.381+0000 7f0cdaa0a740 1 -- 172.21.15.69:0/482415369 >> 172.21.15.69:0/482415369 conn(0x55fc642543c0 msgr2=0x55fc645fbd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:26:00.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.381+0000 7f0cdaa0a740 1 -- 172.21.15.69:0/482415369 shutdown_connections 2024-04-25T11:26:00.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.381+0000 7f0cdaa0a740 1 -- 172.21.15.69:0/482415369 wait complete. 2024-04-25T11:26:00.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.381+0000 7f0cdaa0a740 20 client.15444 trim_cache size 0 max 16384 2024-04-25T11:26:00.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.399+0000 7f0cdaa0a740 1 Processor -- start 2024-04-25T11:26:00.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.399+0000 7f0cdaa0a740 1 -- start start 2024-04-25T11:26:00.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.399+0000 7f0cdaa0a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc6421ee40 0x55fc645fc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.399+0000 7f0cdaa0a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fc642373b0 0x55fc646125d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.399+0000 7f0cdaa0a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fc6423f590 0x55fc645fcc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.399+0000 7f0cdaa0a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55fc646172e0 con 0x55fc6421ee40 2024-04-25T11:26:00.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.399+0000 7f0cdaa0a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55fc646179b0 con 0x55fc6423f590 2024-04-25T11:26:00.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.399+0000 7f0cdaa0a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55fc64618080 con 0x55fc642373b0 2024-04-25T11:26:00.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.400+0000 7f0cd10f2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc6421ee40 0x55fc645fc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.400+0000 7f0cd10f2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc6421ee40 0x55fc645fc720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57220/0 (socket says 172.21.15.69:57220) 2024-04-25T11:26:00.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.400+0000 7f0cd10f2640 1 -- 172.21.15.69:0/3750808551 learned_addr learned my addr 172.21.15.69:0/3750808551 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:26:00.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.400+0000 7f0cd10f2640 1 -- 172.21.15.69:0/3750808551 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fc642373b0 msgr2=0x55fc646125d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:26:00.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.400+0000 7f0cd18f3640 1 --2- 172.21.15.69:0/3750808551 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fc642373b0 0x55fc646125d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.400+0000 7f0cd10f2640 1 --2- 172.21.15.69:0/3750808551 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fc642373b0 0x55fc646125d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.400+0000 7f0cd10f2640 1 -- 172.21.15.69:0/3750808551 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fc6423f590 msgr2=0x55fc645fcc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:26:00.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.400+0000 7f0cd10f2640 1 --2- 172.21.15.69:0/3750808551 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fc6423f590 0x55fc645fcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.400+0000 7f0cd10f2640 1 -- 172.21.15.69:0/3750808551 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc6460afc0 con 0x55fc6421ee40 2024-04-25T11:26:00.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.400+0000 7f0cd18f3640 1 --2- 172.21.15.69:0/3750808551 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fc642373b0 0x55fc646125d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:26:00.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.400+0000 7f0cd10f2640 1 --2- 172.21.15.69:0/3750808551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc6421ee40 0x55fc645fc720 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0cc801a000 tx=0x7f0cc800be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19555be200b190f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:26:00.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.400+0000 7f0cb77fe640 1 -- 172.21.15.69:0/3750808551 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc8030070 con 0x55fc6421ee40 2024-04-25T11:26:00.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.400+0000 7f0cb77fe640 1 -- 172.21.15.69:0/3750808551 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0cc8004cd0 con 0x55fc6421ee40 2024-04-25T11:26:00.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.400+0000 7f0cb77fe640 1 -- 172.21.15.69:0/3750808551 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc8004f00 con 0x55fc6421ee40 2024-04-25T11:26:00.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.401+0000 7f0cdaa0a740 1 -- 172.21.15.69:0/3750808551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc6421ee40 msgr2=0x55fc645fc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:00.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.401+0000 7f0cdaa0a740 1 --2- 172.21.15.69:0/3750808551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc6421ee40 0x55fc645fc720 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0cc801a000 tx=0x7f0cc800be10 comp rx=0 tx=0).stop 2024-04-25T11:26:00.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.401+0000 7f0cdaa0a740 1 -- 172.21.15.69:0/3750808551 shutdown_connections 2024-04-25T11:26:00.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.401+0000 7f0cdaa0a740 1 --2- 172.21.15.69:0/3750808551 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fc6423f590 0x55fc645fcc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.401+0000 7f0cdaa0a740 1 --2- 172.21.15.69:0/3750808551 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fc642373b0 0x55fc646125d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.401+0000 7f0cdaa0a740 1 --2- 172.21.15.69:0/3750808551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc6421ee40 0x55fc645fc720 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.401+0000 7f0cdaa0a740 1 -- 172.21.15.69:0/3750808551 >> 172.21.15.69:0/3750808551 conn(0x55fc642543c0 msgr2=0x55fc646056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:26:00.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.401+0000 7f0cdaa0a740 1 -- 172.21.15.69:0/3750808551 shutdown_connections 2024-04-25T11:26:00.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.401+0000 7f0cdaa0a740 1 -- 172.21.15.69:0/3750808551 wait complete. 2024-04-25T11:26:00.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.401+0000 7f0cdaa0a740 1 -- start start 2024-04-25T11:26:00.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.401+0000 7f0cdaa0a740 1 Processor -- start 2024-04-25T11:26:00.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.402+0000 7f0cdaa0a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc6421ee40 0x55fc6460d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.402+0000 7f0cdaa0a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fc642373b0 0x55fc6460de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.402+0000 7f0cdaa0a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fc6423f590 0x55fc64610d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.402+0000 7f0cdaa0a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55fc64618e20 con 0x55fc6421ee40 2024-04-25T11:26:00.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.402+0000 7f0cdaa0a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55fc64618750 con 0x55fc642373b0 2024-04-25T11:26:00.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.402+0000 7f0cdaa0a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55fc646194f0 con 0x55fc6423f590 2024-04-25T11:26:00.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.402+0000 7f0cdaa0a740 10 client.0 _ll_register_callbacks cb 0x7f0cbc009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:26:00.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.402+0000 7f0cd10f2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc6421ee40 0x55fc6460d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.402+0000 7f0cd10f2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc6421ee40 0x55fc6460d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57228/0 (socket says 172.21.15.69:57228) 2024-04-25T11:26:00.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.402+0000 7f0cd10f2640 1 -- 172.21.15.69:0/826791036 learned_addr learned my addr 172.21.15.69:0/826791036 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:26:00.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.402+0000 7f0cd10f2640 1 -- 172.21.15.69:0/826791036 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fc6423f590 msgr2=0x55fc64610d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:26:00.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.402+0000 7f0cd10f2640 1 --2- 172.21.15.69:0/826791036 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fc6423f590 0x55fc64610d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.402+0000 7f0cd10f2640 1 -- 172.21.15.69:0/826791036 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fc642373b0 msgr2=0x55fc6460de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:26:00.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.402+0000 7f0cd10f2640 1 --2- 172.21.15.69:0/826791036 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fc642373b0 0x55fc6460de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.402+0000 7f0cd10f2640 1 -- 172.21.15.69:0/826791036 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc646112c0 con 0x55fc6421ee40 2024-04-25T11:26:00.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.402+0000 7f0cd10f2640 1 --2- 172.21.15.69:0/826791036 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc6421ee40 0x55fc6460d8d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0cc800b3a0 tx=0x7f0cc802b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edc9b153720ec2af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:26:00.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.402+0000 7f0cb57fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-25T11:26:00.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.402+0000 7f0cb57fa640 1 -- 172.21.15.69:0/826791036 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc8038020 con 0x55fc6421ee40 2024-04-25T11:26:00.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.402+0000 7f0cb57fa640 1 -- 172.21.15.69:0/826791036 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f0cc802b3b0 con 0x55fc6421ee40 2024-04-25T11:26:00.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.402+0000 7f0cdaa0a740 1 -- 172.21.15.69:0/826791036 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fc6456e1c0 con 0x55fc6421ee40 2024-04-25T11:26:00.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.403+0000 7f0cb57fa640 1 -- 172.21.15.69:0/826791036 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0cc8043050 con 0x55fc6421ee40 2024-04-25T11:26:00.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.403+0000 7f0cdaa0a740 10 client.15450 fetch_fsmap learned FSMap version 28 2024-04-25T11:26:00.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.403+0000 7f0cdaa0a740 1 -- 172.21.15.69:0/826791036 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fc646124b0 con 0x55fc6421ee40 2024-04-25T11:26:00.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.403+0000 7f0cb57fa640 1 -- 172.21.15.69:0/826791036 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55fc6456e1c0 con 0x55fc6421ee40 2024-04-25T11:26:00.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.403+0000 7f0cb57fa640 1 -- 172.21.15.69:0/826791036 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0cc8033030 con 0x55fc6421ee40 2024-04-25T11:26:00.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.403+0000 7f0cdaa0a740 10 client.15450 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T11:26:00.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.403+0000 7f0cdaa0a740 10 client.15450 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-25T11:26:00.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.403+0000 7f0cdaa0a740 20 client.15450 populate_metadata read hostname 'smithi069' 2024-04-25T11:26:00.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.403+0000 7f0cdaa0a740 1 --2- 172.21.15.69:0/826791036 >> [v2:172.21.15.146:6834/1876533699,v1:172.21.15.146:6835/1876533699] conn(0x55fc64260270 0x55fc64684970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.403+0000 7f0cdaa0a740 4 client.15450 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:26:00.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.403+0000 7f0cdaa0a740 1 -- 172.21.15.69:0/826791036 --> [v2:172.21.15.146:6834/1876533699,v1:172.21.15.146:6835/1876533699] -- command(tid 0: {"prefix": "status"}) -- 0x55fc644e99a0 con 0x55fc64260270 2024-04-25T11:26:00.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.403+0000 7f0cd18f3640 1 --2- 172.21.15.69:0/826791036 >> [v2:172.21.15.146:6834/1876533699,v1:172.21.15.146:6835/1876533699] conn(0x55fc64260270 0x55fc64684970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.404+0000 7f0cb57fa640 10 client.15450 ms_handle_connect on v2:172.21.15.146:6834/1876533699 2024-04-25T11:26:00.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.404+0000 7f0cd18f3640 1 --2- 172.21.15.69:0/826791036 >> [v2:172.21.15.146:6834/1876533699,v1:172.21.15.146:6835/1876533699] conn(0x55fc64260270 0x55fc64684970 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e213e5c70858fa8f server_cookie=ace632cf468d5b60 in_seq=0 out_seq=0 2024-04-25T11:26:00.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.404+0000 7f0cb57fa640 1 -- 172.21.15.69:0/826791036 <== mds.3 v2:172.21.15.146:6834/1876533699 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55fc644e99a0 con 0x55fc64260270 2024-04-25T11:26:00.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.404+0000 7f0cb57fa640 10 client.15450 handle_command_reply: tid=0 2024-04-25T11:26:00.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.404+0000 7f0cdaa0a740 1 client.15450 shutdown 2024-04-25T11:26:00.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.404+0000 7f0cdaa0a740 1 -- 172.21.15.69:0/826791036 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc6421ee40 msgr2=0x55fc6460d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:00.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.404+0000 7f0cdaa0a740 1 --2- 172.21.15.69:0/826791036 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc6421ee40 0x55fc6460d8d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0cc800b3a0 tx=0x7f0cc802b680 comp rx=0 tx=0).stop 2024-04-25T11:26:00.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.404+0000 7f0cdaa0a740 1 -- 172.21.15.69:0/826791036 shutdown_connections 2024-04-25T11:26:00.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.404+0000 7f0cdaa0a740 1 --2- 172.21.15.69:0/826791036 >> [v2:172.21.15.146:6834/1876533699,v1:172.21.15.146:6835/1876533699] conn(0x55fc64260270 0x55fc64684970 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.404+0000 7f0cdaa0a740 1 --2- 172.21.15.69:0/826791036 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fc6423f590 0x55fc64610d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.404+0000 7f0cdaa0a740 1 --2- 172.21.15.69:0/826791036 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fc642373b0 0x55fc6460de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.404+0000 7f0cb57fa640 0 client.15450 ms_handle_reset on v2:172.21.15.146:6834/1876533699 2024-04-25T11:26:00.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.404+0000 7f0cdaa0a740 1 --2- 172.21.15.69:0/826791036 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fc6421ee40 0x55fc6460d8d0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.404+0000 7f0cdaa0a740 1 -- 172.21.15.69:0/826791036 >> 172.21.15.69:0/826791036 conn(0x55fc642543c0 msgr2=0x55fc645fc720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:26:00.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.404+0000 7f0cdaa0a740 1 -- 172.21.15.69:0/826791036 shutdown_connections 2024-04-25T11:26:00.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.404+0000 7f0cdaa0a740 1 -- 172.21.15.69:0/826791036 wait complete. 2024-04-25T11:26:00.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.405+0000 7f0cdaa0a740 20 client.15450 trim_cache size 0 max 16384 2024-04-25T11:26:00.403 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-25T11:26:00.405 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "f7ed73dc-02f4-11ef-bc93-c7b262605968", 2024-04-25T11:26:00.405 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 3, 2024-04-25T11:26:00.405 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 24458, 2024-04-25T11:26:00.405 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-04-25T11:26:00.405 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-04-25T11:26:00.405 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-04-25T11:26:00.405 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 112.49947344900001, 2024-04-25T11:26:00.405 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 28, 2024-04-25T11:26:00.405 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 84, 2024-04-25T11:26:00.406 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 80, 2024-04-25T11:26:00.406 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 166.096850297 2024-04-25T11:26:00.406 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-25T11:26:00.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.405+0000 7f0cd3b57640 1 -- 172.21.15.69:0/4055819729 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f0cb80afaf0 msgr2=0x7f0cb80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:00.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.405+0000 7f0cd3b57640 1 --2- 172.21.15.69:0/4055819729 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f0cb80afaf0 0x7f0cb80b2000 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0cc40c8640 tx=0x7f0cc408eb20 comp rx=0 tx=0).stop 2024-04-25T11:26:00.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.405+0000 7f0cd3b57640 1 -- 172.21.15.69:0/4055819729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ccc109a40 msgr2=0x7f0ccc1a1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:00.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.405+0000 7f0cd3b57640 1 --2- 172.21.15.69:0/4055819729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ccc109a40 0x7f0ccc1a1580 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4005e30 tx=0x7f0cc4003ba0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.405+0000 7f0cd3b57640 1 -- 172.21.15.69:0/4055819729 shutdown_connections 2024-04-25T11:26:00.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.405+0000 7f0cd3b57640 1 --2- 172.21.15.69:0/4055819729 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f0cb80afaf0 0x7f0cb80b2000 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.405+0000 7f0cd3b57640 1 --2- 172.21.15.69:0/4055819729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ccc109a40 0x7f0ccc1a1580 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.405+0000 7f0cd3b57640 1 --2- 172.21.15.69:0/4055819729 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ccc104d40 0x7f0ccc19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.405+0000 7f0cd3b57640 1 --2- 172.21.15.69:0/4055819729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ccc103b00 0x7f0ccc19a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.405+0000 7f0cd3b57640 1 -- 172.21.15.69:0/4055819729 >> 172.21.15.69:0/4055819729 conn(0x7f0ccc0fecf0 msgr2=0x7f0ccc105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:26:00.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.405+0000 7f0cd3b57640 1 -- 172.21.15.69:0/4055819729 shutdown_connections 2024-04-25T11:26:00.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.405+0000 7f0cd3b57640 1 -- 172.21.15.69:0/4055819729 wait complete. 2024-04-25T11:26:00.409 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-25T11:26:00.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.525+0000 7f462bb8f640 1 -- 172.21.15.69:0/1925694633 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4624104d30 msgr2=0x7f4624105140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:00.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.525+0000 7f462bb8f640 1 --2- 172.21.15.69:0/1925694633 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4624104d30 0x7f4624105140 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4620009900 tx=0x7f462002a410 comp rx=0 tx=0).stop 2024-04-25T11:26:00.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.525+0000 7f462bb8f640 1 -- 172.21.15.69:0/1925694633 shutdown_connections 2024-04-25T11:26:00.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.525+0000 7f462bb8f640 1 --2- 172.21.15.69:0/1925694633 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4624109a30 0x7f462410be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.525+0000 7f462bb8f640 1 --2- 172.21.15.69:0/1925694633 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4624104d30 0x7f4624105140 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.525+0000 7f462bb8f640 1 --2- 172.21.15.69:0/1925694633 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4624103af0 0x7f46241047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.525+0000 7f462bb8f640 1 -- 172.21.15.69:0/1925694633 >> 172.21.15.69:0/1925694633 conn(0x7f46240fece0 msgr2=0x7f4624101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:26:00.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.525+0000 7f462bb8f640 1 -- 172.21.15.69:0/1925694633 shutdown_connections 2024-04-25T11:26:00.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.525+0000 7f462bb8f640 1 -- 172.21.15.69:0/1925694633 wait complete. 2024-04-25T11:26:00.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.526+0000 7f462bb8f640 1 Processor -- start 2024-04-25T11:26:00.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.526+0000 7f462bb8f640 1 -- start start 2024-04-25T11:26:00.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.526+0000 7f462bb8f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4624103af0 0x7f462419a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.526+0000 7f462bb8f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4624104d30 0x7f462419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.526+0000 7f462bb8f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4624109a30 0x7f46241a1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.526+0000 7f462bb8f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f462410f4d0 con 0x7f4624109a30 2024-04-25T11:26:00.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.526+0000 7f462bb8f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f462410f350 con 0x7f4624103af0 2024-04-25T11:26:00.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.526+0000 7f462bb8f640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f462410f650 con 0x7f4624104d30 2024-04-25T11:26:00.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.526+0000 7f462a12c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4624109a30 0x7f46241a1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.526+0000 7f462a12c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4624109a30 0x7f46241a1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57252/0 (socket says 172.21.15.69:57252) 2024-04-25T11:26:00.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.526+0000 7f462a12c640 1 -- 172.21.15.69:0/3189731653 learned_addr learned my addr 172.21.15.69:0/3189731653 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:26:00.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.526+0000 7f462992b640 1 --2- 172.21.15.69:0/3189731653 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4624103af0 0x7f462419a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.526+0000 7f462912a640 1 --2- 172.21.15.69:0/3189731653 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4624104d30 0x7f462419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.527+0000 7f462a12c640 1 -- 172.21.15.69:0/3189731653 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4624104d30 msgr2=0x7f462419a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:00.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.527+0000 7f462a12c640 1 --2- 172.21.15.69:0/3189731653 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4624104d30 0x7f462419a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.527+0000 7f462a12c640 1 -- 172.21.15.69:0/3189731653 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4624103af0 msgr2=0x7f462419a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:00.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.527+0000 7f462a12c640 1 --2- 172.21.15.69:0/3189731653 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4624103af0 0x7f462419a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.527+0000 7f462a12c640 1 -- 172.21.15.69:0/3189731653 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46241a1bc0 con 0x7f4624109a30 2024-04-25T11:26:00.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.527+0000 7f462a12c640 1 --2- 172.21.15.69:0/3189731653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4624109a30 0x7f46241a1600 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f461c00eb10 tx=0x7f461c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:26:00.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.527+0000 7f461affd640 1 -- 172.21.15.69:0/3189731653 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c004b20 con 0x7f4624109a30 2024-04-25T11:26:00.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.527+0000 7f461affd640 1 -- 172.21.15.69:0/3189731653 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f461c004cc0 con 0x7f4624109a30 2024-04-25T11:26:00.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.527+0000 7f462bb8f640 1 -- 172.21.15.69:0/3189731653 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46241a1eb0 con 0x7f4624109a30 2024-04-25T11:26:00.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.527+0000 7f461affd640 1 -- 172.21.15.69:0/3189731653 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c010690 con 0x7f4624109a30 2024-04-25T11:26:00.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.527+0000 7f462bb8f640 1 -- 172.21.15.69:0/3189731653 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46241a2310 con 0x7f4624109a30 2024-04-25T11:26:00.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.529+0000 7f461affd640 1 -- 172.21.15.69:0/3189731653 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f461c005020 con 0x7f4624109a30 2024-04-25T11:26:00.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.530+0000 7f4632a41740 1 Processor -- start 2024-04-25T11:26:00.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.530+0000 7f4632a41740 1 -- start start 2024-04-25T11:26:00.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.530+0000 7f4632a41740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b16936270 0x558b16cd7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.530+0000 7f4632a41740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b1690d3b0 0x558b16cd78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.530+0000 7f4632a41740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b16915590 0x558b16cde7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.530+0000 7f4632a41740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558b16c3c740 con 0x558b16936270 2024-04-25T11:26:00.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.530+0000 7f4632a41740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558b16bd83a0 con 0x558b1690d3b0 2024-04-25T11:26:00.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.530+0000 7f4632a41740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x558b16c38700 con 0x558b16915590 2024-04-25T11:26:00.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.530+0000 7f461affd640 1 --2- 172.21.15.69:0/3189731653 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f46100afaf0 0x7f46100b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.530+0000 7f462912a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b16936270 0x558b16cd7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.530+0000 7f462912a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b16936270 0x558b16cd7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57258/0 (socket says 172.21.15.69:57258) 2024-04-25T11:26:00.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.530+0000 7f462912a640 1 -- 172.21.15.69:0/2780480517 learned_addr learned my addr 172.21.15.69:0/2780480517 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:26:00.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.530+0000 7f462a12c640 1 --2- 172.21.15.69:0/3189731653 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f46100afaf0 0x7f46100b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.530+0000 7f462912a640 1 --2- 172.21.15.69:0/2780480517 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b16915590 0x558b16cde7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.531+0000 7f462992b640 1 --2- 172.21.15.69:0/2780480517 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b1690d3b0 0x558b16cd78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.531+0000 7f461affd640 1 -- 172.21.15.69:0/3189731653 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f461c0df680 con 0x7f4624109a30 2024-04-25T11:26:00.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.531+0000 7f462912a640 1 -- 172.21.15.69:0/2780480517 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b16915590 msgr2=0x558b16cde7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:26:00.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.531+0000 7f462912a640 1 --2- 172.21.15.69:0/2780480517 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b16915590 0x558b16cde7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.531+0000 7f462912a640 1 -- 172.21.15.69:0/2780480517 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b1690d3b0 msgr2=0x558b16cd78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:26:00.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.531+0000 7f462912a640 1 --2- 172.21.15.69:0/2780480517 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b1690d3b0 0x558b16cd78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.531+0000 7f462912a640 1 -- 172.21.15.69:0/2780480517 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b16bf4750 con 0x558b16936270 2024-04-25T11:26:00.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.531+0000 7f462912a640 1 --2- 172.21.15.69:0/2780480517 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b16936270 0x558b16cd7370 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4620005760 tx=0x7f4620009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=651742861fc57cac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:26:00.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.531+0000 7f462a12c640 1 --2- 172.21.15.69:0/3189731653 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f46100afaf0 0x7f46100b2000 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f461c0049d0 tx=0x7f461c090040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:26:00.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.531+0000 7f460ffff640 1 -- 172.21.15.69:0/2780480517 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4620035070 con 0x558b16936270 2024-04-25T11:26:00.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.531+0000 7f460ffff640 1 -- 172.21.15.69:0/2780480517 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4620045070 con 0x558b16936270 2024-04-25T11:26:00.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.531+0000 7f460ffff640 1 -- 172.21.15.69:0/2780480517 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4620003ad0 con 0x558b16936270 2024-04-25T11:26:00.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.532+0000 7f4632a41740 1 -- 172.21.15.69:0/2780480517 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b16936270 msgr2=0x558b16cd7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:00.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.532+0000 7f4632a41740 1 --2- 172.21.15.69:0/2780480517 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b16936270 0x558b16cd7370 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4620005760 tx=0x7f4620009660 comp rx=0 tx=0).stop 2024-04-25T11:26:00.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.532+0000 7f4632a41740 1 -- 172.21.15.69:0/2780480517 shutdown_connections 2024-04-25T11:26:00.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.532+0000 7f4632a41740 1 --2- 172.21.15.69:0/2780480517 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b16915590 0x558b16cde7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.532+0000 7f4632a41740 1 --2- 172.21.15.69:0/2780480517 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b1690d3b0 0x558b16cd78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.532+0000 7f4632a41740 1 --2- 172.21.15.69:0/2780480517 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b16936270 0x558b16cd7370 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.532+0000 7f4632a41740 1 -- 172.21.15.69:0/2780480517 >> 172.21.15.69:0/2780480517 conn(0x558b1692a3c0 msgr2=0x558b16cd2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:26:00.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.532+0000 7f4632a41740 1 -- 172.21.15.69:0/2780480517 shutdown_connections 2024-04-25T11:26:00.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.532+0000 7f4632a41740 1 -- 172.21.15.69:0/2780480517 wait complete. 2024-04-25T11:26:00.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.532+0000 7f4632a41740 1 -- start start 2024-04-25T11:26:00.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.532+0000 7f4632a41740 1 Processor -- start 2024-04-25T11:26:00.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.533+0000 7f4632a41740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b1690d3b0 0x558b16ce1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.533+0000 7f4632a41740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b16915590 0x558b16ce17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.533+0000 7f4632a41740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b16936270 0x558b16ce8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.533+0000 7f4632a41740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558b16cdf1e0 con 0x558b16915590 2024-04-25T11:26:00.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.533+0000 7f4632a41740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558b16bc3fe0 con 0x558b16936270 2024-04-25T11:26:00.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.533+0000 7f4632a41740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x558b16cdf360 con 0x558b1690d3b0 2024-04-25T11:26:00.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.533+0000 7f462992b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b16915590 0x558b16ce17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.533+0000 7f4632a41740 10 client.0 _ll_register_callbacks cb 0x558b16bda150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:26:00.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.533+0000 7f462992b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b16915590 0x558b16ce17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57272/0 (socket says 172.21.15.69:57272) 2024-04-25T11:26:00.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.533+0000 7f462992b640 1 -- 172.21.15.69:0/1406420430 learned_addr learned my addr 172.21.15.69:0/1406420430 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:26:00.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.533+0000 7f462912a640 1 --2- 172.21.15.69:0/1406420430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b16936270 0x558b16ce8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.533+0000 7f462912a640 1 --2- 172.21.15.69:0/1406420430 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b1690d3b0 0x558b16ce1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.533+0000 7f462992b640 1 -- 172.21.15.69:0/1406420430 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b1690d3b0 msgr2=0x558b16ce1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:26:00.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.533+0000 7f462992b640 1 --2- 172.21.15.69:0/1406420430 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b1690d3b0 0x558b16ce1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.533+0000 7f462992b640 1 -- 172.21.15.69:0/1406420430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b16936270 msgr2=0x558b16ce8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:26:00.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.533+0000 7f462992b640 1 --2- 172.21.15.69:0/1406420430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b16936270 0x558b16ce8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.533+0000 7f462992b640 1 -- 172.21.15.69:0/1406420430 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b1694e2d0 con 0x558b16915590 2024-04-25T11:26:00.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.533+0000 7f462912a640 1 --2- 172.21.15.69:0/1406420430 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b1690d3b0 0x558b16ce1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:26:00.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.533+0000 7f462992b640 1 --2- 172.21.15.69:0/1406420430 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b16915590 0x558b16ce17b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4614010760 tx=0x7f4614010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a3838eed9631ea2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:26:00.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.533+0000 7f460d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-25T11:26:00.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.534+0000 7f460d7fa640 1 -- 172.21.15.69:0/1406420430 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4614004250 con 0x558b16915590 2024-04-25T11:26:00.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.534+0000 7f460d7fa640 1 -- 172.21.15.69:0/1406420430 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f46140043f0 con 0x558b16915590 2024-04-25T11:26:00.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.534+0000 7f4632a41740 1 -- 172.21.15.69:0/1406420430 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4620045070 con 0x558b16915590 2024-04-25T11:26:00.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.534+0000 7f460d7fa640 1 -- 172.21.15.69:0/1406420430 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4614010e70 con 0x558b16915590 2024-04-25T11:26:00.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.534+0000 7f4632a41740 10 client.15462 fetch_fsmap learned FSMap version 28 2024-04-25T11:26:00.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.534+0000 7f4632a41740 1 -- 172.21.15.69:0/1406420430 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558b16a186b0 con 0x558b16915590 2024-04-25T11:26:00.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.534+0000 7f460d7fa640 1 -- 172.21.15.69:0/1406420430 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4620045070 con 0x558b16915590 2024-04-25T11:26:00.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.534+0000 7f460d7fa640 1 -- 172.21.15.69:0/1406420430 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f46140047a0 con 0x558b16915590 2024-04-25T11:26:00.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.534+0000 7f4632a41740 10 client.15462 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T11:26:00.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.534+0000 7f4632a41740 10 client.15462 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T11:26:00.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.534+0000 7f4632a41740 20 client.15462 populate_metadata read hostname 'smithi069' 2024-04-25T11:26:00.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.534+0000 7f4632a41740 1 --2- 172.21.15.69:0/1406420430 >> [v2:172.21.15.152:6838/3008615657,v1:172.21.15.152:6839/3008615657] conn(0x558b168f4e40 0x558b16cef040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.534+0000 7f4632a41740 4 client.15462 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:26:00.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.534+0000 7f4632a41740 1 -- 172.21.15.69:0/1406420430 --> [v2:172.21.15.152:6838/3008615657,v1:172.21.15.152:6839/3008615657] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b16c4a350 con 0x558b168f4e40 2024-04-25T11:26:00.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.535+0000 7f462912a640 1 --2- 172.21.15.69:0/1406420430 >> [v2:172.21.15.152:6838/3008615657,v1:172.21.15.152:6839/3008615657] conn(0x558b168f4e40 0x558b16cef040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.535+0000 7f460d7fa640 10 client.15462 ms_handle_connect on v2:172.21.15.152:6838/3008615657 2024-04-25T11:26:00.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.535+0000 7f462912a640 1 --2- 172.21.15.69:0/1406420430 >> [v2:172.21.15.152:6838/3008615657,v1:172.21.15.152:6839/3008615657] conn(0x558b168f4e40 0x558b16cef040 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9d611d5a301a38a3 server_cookie=8cdad724335bf86e in_seq=0 out_seq=0 2024-04-25T11:26:00.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.536+0000 7f460d7fa640 1 -- 172.21.15.69:0/1406420430 <== mds.2 v2:172.21.15.152:6838/3008615657 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558b16c4a350 con 0x558b168f4e40 2024-04-25T11:26:00.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.536+0000 7f460d7fa640 10 client.15462 handle_command_reply: tid=0 2024-04-25T11:26:00.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.536+0000 7f4632a41740 1 client.15462 shutdown 2024-04-25T11:26:00.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.536+0000 7f4632a41740 1 -- 172.21.15.69:0/1406420430 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b16915590 msgr2=0x558b16ce17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:00.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.536+0000 7f4632a41740 1 --2- 172.21.15.69:0/1406420430 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b16915590 0x558b16ce17b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4614010760 tx=0x7f4614010c20 comp rx=0 tx=0).stop 2024-04-25T11:26:00.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.536+0000 7f4632a41740 1 -- 172.21.15.69:0/1406420430 shutdown_connections 2024-04-25T11:26:00.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.536+0000 7f4632a41740 1 --2- 172.21.15.69:0/1406420430 >> [v2:172.21.15.152:6838/3008615657,v1:172.21.15.152:6839/3008615657] conn(0x558b168f4e40 0x558b16cef040 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.536+0000 7f4632a41740 1 --2- 172.21.15.69:0/1406420430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b16936270 0x558b16ce8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.536+0000 7f4632a41740 1 --2- 172.21.15.69:0/1406420430 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b16915590 0x558b16ce17b0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.536+0000 7f4632a41740 1 --2- 172.21.15.69:0/1406420430 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b1690d3b0 0x558b16ce1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.536+0000 7f460d7fa640 0 client.15462 ms_handle_reset on v2:172.21.15.152:6838/3008615657 2024-04-25T11:26:00.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.536+0000 7f4632a41740 1 -- 172.21.15.69:0/1406420430 >> 172.21.15.69:0/1406420430 conn(0x558b1692a3c0 msgr2=0x558b16cd1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:26:00.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.536+0000 7f4632a41740 1 -- 172.21.15.69:0/1406420430 shutdown_connections 2024-04-25T11:26:00.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.536+0000 7f4632a41740 1 -- 172.21.15.69:0/1406420430 wait complete. 2024-04-25T11:26:00.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.537+0000 7f4632a41740 20 client.15462 trim_cache size 0 max 16384 2024-04-25T11:26:00.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.555+0000 7f4632a41740 1 Processor -- start 2024-04-25T11:26:00.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.555+0000 7f4632a41740 1 -- start start 2024-04-25T11:26:00.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.555+0000 7f4632a41740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b1690d3b0 0x558b16cd26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.555+0000 7f4632a41740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b16915590 0x558b16ce8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.555+0000 7f4632a41740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b16936270 0x558b16cd2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.555+0000 7f4632a41740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558b16ced380 con 0x558b1690d3b0 2024-04-25T11:26:00.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.555+0000 7f4632a41740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558b16ceda50 con 0x558b16936270 2024-04-25T11:26:00.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.555+0000 7f4632a41740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x558b16cee120 con 0x558b16915590 2024-04-25T11:26:00.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.555+0000 7f462912a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b1690d3b0 0x558b16cd26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.555+0000 7f462912a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b1690d3b0 0x558b16cd26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57278/0 (socket says 172.21.15.69:57278) 2024-04-25T11:26:00.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.555+0000 7f462912a640 1 -- 172.21.15.69:0/3378749132 learned_addr learned my addr 172.21.15.69:0/3378749132 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:26:00.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.555+0000 7f462912a640 1 -- 172.21.15.69:0/3378749132 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b16915590 msgr2=0x558b16ce8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:26:00.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.555+0000 7f462912a640 1 --2- 172.21.15.69:0/3378749132 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b16915590 0x558b16ce8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.555+0000 7f462912a640 1 -- 172.21.15.69:0/3378749132 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b16936270 msgr2=0x558b16cd2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:26:00.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.555+0000 7f462912a640 1 --2- 172.21.15.69:0/3378749132 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b16936270 0x558b16cd2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.555+0000 7f462912a640 1 -- 172.21.15.69:0/3378749132 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b16cd7f30 con 0x558b1690d3b0 2024-04-25T11:26:00.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.556+0000 7f462912a640 1 --2- 172.21.15.69:0/3378749132 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b1690d3b0 0x558b16cd26f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f462002a930 tx=0x7f4620031870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18aa96966c5f695 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:26:00.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.556+0000 7f460f7fe640 1 -- 172.21.15.69:0/3378749132 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4620035040 con 0x558b1690d3b0 2024-04-25T11:26:00.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.556+0000 7f460f7fe640 1 -- 172.21.15.69:0/3378749132 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f462004f070 con 0x558b1690d3b0 2024-04-25T11:26:00.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.556+0000 7f460f7fe640 1 -- 172.21.15.69:0/3378749132 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462003a910 con 0x558b1690d3b0 2024-04-25T11:26:00.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.556+0000 7f4632a41740 1 -- 172.21.15.69:0/3378749132 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b1690d3b0 msgr2=0x558b16cd26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:00.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.556+0000 7f4632a41740 1 --2- 172.21.15.69:0/3378749132 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b1690d3b0 0x558b16cd26f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f462002a930 tx=0x7f4620031870 comp rx=0 tx=0).stop 2024-04-25T11:26:00.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.556+0000 7f4632a41740 1 -- 172.21.15.69:0/3378749132 shutdown_connections 2024-04-25T11:26:00.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.556+0000 7f4632a41740 1 --2- 172.21.15.69:0/3378749132 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b16936270 0x558b16cd2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.556+0000 7f4632a41740 1 --2- 172.21.15.69:0/3378749132 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b16915590 0x558b16ce8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.556+0000 7f4632a41740 1 --2- 172.21.15.69:0/3378749132 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b1690d3b0 0x558b16cd26f0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.556+0000 7f4632a41740 1 -- 172.21.15.69:0/3378749132 >> 172.21.15.69:0/3378749132 conn(0x558b1692a3c0 msgr2=0x558b16cdf8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:26:00.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.557+0000 7f4632a41740 1 -- 172.21.15.69:0/3378749132 shutdown_connections 2024-04-25T11:26:00.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.557+0000 7f4632a41740 1 -- 172.21.15.69:0/3378749132 wait complete. 2024-04-25T11:26:00.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.557+0000 7f4632a41740 1 -- start start 2024-04-25T11:26:00.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.557+0000 7f4632a41740 1 Processor -- start 2024-04-25T11:26:00.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.557+0000 7f4632a41740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b1690d3b0 0x558b16ce3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.557+0000 7f4632a41740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b16915590 0x558b16ce3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.557+0000 7f4632a41740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b16936270 0x558b16d55eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.557+0000 7f4632a41740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558b16ceeec0 con 0x558b16915590 2024-04-25T11:26:00.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.557+0000 7f4632a41740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558b16cee7f0 con 0x558b1690d3b0 2024-04-25T11:26:00.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.557+0000 7f4632a41740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x558b16cef590 con 0x558b16936270 2024-04-25T11:26:00.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.557+0000 7f4632a41740 10 client.0 _ll_register_callbacks cb 0x7f461400c410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:26:00.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.557+0000 7f462992b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b16915590 0x558b16ce3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.557+0000 7f462992b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b16915590 0x558b16ce3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57280/0 (socket says 172.21.15.69:57280) 2024-04-25T11:26:00.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.557+0000 7f462992b640 1 -- 172.21.15.69:0/1981213315 learned_addr learned my addr 172.21.15.69:0/1981213315 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:26:00.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.557+0000 7f462912a640 1 --2- 172.21.15.69:0/1981213315 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b1690d3b0 0x558b16ce3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.558+0000 7f462912a640 1 --2- 172.21.15.69:0/1981213315 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b16936270 0x558b16d55eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.558+0000 7f462992b640 1 -- 172.21.15.69:0/1981213315 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b16936270 msgr2=0x558b16d55eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:26:00.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.558+0000 7f462992b640 1 --2- 172.21.15.69:0/1981213315 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b16936270 0x558b16d55eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.558+0000 7f462992b640 1 -- 172.21.15.69:0/1981213315 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b1690d3b0 msgr2=0x558b16ce3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:26:00.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.558+0000 7f462992b640 1 --2- 172.21.15.69:0/1981213315 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b1690d3b0 0x558b16ce3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.558+0000 7f462992b640 1 -- 172.21.15.69:0/1981213315 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b16cdc0f0 con 0x558b16915590 2024-04-25T11:26:00.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.558+0000 7f462912a640 1 --2- 172.21.15.69:0/1981213315 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b16936270 0x558b16d55eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:26:00.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.558+0000 7f462992b640 1 --2- 172.21.15.69:0/1981213315 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b16915590 0x558b16ce3b70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f46140058c0 tx=0x7f461400ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32d31a02375583a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:26:00.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.558+0000 7f460d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-25T11:26:00.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.558+0000 7f460d7fa640 1 -- 172.21.15.69:0/1981213315 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4614005000 con 0x558b16915590 2024-04-25T11:26:00.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.558+0000 7f460d7fa640 1 -- 172.21.15.69:0/1981213315 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f46140051a0 con 0x558b16915590 2024-04-25T11:26:00.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.558+0000 7f4632a41740 1 -- 172.21.15.69:0/1981213315 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f462004f070 con 0x558b16915590 2024-04-25T11:26:00.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.558+0000 7f460d7fa640 1 -- 172.21.15.69:0/1981213315 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4614036050 con 0x558b16915590 2024-04-25T11:26:00.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.558+0000 7f4632a41740 10 client.15468 fetch_fsmap learned FSMap version 28 2024-04-25T11:26:00.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.558+0000 7f4632a41740 1 -- 172.21.15.69:0/1981213315 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558b16d574d0 con 0x558b16915590 2024-04-25T11:26:00.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.558+0000 7f460d7fa640 1 -- 172.21.15.69:0/1981213315 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f462004f070 con 0x558b16915590 2024-04-25T11:26:00.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.558+0000 7f460d7fa640 1 -- 172.21.15.69:0/1981213315 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4614003c80 con 0x558b16915590 2024-04-25T11:26:00.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.558+0000 7f4632a41740 10 client.15468 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T11:26:00.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.559+0000 7f4632a41740 10 client.15468 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T11:26:00.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.559+0000 7f4632a41740 20 client.15468 populate_metadata read hostname 'smithi069' 2024-04-25T11:26:00.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.559+0000 7f4632a41740 1 --2- 172.21.15.69:0/1981213315 >> [v2:172.21.15.152:6838/3008615657,v1:172.21.15.152:6839/3008615657] conn(0x558b168ca280 0x558b16d5ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.559+0000 7f4632a41740 4 client.15468 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:26:00.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.559+0000 7f4632a41740 1 -- 172.21.15.69:0/1981213315 --> [v2:172.21.15.152:6838/3008615657,v1:172.21.15.152:6839/3008615657] -- command(tid 0: {"prefix": "status"}) -- 0x558b16c441c0 con 0x558b168ca280 2024-04-25T11:26:00.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.559+0000 7f462912a640 1 --2- 172.21.15.69:0/1981213315 >> [v2:172.21.15.152:6838/3008615657,v1:172.21.15.152:6839/3008615657] conn(0x558b168ca280 0x558b16d5ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.559+0000 7f460d7fa640 10 client.15468 ms_handle_connect on v2:172.21.15.152:6838/3008615657 2024-04-25T11:26:00.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.559+0000 7f462912a640 1 --2- 172.21.15.69:0/1981213315 >> [v2:172.21.15.152:6838/3008615657,v1:172.21.15.152:6839/3008615657] conn(0x558b168ca280 0x558b16d5ade0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8b679b9950949fc3 server_cookie=75b9aaeb0a2369ee in_seq=0 out_seq=0 2024-04-25T11:26:00.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.559+0000 7f460d7fa640 1 -- 172.21.15.69:0/1981213315 <== mds.2 v2:172.21.15.152:6838/3008615657 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x558b16c441c0 con 0x558b168ca280 2024-04-25T11:26:00.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.559+0000 7f460d7fa640 10 client.15468 handle_command_reply: tid=0 2024-04-25T11:26:00.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.559+0000 7f4632a41740 1 client.15468 shutdown 2024-04-25T11:26:00.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.560+0000 7f4632a41740 1 -- 172.21.15.69:0/1981213315 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b16915590 msgr2=0x558b16ce3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:00.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.560+0000 7f4632a41740 1 --2- 172.21.15.69:0/1981213315 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b16915590 0x558b16ce3b70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f46140058c0 tx=0x7f461400ebe0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.560+0000 7f4632a41740 1 -- 172.21.15.69:0/1981213315 shutdown_connections 2024-04-25T11:26:00.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.560+0000 7f4632a41740 1 --2- 172.21.15.69:0/1981213315 >> [v2:172.21.15.152:6838/3008615657,v1:172.21.15.152:6839/3008615657] conn(0x558b168ca280 0x558b16d5ade0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.560+0000 7f4632a41740 1 --2- 172.21.15.69:0/1981213315 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b16936270 0x558b16d55eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.560+0000 7f4632a41740 1 --2- 172.21.15.69:0/1981213315 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b16915590 0x558b16ce3b70 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.560+0000 7f4632a41740 1 --2- 172.21.15.69:0/1981213315 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558b1690d3b0 0x558b16ce3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.560+0000 7f460d7fa640 0 client.15468 ms_handle_reset on v2:172.21.15.152:6838/3008615657 2024-04-25T11:26:00.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.560+0000 7f4632a41740 1 -- 172.21.15.69:0/1981213315 >> 172.21.15.69:0/1981213315 conn(0x558b1692a3c0 msgr2=0x558b16cd26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:26:00.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.560+0000 7f4632a41740 1 -- 172.21.15.69:0/1981213315 shutdown_connections 2024-04-25T11:26:00.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.560+0000 7f4632a41740 1 -- 172.21.15.69:0/1981213315 wait complete. 2024-04-25T11:26:00.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.560+0000 7f4632a41740 20 client.15468 trim_cache size 0 max 16384 2024-04-25T11:26:00.559 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-25T11:26:00.559 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "f7ed73dc-02f4-11ef-bc93-c7b262605968", 2024-04-25T11:26:00.559 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 2, 2024-04-25T11:26:00.560 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 24475, 2024-04-25T11:26:00.560 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-04-25T11:26:00.560 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-04-25T11:26:00.560 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-04-25T11:26:00.560 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 113.66755669299999, 2024-04-25T11:26:00.560 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 28, 2024-04-25T11:26:00.560 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 84, 2024-04-25T11:26:00.560 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 80, 2024-04-25T11:26:00.560 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 154.60864086000001 2024-04-25T11:26:00.560 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-25T11:26:00.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.560+0000 7f462bb8f640 1 -- 172.21.15.69:0/3189731653 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f46100afaf0 msgr2=0x7f46100b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:00.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.560+0000 7f462bb8f640 1 --2- 172.21.15.69:0/3189731653 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f46100afaf0 0x7f46100b2000 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f461c0049d0 tx=0x7f461c090040 comp rx=0 tx=0).stop 2024-04-25T11:26:00.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.560+0000 7f462bb8f640 1 -- 172.21.15.69:0/3189731653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4624109a30 msgr2=0x7f46241a1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:00.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.560+0000 7f462bb8f640 1 --2- 172.21.15.69:0/3189731653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4624109a30 0x7f46241a1600 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f461c00eb10 tx=0x7f461c00c2d0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.560+0000 7f462bb8f640 1 -- 172.21.15.69:0/3189731653 shutdown_connections 2024-04-25T11:26:00.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.560+0000 7f462bb8f640 1 --2- 172.21.15.69:0/3189731653 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f46100afaf0 0x7f46100b2000 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.560+0000 7f462bb8f640 1 --2- 172.21.15.69:0/3189731653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4624109a30 0x7f46241a1600 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.560+0000 7f462bb8f640 1 --2- 172.21.15.69:0/3189731653 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4624104d30 0x7f462419a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.560+0000 7f462bb8f640 1 --2- 172.21.15.69:0/3189731653 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4624103af0 0x7f462419a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.560+0000 7f462bb8f640 1 -- 172.21.15.69:0/3189731653 >> 172.21.15.69:0/3189731653 conn(0x7f46240fece0 msgr2=0x7f4624105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:26:00.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.560+0000 7f462bb8f640 1 -- 172.21.15.69:0/3189731653 shutdown_connections 2024-04-25T11:26:00.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.560+0000 7f462bb8f640 1 -- 172.21.15.69:0/3189731653 wait complete. 2024-04-25T11:26:00.564 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-25T11:26:00.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.680+0000 7fe07ffce640 1 -- 172.21.15.69:0/4201606756 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0781029e0 msgr2=0x7fe0781036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:00.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.680+0000 7fe07ffce640 1 --2- 172.21.15.69:0/4201606756 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0781029e0 0x7fe0781036e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe074009900 tx=0x7fe07402a410 comp rx=0 tx=0).stop 2024-04-25T11:26:00.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.680+0000 7fe07ffce640 1 -- 172.21.15.69:0/4201606756 shutdown_connections 2024-04-25T11:26:00.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.680+0000 7fe07ffce640 1 --2- 172.21.15.69:0/4201606756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe078108920 0x7fe07810ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.680+0000 7fe07ffce640 1 --2- 172.21.15.69:0/4201606756 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe078103c20 0x7fe078104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.680+0000 7fe07ffce640 1 --2- 172.21.15.69:0/4201606756 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe0781029e0 0x7fe0781036e0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.680+0000 7fe07ffce640 1 -- 172.21.15.69:0/4201606756 >> 172.21.15.69:0/4201606756 conn(0x7fe0780fdbd0 msgr2=0x7fe078100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:26:00.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.680+0000 7fe07ffce640 1 -- 172.21.15.69:0/4201606756 shutdown_connections 2024-04-25T11:26:00.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.680+0000 7fe07ffce640 1 -- 172.21.15.69:0/4201606756 wait complete. 2024-04-25T11:26:00.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.680+0000 7fe07ffce640 1 Processor -- start 2024-04-25T11:26:00.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.681+0000 7fe07ffce640 1 -- start start 2024-04-25T11:26:00.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.681+0000 7fe07ffce640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0781029e0 0x7fe07819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.681+0000 7fe07ffce640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe078103c20 0x7fe07819a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.681+0000 7fe07ffce640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe078108920 0x7fe0781a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.681+0000 7fe07ffce640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe07810e370 con 0x7fe078103c20 2024-04-25T11:26:00.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.681+0000 7fe07ffce640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe07810e1f0 con 0x7fe0781029e0 2024-04-25T11:26:00.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.681+0000 7fe07ffce640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe07810e4f0 con 0x7fe078108920 2024-04-25T11:26:00.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.681+0000 7fe07d569640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe078103c20 0x7fe07819a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.681+0000 7fe07d569640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe078103c20 0x7fe07819a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57306/0 (socket says 172.21.15.69:57306) 2024-04-25T11:26:00.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.681+0000 7fe07d569640 1 -- 172.21.15.69:0/1201228869 learned_addr learned my addr 172.21.15.69:0/1201228869 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:26:00.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.681+0000 7fe07dd6a640 1 --2- 172.21.15.69:0/1201228869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0781029e0 0x7fe07819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.681+0000 7fe07e56b640 1 --2- 172.21.15.69:0/1201228869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe078108920 0x7fe0781a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.681+0000 7fe07d569640 1 -- 172.21.15.69:0/1201228869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe078108920 msgr2=0x7fe0781a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:00.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.681+0000 7fe07d569640 1 --2- 172.21.15.69:0/1201228869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe078108920 0x7fe0781a1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.681+0000 7fe07d569640 1 -- 172.21.15.69:0/1201228869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0781029e0 msgr2=0x7fe07819a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:00.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.681+0000 7fe07d569640 1 --2- 172.21.15.69:0/1201228869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0781029e0 0x7fe07819a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.681+0000 7fe07d569640 1 -- 172.21.15.69:0/1201228869 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0781a1fa0 con 0x7fe078103c20 2024-04-25T11:26:00.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.681+0000 7fe07dd6a640 1 --2- 172.21.15.69:0/1201228869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0781029e0 0x7fe07819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:26:00.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.681+0000 7fe07d569640 1 --2- 172.21.15.69:0/1201228869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe078103c20 0x7fe07819a9a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe06800d950 tx=0x7fe06800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:26:00.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.681+0000 7fe06effd640 1 -- 172.21.15.69:0/1201228869 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe068017030 con 0x7fe078103c20 2024-04-25T11:26:00.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.681+0000 7fe06effd640 1 -- 172.21.15.69:0/1201228869 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe068004020 con 0x7fe078103c20 2024-04-25T11:26:00.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.681+0000 7fe07ffce640 1 -- 172.21.15.69:0/1201228869 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0781a2290 con 0x7fe078103c20 2024-04-25T11:26:00.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.681+0000 7fe06effd640 1 -- 172.21.15.69:0/1201228869 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0680055d0 con 0x7fe078103c20 2024-04-25T11:26:00.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.682+0000 7fe07ffce640 1 -- 172.21.15.69:0/1201228869 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0781a2830 con 0x7fe078103c20 2024-04-25T11:26:00.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.684+0000 7fe086e7d740 1 Processor -- start 2024-04-25T11:26:00.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.684+0000 7fe06effd640 1 -- 172.21.15.69:0/1201228869 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fe068009a30 con 0x7fe078103c20 2024-04-25T11:26:00.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.684+0000 7fe086e7d740 1 -- start start 2024-04-25T11:26:00.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.684+0000 7fe086e7d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e00829270 0x562e00bca370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.684+0000 7fe086e7d740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562e008003b0 0x562e00bca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.684+0000 7fe086e7d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e00808590 0x562e00bd17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.684+0000 7fe086e7d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562e00b2f740 con 0x562e00829270 2024-04-25T11:26:00.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.684+0000 7fe086e7d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x562e00acb3a0 con 0x562e00808590 2024-04-25T11:26:00.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.684+0000 7fe086e7d740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x562e00b2b700 con 0x562e008003b0 2024-04-25T11:26:00.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.685+0000 7fe07e56b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e00829270 0x562e00bca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.685+0000 7fe07e56b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e00829270 0x562e00bca370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57316/0 (socket says 172.21.15.69:57316) 2024-04-25T11:26:00.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.685+0000 7fe07e56b640 1 -- 172.21.15.69:0/2089981187 learned_addr learned my addr 172.21.15.69:0/2089981187 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:26:00.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.685+0000 7fe07d569640 1 --2- 172.21.15.69:0/2089981187 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e00808590 0x562e00bd17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.685+0000 7fe07dd6a640 1 --2- 172.21.15.69:0/2089981187 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562e008003b0 0x562e00bca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.685+0000 7fe06effd640 1 --2- 172.21.15.69:0/1201228869 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fe0640afb70 0x7fe0640b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.685+0000 7fe07e56b640 1 -- 172.21.15.69:0/2089981187 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562e008003b0 msgr2=0x562e00bca8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:26:00.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.685+0000 7fe07e56b640 1 --2- 172.21.15.69:0/2089981187 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562e008003b0 0x562e00bca8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.685+0000 7fe07e56b640 1 -- 172.21.15.69:0/2089981187 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e00808590 msgr2=0x562e00bd17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:26:00.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.685+0000 7fe07e56b640 1 --2- 172.21.15.69:0/2089981187 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e00808590 0x562e00bd17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.685+0000 7fe07e56b640 1 -- 172.21.15.69:0/2089981187 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e00ae7750 con 0x562e00829270 2024-04-25T11:26:00.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.685+0000 7fe06effd640 1 -- 172.21.15.69:0/1201228869 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7fe0680dea90 con 0x7fe078103c20 2024-04-25T11:26:00.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.685+0000 7fe07e56b640 1 --2- 172.21.15.69:0/1201228869 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fe0640afb70 0x7fe0640b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.685+0000 7fe07e56b640 1 --2- 172.21.15.69:0/2089981187 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e00829270 0x562e00bca370 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe070009e60 tx=0x7fe070010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c0b42c6de6e8789 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:26:00.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.685+0000 7fe063fff640 1 -- 172.21.15.69:0/2089981187 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe070021070 con 0x562e00829270 2024-04-25T11:26:00.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.686+0000 7fe063fff640 1 -- 172.21.15.69:0/2089981187 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe07001be20 con 0x562e00829270 2024-04-25T11:26:00.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.686+0000 7fe063fff640 1 -- 172.21.15.69:0/2089981187 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe070016070 con 0x562e00829270 2024-04-25T11:26:00.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.686+0000 7fe086e7d740 1 -- 172.21.15.69:0/2089981187 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e00829270 msgr2=0x562e00bca370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:00.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.686+0000 7fe07e56b640 1 --2- 172.21.15.69:0/1201228869 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fe0640afb70 0x7fe0640b2080 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe070011010 tx=0x7fe070021000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:26:00.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.686+0000 7fe086e7d740 1 --2- 172.21.15.69:0/2089981187 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e00829270 0x562e00bca370 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe070009e60 tx=0x7fe070010740 comp rx=0 tx=0).stop 2024-04-25T11:26:00.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.686+0000 7fe086e7d740 1 -- 172.21.15.69:0/2089981187 shutdown_connections 2024-04-25T11:26:00.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.686+0000 7fe086e7d740 1 --2- 172.21.15.69:0/2089981187 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e00808590 0x562e00bd17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.686+0000 7fe086e7d740 1 --2- 172.21.15.69:0/2089981187 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562e008003b0 0x562e00bca8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.686+0000 7fe086e7d740 1 --2- 172.21.15.69:0/2089981187 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e00829270 0x562e00bca370 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.686+0000 7fe086e7d740 1 -- 172.21.15.69:0/2089981187 >> 172.21.15.69:0/2089981187 conn(0x562e0081d3c0 msgr2=0x562e00bc5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:26:00.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.686+0000 7fe086e7d740 1 -- 172.21.15.69:0/2089981187 shutdown_connections 2024-04-25T11:26:00.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.687+0000 7fe086e7d740 1 -- 172.21.15.69:0/2089981187 wait complete. 2024-04-25T11:26:00.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.687+0000 7fe086e7d740 1 -- start start 2024-04-25T11:26:00.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.687+0000 7fe086e7d740 1 Processor -- start 2024-04-25T11:26:00.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.687+0000 7fe086e7d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e008003b0 0x562e00bd4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.687+0000 7fe086e7d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e00808590 0x562e00bd4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.687+0000 7fe086e7d740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562e00829270 0x562e00bdb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.687+0000 7fe086e7d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562e00bd2240 con 0x562e008003b0 2024-04-25T11:26:00.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.687+0000 7fe086e7d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x562e00ab6fe0 con 0x562e00808590 2024-04-25T11:26:00.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.687+0000 7fe086e7d740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x562e00bd23c0 con 0x562e00829270 2024-04-25T11:26:00.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.687+0000 7fe086e7d740 10 client.0 _ll_register_callbacks cb 0x562e00acd150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:26:00.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.687+0000 7fe07dd6a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e008003b0 0x562e00bd4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.687+0000 7fe07dd6a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e008003b0 0x562e00bd4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57322/0 (socket says 172.21.15.69:57322) 2024-04-25T11:26:00.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.687+0000 7fe07dd6a640 1 -- 172.21.15.69:0/3828222286 learned_addr learned my addr 172.21.15.69:0/3828222286 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:26:00.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.688+0000 7fe07e56b640 1 --2- 172.21.15.69:0/3828222286 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562e00829270 0x562e00bdb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.688+0000 7fe07d569640 1 --2- 172.21.15.69:0/3828222286 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e00808590 0x562e00bd4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.688+0000 7fe07dd6a640 1 -- 172.21.15.69:0/3828222286 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562e00829270 msgr2=0x562e00bdb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:26:00.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.688+0000 7fe07dd6a640 1 --2- 172.21.15.69:0/3828222286 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562e00829270 0x562e00bdb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.688+0000 7fe07dd6a640 1 -- 172.21.15.69:0/3828222286 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e00808590 msgr2=0x562e00bd4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:26:00.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.688+0000 7fe07dd6a640 1 --2- 172.21.15.69:0/3828222286 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e00808590 0x562e00bd4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.688+0000 7fe07dd6a640 1 -- 172.21.15.69:0/3828222286 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e008412d0 con 0x562e008003b0 2024-04-25T11:26:00.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.688+0000 7fe07dd6a640 1 --2- 172.21.15.69:0/3828222286 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e008003b0 0x562e00bd4220 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe074009900 tx=0x7fe074031990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f57e683aa5ea05f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:26:00.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.688+0000 7fe0617fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-25T11:26:00.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.688+0000 7fe0617fa640 1 -- 172.21.15.69:0/3828222286 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe074035040 con 0x562e008003b0 2024-04-25T11:26:00.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.688+0000 7fe0617fa640 1 -- 172.21.15.69:0/3828222286 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7fe074004040 con 0x562e008003b0 2024-04-25T11:26:00.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.688+0000 7fe086e7d740 1 -- 172.21.15.69:0/3828222286 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe07001be20 con 0x562e008003b0 2024-04-25T11:26:00.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.688+0000 7fe0617fa640 1 -- 172.21.15.69:0/3828222286 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe07404e050 con 0x562e008003b0 2024-04-25T11:26:00.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.688+0000 7fe086e7d740 10 client.15480 fetch_fsmap learned FSMap version 28 2024-04-25T11:26:00.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.688+0000 7fe086e7d740 1 -- 172.21.15.69:0/3828222286 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562e0090b6b0 con 0x562e008003b0 2024-04-25T11:26:00.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.688+0000 7fe0617fa640 1 -- 172.21.15.69:0/3828222286 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe07001be20 con 0x562e008003b0 2024-04-25T11:26:00.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.689+0000 7fe0617fa640 1 -- 172.21.15.69:0/3828222286 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe07403d450 con 0x562e008003b0 2024-04-25T11:26:00.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.689+0000 7fe086e7d740 10 client.15480 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T11:26:00.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.689+0000 7fe086e7d740 10 client.15480 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T11:26:00.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.689+0000 7fe086e7d740 20 client.15480 populate_metadata read hostname 'smithi069' 2024-04-25T11:26:00.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.689+0000 7fe086e7d740 1 --2- 172.21.15.69:0/3828222286 >> [v2:172.21.15.146:6836/2730317134,v1:172.21.15.146:6837/2730317134] conn(0x562e007e7e40 0x562e00be1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.689+0000 7fe086e7d740 4 client.15480 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:26:00.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.689+0000 7fe086e7d740 1 -- 172.21.15.69:0/3828222286 --> [v2:172.21.15.146:6836/2730317134,v1:172.21.15.146:6837/2730317134] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e00b3d350 con 0x562e007e7e40 2024-04-25T11:26:00.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.689+0000 7fe07d569640 1 --2- 172.21.15.69:0/3828222286 >> [v2:172.21.15.146:6836/2730317134,v1:172.21.15.146:6837/2730317134] conn(0x562e007e7e40 0x562e00be1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.689+0000 7fe0617fa640 10 client.15480 ms_handle_connect on v2:172.21.15.146:6836/2730317134 2024-04-25T11:26:00.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.689+0000 7fe07d569640 1 --2- 172.21.15.69:0/3828222286 >> [v2:172.21.15.146:6836/2730317134,v1:172.21.15.146:6837/2730317134] conn(0x562e007e7e40 0x562e00be1ef0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=73a74e8e0b911dec server_cookie=65b98a4ed969d6c7 in_seq=0 out_seq=0 2024-04-25T11:26:00.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.690+0000 7fe0617fa640 1 -- 172.21.15.69:0/3828222286 <== mds.1 v2:172.21.15.146:6836/2730317134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562e00b3d350 con 0x562e007e7e40 2024-04-25T11:26:00.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.690+0000 7fe0617fa640 10 client.15480 handle_command_reply: tid=0 2024-04-25T11:26:00.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.690+0000 7fe086e7d740 1 client.15480 shutdown 2024-04-25T11:26:00.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.690+0000 7fe086e7d740 1 -- 172.21.15.69:0/3828222286 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e008003b0 msgr2=0x562e00bd4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:00.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.690+0000 7fe086e7d740 1 --2- 172.21.15.69:0/3828222286 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e008003b0 0x562e00bd4220 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe074009900 tx=0x7fe074031990 comp rx=0 tx=0).stop 2024-04-25T11:26:00.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.691+0000 7fe086e7d740 1 -- 172.21.15.69:0/3828222286 shutdown_connections 2024-04-25T11:26:00.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.691+0000 7fe086e7d740 1 --2- 172.21.15.69:0/3828222286 >> [v2:172.21.15.146:6836/2730317134,v1:172.21.15.146:6837/2730317134] conn(0x562e007e7e40 0x562e00be1ef0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.691+0000 7fe086e7d740 1 --2- 172.21.15.69:0/3828222286 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562e00829270 0x562e00bdb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.691+0000 7fe086e7d740 1 --2- 172.21.15.69:0/3828222286 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e00808590 0x562e00bd4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.691+0000 7fe086e7d740 1 --2- 172.21.15.69:0/3828222286 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e008003b0 0x562e00bd4220 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.691+0000 7fe0617fa640 0 client.15480 ms_handle_reset on v2:172.21.15.146:6836/2730317134 2024-04-25T11:26:00.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.691+0000 7fe086e7d740 1 -- 172.21.15.69:0/3828222286 >> 172.21.15.69:0/3828222286 conn(0x562e0081d3c0 msgr2=0x562e00bc4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:26:00.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.691+0000 7fe086e7d740 1 -- 172.21.15.69:0/3828222286 shutdown_connections 2024-04-25T11:26:00.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.691+0000 7fe086e7d740 1 -- 172.21.15.69:0/3828222286 wait complete. 2024-04-25T11:26:00.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.691+0000 7fe086e7d740 20 client.15480 trim_cache size 0 max 16384 2024-04-25T11:26:00.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.709+0000 7fe086e7d740 1 Processor -- start 2024-04-25T11:26:00.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.709+0000 7fe086e7d740 1 -- start start 2024-04-25T11:26:00.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.709+0000 7fe086e7d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e008003b0 0x562e00bd2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.709+0000 7fe086e7d740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562e00808590 0x562e00bdb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.709+0000 7fe086e7d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e00829270 0x562e00bc0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.709+0000 7fe086e7d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562e00be0230 con 0x562e008003b0 2024-04-25T11:26:00.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.709+0000 7fe086e7d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x562e00be0900 con 0x562e00829270 2024-04-25T11:26:00.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.709+0000 7fe086e7d740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x562e00be0fd0 con 0x562e00808590 2024-04-25T11:26:00.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.710+0000 7fe07dd6a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e008003b0 0x562e00bd2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.710+0000 7fe07dd6a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e008003b0 0x562e00bd2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57332/0 (socket says 172.21.15.69:57332) 2024-04-25T11:26:00.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.710+0000 7fe07dd6a640 1 -- 172.21.15.69:0/2724645937 learned_addr learned my addr 172.21.15.69:0/2724645937 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:26:00.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.710+0000 7fe07dd6a640 1 -- 172.21.15.69:0/2724645937 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562e00808590 msgr2=0x562e00bdb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:26:00.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.710+0000 7fe07d569640 1 --2- 172.21.15.69:0/2724645937 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562e00808590 0x562e00bdb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.710+0000 7fe07e56b640 1 --2- 172.21.15.69:0/2724645937 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e00829270 0x562e00bc0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.710+0000 7fe07dd6a640 1 --2- 172.21.15.69:0/2724645937 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562e00808590 0x562e00bdb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.710+0000 7fe07dd6a640 1 -- 172.21.15.69:0/2724645937 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e00829270 msgr2=0x562e00bc0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:26:00.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.710+0000 7fe07dd6a640 1 --2- 172.21.15.69:0/2724645937 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e00829270 0x562e00bc0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.710+0000 7fe07dd6a640 1 -- 172.21.15.69:0/2724645937 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e00bcf0c0 con 0x562e008003b0 2024-04-25T11:26:00.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.710+0000 7fe07e56b640 1 --2- 172.21.15.69:0/2724645937 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e00829270 0x562e00bc0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:26:00.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.710+0000 7fe07d569640 1 --2- 172.21.15.69:0/2724645937 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562e00808590 0x562e00bdb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:26:00.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.710+0000 7fe07dd6a640 1 --2- 172.21.15.69:0/2724645937 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e008003b0 0x562e00bd2e70 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe074000c00 tx=0x7fe074001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bfbde513992b8e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:26:00.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.710+0000 7fe0637fe640 1 -- 172.21.15.69:0/2724645937 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe074035070 con 0x562e008003b0 2024-04-25T11:26:00.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.710+0000 7fe0637fe640 1 -- 172.21.15.69:0/2724645937 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe074041070 con 0x562e008003b0 2024-04-25T11:26:00.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.710+0000 7fe0637fe640 1 -- 172.21.15.69:0/2724645937 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07404b040 con 0x562e008003b0 2024-04-25T11:26:00.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.711+0000 7fe086e7d740 1 -- 172.21.15.69:0/2724645937 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e008003b0 msgr2=0x562e00bd2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:00.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.711+0000 7fe086e7d740 1 --2- 172.21.15.69:0/2724645937 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e008003b0 0x562e00bd2e70 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe074000c00 tx=0x7fe074001e70 comp rx=0 tx=0).stop 2024-04-25T11:26:00.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.711+0000 7fe086e7d740 1 -- 172.21.15.69:0/2724645937 shutdown_connections 2024-04-25T11:26:00.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.711+0000 7fe086e7d740 1 --2- 172.21.15.69:0/2724645937 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e00829270 0x562e00bc0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.711+0000 7fe086e7d740 1 --2- 172.21.15.69:0/2724645937 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562e00808590 0x562e00bdb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.711+0000 7fe086e7d740 1 --2- 172.21.15.69:0/2724645937 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e008003b0 0x562e00bd2e70 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.711+0000 7fe086e7d740 1 -- 172.21.15.69:0/2724645937 >> 172.21.15.69:0/2724645937 conn(0x562e0081d3c0 msgr2=0x562e00bd2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:26:00.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.711+0000 7fe086e7d740 1 -- 172.21.15.69:0/2724645937 shutdown_connections 2024-04-25T11:26:00.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.711+0000 7fe086e7d740 1 -- 172.21.15.69:0/2724645937 wait complete. 2024-04-25T11:26:00.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.711+0000 7fe086e7d740 1 -- start start 2024-04-25T11:26:00.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.711+0000 7fe086e7d740 1 Processor -- start 2024-04-25T11:26:00.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.712+0000 7fe086e7d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e008003b0 0x562e00bd8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.712+0000 7fe086e7d740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562e00808590 0x562e00bd94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.712+0000 7fe086e7d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e00829270 0x562e00bda3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.712+0000 7fe086e7d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562e00be1d70 con 0x562e00829270 2024-04-25T11:26:00.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.712+0000 7fe086e7d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x562e00be16a0 con 0x562e008003b0 2024-04-25T11:26:00.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.712+0000 7fe086e7d740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x562e00be2440 con 0x562e00808590 2024-04-25T11:26:00.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.712+0000 7fe086e7d740 10 client.0 _ll_register_callbacks cb 0x7fe074005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:26:00.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.712+0000 7fe07e56b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e00829270 0x562e00bda3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.712+0000 7fe07e56b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e00829270 0x562e00bda3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57338/0 (socket says 172.21.15.69:57338) 2024-04-25T11:26:00.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.712+0000 7fe07e56b640 1 -- 172.21.15.69:0/3779986495 learned_addr learned my addr 172.21.15.69:0/3779986495 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:26:00.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.712+0000 7fe07dd6a640 1 --2- 172.21.15.69:0/3779986495 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e008003b0 0x562e00bd8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.712+0000 7fe07d569640 1 --2- 172.21.15.69:0/3779986495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562e00808590 0x562e00bd94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.712+0000 7fe07e56b640 1 -- 172.21.15.69:0/3779986495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562e00808590 msgr2=0x562e00bd94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:26:00.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.712+0000 7fe07e56b640 1 --2- 172.21.15.69:0/3779986495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562e00808590 0x562e00bd94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.712+0000 7fe07e56b640 1 -- 172.21.15.69:0/3779986495 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e008003b0 msgr2=0x562e00bd8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:26:00.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.712+0000 7fe07e56b640 1 --2- 172.21.15.69:0/3779986495 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e008003b0 0x562e00bd8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.712+0000 7fe07e56b640 1 -- 172.21.15.69:0/3779986495 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e00bda9a0 con 0x562e00829270 2024-04-25T11:26:00.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.712+0000 7fe07d569640 1 --2- 172.21.15.69:0/3779986495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562e00808590 0x562e00bd94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:26:00.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.712+0000 7fe07e56b640 1 --2- 172.21.15.69:0/3779986495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e00829270 0x562e00bda3e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe07001e000 tx=0x7fe070024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe98e8bdd78a0b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:26:00.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.712+0000 7fe0617fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-25T11:26:00.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.713+0000 7fe0617fa640 1 -- 172.21.15.69:0/3779986495 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe070037060 con 0x562e00829270 2024-04-25T11:26:00.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.713+0000 7fe0617fa640 1 -- 172.21.15.69:0/3779986495 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7fe0700049e0 con 0x562e00829270 2024-04-25T11:26:00.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.713+0000 7fe086e7d740 1 -- 172.21.15.69:0/3779986495 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe074041070 con 0x562e00829270 2024-04-25T11:26:00.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.713+0000 7fe0617fa640 1 -- 172.21.15.69:0/3779986495 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe07003e050 con 0x562e00829270 2024-04-25T11:26:00.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.713+0000 7fe086e7d740 10 client.15486 fetch_fsmap learned FSMap version 28 2024-04-25T11:26:00.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.713+0000 7fe086e7d740 1 -- 172.21.15.69:0/3779986495 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562e00c4ad70 con 0x562e00829270 2024-04-25T11:26:00.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.713+0000 7fe0617fa640 1 -- 172.21.15.69:0/3779986495 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe074041070 con 0x562e00829270 2024-04-25T11:26:00.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.713+0000 7fe0617fa640 1 -- 172.21.15.69:0/3779986495 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe070031070 con 0x562e00829270 2024-04-25T11:26:00.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.713+0000 7fe086e7d740 10 client.15486 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T11:26:00.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.713+0000 7fe086e7d740 10 client.15486 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T11:26:00.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.713+0000 7fe086e7d740 20 client.15486 populate_metadata read hostname 'smithi069' 2024-04-25T11:26:00.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.713+0000 7fe086e7d740 1 --2- 172.21.15.69:0/3779986495 >> [v2:172.21.15.146:6836/2730317134,v1:172.21.15.146:6837/2730317134] conn(0x562e007bd280 0x562e00c4e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.713+0000 7fe086e7d740 4 client.15486 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:26:00.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.713+0000 7fe086e7d740 1 -- 172.21.15.69:0/3779986495 --> [v2:172.21.15.146:6836/2730317134,v1:172.21.15.146:6837/2730317134] -- command(tid 0: {"prefix": "status"}) -- 0x562e00b371c0 con 0x562e007bd280 2024-04-25T11:26:00.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.714+0000 7fe07dd6a640 1 --2- 172.21.15.69:0/3779986495 >> [v2:172.21.15.146:6836/2730317134,v1:172.21.15.146:6837/2730317134] conn(0x562e007bd280 0x562e00c4e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.714+0000 7fe0617fa640 10 client.15486 ms_handle_connect on v2:172.21.15.146:6836/2730317134 2024-04-25T11:26:00.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.714+0000 7fe07dd6a640 1 --2- 172.21.15.69:0/3779986495 >> [v2:172.21.15.146:6836/2730317134,v1:172.21.15.146:6837/2730317134] conn(0x562e007bd280 0x562e00c4e340 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6129406831e381f8 server_cookie=2ff8cb06adcfb7e2 in_seq=0 out_seq=0 2024-04-25T11:26:00.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.714+0000 7fe0617fa640 1 -- 172.21.15.69:0/3779986495 <== mds.1 v2:172.21.15.146:6836/2730317134 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x562e00b371c0 con 0x562e007bd280 2024-04-25T11:26:00.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.714+0000 7fe0617fa640 10 client.15486 handle_command_reply: tid=0 2024-04-25T11:26:00.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.714+0000 7fe086e7d740 1 client.15486 shutdown 2024-04-25T11:26:00.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.714+0000 7fe086e7d740 1 -- 172.21.15.69:0/3779986495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e00829270 msgr2=0x562e00bda3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:00.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.714+0000 7fe086e7d740 1 --2- 172.21.15.69:0/3779986495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e00829270 0x562e00bda3e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe07001e000 tx=0x7fe070024000 comp rx=0 tx=0).stop 2024-04-25T11:26:00.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.714+0000 7fe086e7d740 1 -- 172.21.15.69:0/3779986495 shutdown_connections 2024-04-25T11:26:00.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.714+0000 7fe086e7d740 1 --2- 172.21.15.69:0/3779986495 >> [v2:172.21.15.146:6836/2730317134,v1:172.21.15.146:6837/2730317134] conn(0x562e007bd280 0x562e00c4e340 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.714+0000 7fe086e7d740 1 --2- 172.21.15.69:0/3779986495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e00829270 0x562e00bda3e0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.714+0000 7fe0617fa640 0 client.15486 ms_handle_reset on v2:172.21.15.146:6836/2730317134 2024-04-25T11:26:00.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.714+0000 7fe086e7d740 1 --2- 172.21.15.69:0/3779986495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562e00808590 0x562e00bd94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.714+0000 7fe086e7d740 1 --2- 172.21.15.69:0/3779986495 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562e008003b0 0x562e00bd8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.714+0000 7fe086e7d740 1 -- 172.21.15.69:0/3779986495 >> 172.21.15.69:0/3779986495 conn(0x562e0081d3c0 msgr2=0x562e00bc8ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:26:00.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.715+0000 7fe086e7d740 1 -- 172.21.15.69:0/3779986495 shutdown_connections 2024-04-25T11:26:00.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.715+0000 7fe086e7d740 1 -- 172.21.15.69:0/3779986495 wait complete. 2024-04-25T11:26:00.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.715+0000 7fe086e7d740 20 client.15486 trim_cache size 0 max 16384 2024-04-25T11:26:00.715 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-25T11:26:00.715 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "f7ed73dc-02f4-11ef-bc93-c7b262605968", 2024-04-25T11:26:00.715 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 1, 2024-04-25T11:26:00.715 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 24485, 2024-04-25T11:26:00.715 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-04-25T11:26:00.715 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-04-25T11:26:00.715 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-04-25T11:26:00.715 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 114.831437261, 2024-04-25T11:26:00.715 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 28, 2024-04-25T11:26:00.715 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 84, 2024-04-25T11:26:00.715 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 80, 2024-04-25T11:26:00.715 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 157.56648245 2024-04-25T11:26:00.716 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-25T11:26:00.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.715+0000 7fe07ffce640 1 -- 172.21.15.69:0/1201228869 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fe0640afb70 msgr2=0x7fe0640b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:00.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.715+0000 7fe07ffce640 1 --2- 172.21.15.69:0/1201228869 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fe0640afb70 0x7fe0640b2080 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe070011010 tx=0x7fe070021000 comp rx=0 tx=0).stop 2024-04-25T11:26:00.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.715+0000 7fe07ffce640 1 -- 172.21.15.69:0/1201228869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe078103c20 msgr2=0x7fe07819a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:00.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.715+0000 7fe07ffce640 1 --2- 172.21.15.69:0/1201228869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe078103c20 0x7fe07819a9a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe06800d950 tx=0x7fe06800de10 comp rx=0 tx=0).stop 2024-04-25T11:26:00.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.715+0000 7fe07ffce640 1 -- 172.21.15.69:0/1201228869 shutdown_connections 2024-04-25T11:26:00.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.715+0000 7fe07ffce640 1 --2- 172.21.15.69:0/1201228869 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fe0640afb70 0x7fe0640b2080 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.715+0000 7fe07ffce640 1 --2- 172.21.15.69:0/1201228869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe078108920 0x7fe0781a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.715+0000 7fe07ffce640 1 --2- 172.21.15.69:0/1201228869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe078103c20 0x7fe07819a9a0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.715+0000 7fe07ffce640 1 --2- 172.21.15.69:0/1201228869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe0781029e0 0x7fe07819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.715+0000 7fe07ffce640 1 -- 172.21.15.69:0/1201228869 >> 172.21.15.69:0/1201228869 conn(0x7fe0780fdbd0 msgr2=0x7fe078109ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:26:00.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.715+0000 7fe07ffce640 1 -- 172.21.15.69:0/1201228869 shutdown_connections 2024-04-25T11:26:00.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.715+0000 7fe07ffce640 1 -- 172.21.15.69:0/1201228869 wait complete. 2024-04-25T11:26:00.719 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T11:26:00.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.835+0000 7f289d2f2640 1 -- 172.21.15.69:0/1480744339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2898108920 msgr2=0x7f289810ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:00.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.835+0000 7f289d2f2640 1 --2- 172.21.15.69:0/1480744339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2898108920 0x7f289810ad40 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2890009ca0 tx=0x7f289002a8c0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.835+0000 7f289d2f2640 1 -- 172.21.15.69:0/1480744339 shutdown_connections 2024-04-25T11:26:00.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.835+0000 7f289d2f2640 1 --2- 172.21.15.69:0/1480744339 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2898108920 0x7f289810ad40 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.835+0000 7f289d2f2640 1 --2- 172.21.15.69:0/1480744339 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2898103c20 0x7f2898104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.835+0000 7f289d2f2640 1 --2- 172.21.15.69:0/1480744339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f28981029e0 0x7f28981036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.835+0000 7f289d2f2640 1 -- 172.21.15.69:0/1480744339 >> 172.21.15.69:0/1480744339 conn(0x7f28980fdbd0 msgr2=0x7f2898100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:26:00.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.835+0000 7f289d2f2640 1 -- 172.21.15.69:0/1480744339 shutdown_connections 2024-04-25T11:26:00.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.835+0000 7f289d2f2640 1 -- 172.21.15.69:0/1480744339 wait complete. 2024-04-25T11:26:00.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.836+0000 7f289d2f2640 1 Processor -- start 2024-04-25T11:26:00.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.836+0000 7f289d2f2640 1 -- start start 2024-04-25T11:26:00.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.836+0000 7f289d2f2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28981029e0 0x7f2898198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.836+0000 7f289d2f2640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2898103c20 0x7f2898199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.836+0000 7f289d2f2640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2898108920 0x7f28981a04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.836+0000 7f289d2f2640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f289810e410 con 0x7f28981029e0 2024-04-25T11:26:00.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.836+0000 7f289d2f2640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f289810e290 con 0x7f2898103c20 2024-04-25T11:26:00.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.836+0000 7f289d2f2640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f289810e590 con 0x7f2898108920 2024-04-25T11:26:00.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.836+0000 7f2896ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28981029e0 0x7f2898198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.836+0000 7f2896ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28981029e0 0x7f2898198fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57356/0 (socket says 172.21.15.69:57356) 2024-04-25T11:26:00.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.836+0000 7f2896ffd640 1 -- 172.21.15.69:0/1771533510 learned_addr learned my addr 172.21.15.69:0/1771533510 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:26:00.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.836+0000 7f28967fc640 1 --2- 172.21.15.69:0/1771533510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2898103c20 0x7f2898199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.836+0000 7f28977fe640 1 --2- 172.21.15.69:0/1771533510 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2898108920 0x7f28981a04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.836+0000 7f2896ffd640 1 -- 172.21.15.69:0/1771533510 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2898108920 msgr2=0x7f28981a04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:00.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.836+0000 7f2896ffd640 1 --2- 172.21.15.69:0/1771533510 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2898108920 0x7f28981a04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.836+0000 7f2896ffd640 1 -- 172.21.15.69:0/1771533510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2898103c20 msgr2=0x7f2898199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:00.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.836+0000 7f2896ffd640 1 --2- 172.21.15.69:0/1771533510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2898103c20 0x7f2898199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.836+0000 7f2896ffd640 1 -- 172.21.15.69:0/1771533510 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28981a0a60 con 0x7f28981029e0 2024-04-25T11:26:00.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.836+0000 7f28967fc640 1 --2- 172.21.15.69:0/1771533510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2898103c20 0x7f2898199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:26:00.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.836+0000 7f28977fe640 1 --2- 172.21.15.69:0/1771533510 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2898108920 0x7f28981a04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:26:00.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.837+0000 7f2896ffd640 1 --2- 172.21.15.69:0/1771533510 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28981029e0 0x7f2898198fd0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f288c005d00 tx=0x7f288c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:26:00.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.837+0000 7f287ffff640 1 -- 172.21.15.69:0/1771533510 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288c017070 con 0x7f28981029e0 2024-04-25T11:26:00.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.837+0000 7f287ffff640 1 -- 172.21.15.69:0/1771533510 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f288c009c30 con 0x7f28981029e0 2024-04-25T11:26:00.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.837+0000 7f289d2f2640 1 -- 172.21.15.69:0/1771533510 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28981a0d50 con 0x7f28981029e0 2024-04-25T11:26:00.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.837+0000 7f287ffff640 1 -- 172.21.15.69:0/1771533510 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288c00f620 con 0x7f28981029e0 2024-04-25T11:26:00.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.837+0000 7f289d2f2640 1 -- 172.21.15.69:0/1771533510 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28981a1190 con 0x7f28981029e0 2024-04-25T11:26:00.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.839+0000 7f28a41a2740 1 Processor -- start 2024-04-25T11:26:00.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.839+0000 7f287ffff640 1 -- 172.21.15.69:0/1771533510 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f288c01a070 con 0x7f28981029e0 2024-04-25T11:26:00.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.839+0000 7f28a41a2740 1 -- start start 2024-04-25T11:26:00.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.839+0000 7f28a41a2740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f88c8e5270 0x55f88cc86370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.839+0000 7f28a41a2740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f88c8bc3b0 0x55f88cc868f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.839+0000 7f28a41a2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f88c8c4590 0x55f88cc8d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.839+0000 7f28a41a2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f88cbeb740 con 0x55f88c8c4590 2024-04-25T11:26:00.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.839+0000 7f28a41a2740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55f88cb873a0 con 0x55f88c8e5270 2024-04-25T11:26:00.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.839+0000 7f28a41a2740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55f88cbe7700 con 0x55f88c8bc3b0 2024-04-25T11:26:00.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.840+0000 7f28967fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f88c8c4590 0x55f88cc8d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.840+0000 7f28967fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f88c8c4590 0x55f88cc8d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57372/0 (socket says 172.21.15.69:57372) 2024-04-25T11:26:00.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.840+0000 7f2896ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f88c8bc3b0 0x55f88cc868f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.840+0000 7f28967fc640 1 -- 172.21.15.69:0/3914244893 learned_addr learned my addr 172.21.15.69:0/3914244893 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:26:00.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.840+0000 7f28977fe640 1 --2- 172.21.15.69:0/3914244893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f88c8e5270 0x55f88cc86370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.840+0000 7f287ffff640 1 --2- 172.21.15.69:0/1771533510 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f28800afaf0 0x7f28800b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.840+0000 7f28967fc640 1 -- 172.21.15.69:0/3914244893 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f88c8bc3b0 msgr2=0x55f88cc868f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:26:00.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.840+0000 7f28967fc640 1 --2- 172.21.15.69:0/3914244893 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f88c8bc3b0 0x55f88cc868f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.840+0000 7f28967fc640 1 -- 172.21.15.69:0/3914244893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f88c8e5270 msgr2=0x55f88cc86370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:26:00.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.840+0000 7f28967fc640 1 --2- 172.21.15.69:0/3914244893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f88c8e5270 0x55f88cc86370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.840+0000 7f28967fc640 1 -- 172.21.15.69:0/3914244893 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f88cba3750 con 0x55f88c8c4590 2024-04-25T11:26:00.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.840+0000 7f28977fe640 1 --2- 172.21.15.69:0/1771533510 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f28800afaf0 0x7f28800b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.840+0000 7f28967fc640 1 --2- 172.21.15.69:0/3914244893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f88c8c4590 0x55f88cc8d7f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f288800bc10 tx=0x7f288800d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=994e8b7e02183538 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:26:00.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.840+0000 7f287ffff640 1 -- 172.21.15.69:0/1771533510 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f288c0deab0 con 0x7f28981029e0 2024-04-25T11:26:00.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.840+0000 7f287d7fa640 1 -- 172.21.15.69:0/3914244893 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2888005220 con 0x55f88c8c4590 2024-04-25T11:26:00.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.840+0000 7f287d7fa640 1 -- 172.21.15.69:0/3914244893 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2888004c30 con 0x55f88c8c4590 2024-04-25T11:26:00.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.840+0000 7f287d7fa640 1 -- 172.21.15.69:0/3914244893 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288800f7a0 con 0x55f88c8c4590 2024-04-25T11:26:00.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.841+0000 7f28a41a2740 1 -- 172.21.15.69:0/3914244893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f88c8c4590 msgr2=0x55f88cc8d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:00.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.841+0000 7f28a41a2740 1 --2- 172.21.15.69:0/3914244893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f88c8c4590 0x55f88cc8d7f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f288800bc10 tx=0x7f288800d710 comp rx=0 tx=0).stop 2024-04-25T11:26:00.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.841+0000 7f28977fe640 1 --2- 172.21.15.69:0/1771533510 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f28800afaf0 0x7f28800b2000 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2890006010 tx=0x7f28900095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:26:00.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.841+0000 7f28a41a2740 1 -- 172.21.15.69:0/3914244893 shutdown_connections 2024-04-25T11:26:00.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.841+0000 7f28a41a2740 1 --2- 172.21.15.69:0/3914244893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f88c8c4590 0x55f88cc8d7f0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.841+0000 7f28a41a2740 1 --2- 172.21.15.69:0/3914244893 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f88c8bc3b0 0x55f88cc868f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.841+0000 7f28a41a2740 1 --2- 172.21.15.69:0/3914244893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f88c8e5270 0x55f88cc86370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.841+0000 7f28a41a2740 1 -- 172.21.15.69:0/3914244893 >> 172.21.15.69:0/3914244893 conn(0x55f88c8d93c0 msgr2=0x55f88cc81d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:26:00.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.841+0000 7f28a41a2740 1 -- 172.21.15.69:0/3914244893 shutdown_connections 2024-04-25T11:26:00.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.841+0000 7f28a41a2740 1 -- 172.21.15.69:0/3914244893 wait complete. 2024-04-25T11:26:00.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.842+0000 7f28a41a2740 1 -- start start 2024-04-25T11:26:00.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.842+0000 7f28a41a2740 1 Processor -- start 2024-04-25T11:26:00.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.842+0000 7f28a41a2740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f88c8bc3b0 0x55f88cc90260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.842+0000 7f28a41a2740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f88c8c4590 0x55f88cc907a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.842+0000 7f28a41a2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f88c8e5270 0x55f88cc97700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.842+0000 7f28a41a2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f88cc8e240 con 0x55f88c8e5270 2024-04-25T11:26:00.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.842+0000 7f28a41a2740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55f88cb72fe0 con 0x55f88c8bc3b0 2024-04-25T11:26:00.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.842+0000 7f28a41a2740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55f88cc8e3c0 con 0x55f88c8c4590 2024-04-25T11:26:00.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.842+0000 7f28a41a2740 10 client.0 _ll_register_callbacks cb 0x55f88cb89150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:26:00.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.842+0000 7f28977fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f88c8e5270 0x55f88cc97700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.842+0000 7f28977fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f88c8e5270 0x55f88cc97700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57382/0 (socket says 172.21.15.69:57382) 2024-04-25T11:26:00.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.842+0000 7f28977fe640 1 -- 172.21.15.69:0/507431287 learned_addr learned my addr 172.21.15.69:0/507431287 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:26:00.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.842+0000 7f28967fc640 1 --2- 172.21.15.69:0/507431287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f88c8c4590 0x55f88cc907a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.842+0000 7f2896ffd640 1 --2- 172.21.15.69:0/507431287 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f88c8bc3b0 0x55f88cc90260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.842+0000 7f28977fe640 1 -- 172.21.15.69:0/507431287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f88c8c4590 msgr2=0x55f88cc907a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:26:00.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.842+0000 7f28977fe640 1 --2- 172.21.15.69:0/507431287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f88c8c4590 0x55f88cc907a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.842+0000 7f28977fe640 1 -- 172.21.15.69:0/507431287 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f88c8bc3b0 msgr2=0x55f88cc90260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:26:00.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.842+0000 7f28977fe640 1 --2- 172.21.15.69:0/507431287 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f88c8bc3b0 0x55f88cc90260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.842+0000 7f28977fe640 1 -- 172.21.15.69:0/507431287 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f88c8fd2d0 con 0x55f88c8e5270 2024-04-25T11:26:00.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.842+0000 7f28967fc640 1 --2- 172.21.15.69:0/507431287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f88c8c4590 0x55f88cc907a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:26:00.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.842+0000 7f28977fe640 1 --2- 172.21.15.69:0/507431287 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f88c8e5270 0x55f88cc97700 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2890035000 tx=0x7f289002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bac0d090a69cad3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:26:00.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.842+0000 7f287aff5640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-25T11:26:00.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.843+0000 7f287aff5640 1 -- 172.21.15.69:0/507431287 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28900047c0 con 0x55f88c8e5270 2024-04-25T11:26:00.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.843+0000 7f287aff5640 1 -- 172.21.15.69:0/507431287 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f28900305c0 con 0x55f88c8e5270 2024-04-25T11:26:00.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.843+0000 7f28a41a2740 1 -- 172.21.15.69:0/507431287 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2888004c30 con 0x55f88c8e5270 2024-04-25T11:26:00.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.843+0000 7f287aff5640 1 -- 172.21.15.69:0/507431287 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f28900307c0 con 0x55f88c8e5270 2024-04-25T11:26:00.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.843+0000 7f28a41a2740 10 client.15498 fetch_fsmap learned FSMap version 28 2024-04-25T11:26:00.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.843+0000 7f28a41a2740 1 -- 172.21.15.69:0/507431287 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f88c9c76b0 con 0x55f88c8e5270 2024-04-25T11:26:00.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.843+0000 7f287aff5640 1 -- 172.21.15.69:0/507431287 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2888004c30 con 0x55f88c8e5270 2024-04-25T11:26:00.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.843+0000 7f287aff5640 1 -- 172.21.15.69:0/507431287 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f28900477a0 con 0x55f88c8e5270 2024-04-25T11:26:00.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.843+0000 7f28a41a2740 10 client.15498 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T11:26:00.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.843+0000 7f28a41a2740 10 client.15498 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T11:26:00.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.843+0000 7f28a41a2740 20 client.15498 populate_metadata read hostname 'smithi069' 2024-04-25T11:26:00.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.843+0000 7f28a41a2740 1 --2- 172.21.15.69:0/507431287 >> [v2:172.21.15.146:6838/2013283448,v1:172.21.15.146:6839/2013283448] conn(0x55f88c8a3e40 0x55f88cc9e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.843+0000 7f28a41a2740 4 client.15498 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:26:00.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.843+0000 7f28a41a2740 1 -- 172.21.15.69:0/507431287 --> [v2:172.21.15.146:6838/2013283448,v1:172.21.15.146:6839/2013283448] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f88cbf9350 con 0x55f88c8a3e40 2024-04-25T11:26:00.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.844+0000 7f2896ffd640 1 --2- 172.21.15.69:0/507431287 >> [v2:172.21.15.146:6838/2013283448,v1:172.21.15.146:6839/2013283448] conn(0x55f88c8a3e40 0x55f88cc9e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.844+0000 7f287aff5640 10 client.15498 ms_handle_connect on v2:172.21.15.146:6838/2013283448 2024-04-25T11:26:00.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.844+0000 7f2896ffd640 1 --2- 172.21.15.69:0/507431287 >> [v2:172.21.15.146:6838/2013283448,v1:172.21.15.146:6839/2013283448] conn(0x55f88c8a3e40 0x55f88cc9e020 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77ac3da93b59527f server_cookie=b79fd02adbbaf6f9 in_seq=0 out_seq=0 2024-04-25T11:26:00.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.845+0000 7f287aff5640 1 -- 172.21.15.69:0/507431287 <== mds.0 v2:172.21.15.146:6838/2013283448 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f88cbf9350 con 0x55f88c8a3e40 2024-04-25T11:26:00.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.845+0000 7f287aff5640 10 client.15498 handle_command_reply: tid=0 2024-04-25T11:26:00.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.845+0000 7f28a41a2740 1 client.15498 shutdown 2024-04-25T11:26:00.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.845+0000 7f28a41a2740 1 -- 172.21.15.69:0/507431287 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f88c8e5270 msgr2=0x55f88cc97700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:00.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.845+0000 7f28a41a2740 1 --2- 172.21.15.69:0/507431287 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f88c8e5270 0x55f88cc97700 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2890035000 tx=0x7f289002a8c0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.845+0000 7f28a41a2740 1 -- 172.21.15.69:0/507431287 shutdown_connections 2024-04-25T11:26:00.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.845+0000 7f28a41a2740 1 --2- 172.21.15.69:0/507431287 >> [v2:172.21.15.146:6838/2013283448,v1:172.21.15.146:6839/2013283448] conn(0x55f88c8a3e40 0x55f88cc9e020 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.845+0000 7f28a41a2740 1 --2- 172.21.15.69:0/507431287 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f88c8e5270 0x55f88cc97700 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.845+0000 7f28a41a2740 1 --2- 172.21.15.69:0/507431287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f88c8c4590 0x55f88cc907a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.845+0000 7f28a41a2740 1 --2- 172.21.15.69:0/507431287 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f88c8bc3b0 0x55f88cc90260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.845+0000 7f287aff5640 0 client.15498 ms_handle_reset on v2:172.21.15.146:6838/2013283448 2024-04-25T11:26:00.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.845+0000 7f28a41a2740 1 -- 172.21.15.69:0/507431287 >> 172.21.15.69:0/507431287 conn(0x55f88c8d93c0 msgr2=0x55f88cc80d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:26:00.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.845+0000 7f28a41a2740 1 -- 172.21.15.69:0/507431287 shutdown_connections 2024-04-25T11:26:00.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.845+0000 7f28a41a2740 1 -- 172.21.15.69:0/507431287 wait complete. 2024-04-25T11:26:00.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.845+0000 7f28a41a2740 20 client.15498 trim_cache size 0 max 16384 2024-04-25T11:26:00.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.863+0000 7f28a41a2740 1 Processor -- start 2024-04-25T11:26:00.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.863+0000 7f28a41a2740 1 -- start start 2024-04-25T11:26:00.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.863+0000 7f28a41a2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f88c8bc3b0 0x55f88cc8a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.863+0000 7f28a41a2740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f88c8c4590 0x55f88cc97620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.863+0000 7f28a41a2740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f88c8e5270 0x55f88cc8abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.863+0000 7f28a41a2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f88cc9c360 con 0x55f88c8bc3b0 2024-04-25T11:26:00.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.863+0000 7f28a41a2740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55f88cc9ca30 con 0x55f88c8c4590 2024-04-25T11:26:00.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.863+0000 7f28a41a2740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55f88cc9d100 con 0x55f88c8e5270 2024-04-25T11:26:00.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.863+0000 7f2896ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f88c8bc3b0 0x55f88cc8a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.863+0000 7f2896ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f88c8bc3b0 0x55f88cc8a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57396/0 (socket says 172.21.15.69:57396) 2024-04-25T11:26:00.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.863+0000 7f2896ffd640 1 -- 172.21.15.69:0/703190869 learned_addr learned my addr 172.21.15.69:0/703190869 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:26:00.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.864+0000 7f28977fe640 1 --2- 172.21.15.69:0/703190869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f88c8e5270 0x55f88cc8abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.864+0000 7f2896ffd640 1 -- 172.21.15.69:0/703190869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f88c8e5270 msgr2=0x55f88cc8abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:26:00.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.864+0000 7f2896ffd640 1 --2- 172.21.15.69:0/703190869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f88c8e5270 0x55f88cc8abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.864+0000 7f2896ffd640 1 -- 172.21.15.69:0/703190869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f88c8c4590 msgr2=0x55f88cc97620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:26:00.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.864+0000 7f28967fc640 1 --2- 172.21.15.69:0/703190869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f88c8c4590 0x55f88cc97620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.864+0000 7f2896ffd640 1 --2- 172.21.15.69:0/703190869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f88c8c4590 0x55f88cc97620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.864+0000 7f2896ffd640 1 -- 172.21.15.69:0/703190869 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f88cc8e530 con 0x55f88c8bc3b0 2024-04-25T11:26:00.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.864+0000 7f28977fe640 1 --2- 172.21.15.69:0/703190869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f88c8e5270 0x55f88cc8abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:26:00.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.864+0000 7f2896ffd640 1 --2- 172.21.15.69:0/703190869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f88c8bc3b0 0x55f88cc8a6b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f288c0a6b50 tx=0x7f288c0c9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8750bc786dca7aad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:26:00.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.864+0000 7f287cff9640 1 -- 172.21.15.69:0/703190869 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288c021040 con 0x55f88c8bc3b0 2024-04-25T11:26:00.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.864+0000 7f287cff9640 1 -- 172.21.15.69:0/703190869 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f288c010e40 con 0x55f88c8bc3b0 2024-04-25T11:26:00.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.864+0000 7f287cff9640 1 -- 172.21.15.69:0/703190869 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288c0e8040 con 0x55f88c8bc3b0 2024-04-25T11:26:00.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.865+0000 7f28a41a2740 1 -- 172.21.15.69:0/703190869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f88c8bc3b0 msgr2=0x55f88cc8a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:00.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.865+0000 7f28a41a2740 1 --2- 172.21.15.69:0/703190869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f88c8bc3b0 0x55f88cc8a6b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f288c0a6b50 tx=0x7f288c0c9040 comp rx=0 tx=0).stop 2024-04-25T11:26:00.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.865+0000 7f28a41a2740 1 -- 172.21.15.69:0/703190869 shutdown_connections 2024-04-25T11:26:00.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.865+0000 7f28a41a2740 1 --2- 172.21.15.69:0/703190869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f88c8e5270 0x55f88cc8abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.865+0000 7f28a41a2740 1 --2- 172.21.15.69:0/703190869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f88c8c4590 0x55f88cc97620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.865+0000 7f28a41a2740 1 --2- 172.21.15.69:0/703190869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f88c8bc3b0 0x55f88cc8a6b0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.865+0000 7f28a41a2740 1 -- 172.21.15.69:0/703190869 >> 172.21.15.69:0/703190869 conn(0x55f88c8d93c0 msgr2=0x55f88cc8e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:26:00.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.865+0000 7f28a41a2740 1 -- 172.21.15.69:0/703190869 shutdown_connections 2024-04-25T11:26:00.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.865+0000 7f28a41a2740 1 -- 172.21.15.69:0/703190869 wait complete. 2024-04-25T11:26:00.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.865+0000 7f28a41a2740 1 -- start start 2024-04-25T11:26:00.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.865+0000 7f28a41a2740 1 Processor -- start 2024-04-25T11:26:00.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.865+0000 7f28a41a2740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f88c8c4590 0x55f88cc93320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.866+0000 7f28a41a2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f88c8e5270 0x55f88cc93860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.866+0000 7f28a41a2740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f88c879280 0x55f88cc96700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.866+0000 7f28a41a2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f88cc9dea0 con 0x55f88c8e5270 2024-04-25T11:26:00.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.866+0000 7f28a41a2740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55f88cc9d7d0 con 0x55f88c879280 2024-04-25T11:26:00.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.866+0000 7f28a41a2740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55f88cc9e570 con 0x55f88c8c4590 2024-04-25T11:26:00.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.866+0000 7f28a41a2740 10 client.0 _ll_register_callbacks cb 0x7f28900317c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:26:00.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.866+0000 7f28967fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f88c8e5270 0x55f88cc93860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.866+0000 7f28967fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f88c8e5270 0x55f88cc93860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57400/0 (socket says 172.21.15.69:57400) 2024-04-25T11:26:00.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.866+0000 7f28967fc640 1 -- 172.21.15.69:0/3403886384 learned_addr learned my addr 172.21.15.69:0/3403886384 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:26:00.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.866+0000 7f2896ffd640 1 --2- 172.21.15.69:0/3403886384 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f88c8c4590 0x55f88cc93320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.866+0000 7f28977fe640 1 --2- 172.21.15.69:0/3403886384 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f88c879280 0x55f88cc96700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.866+0000 7f28967fc640 1 -- 172.21.15.69:0/3403886384 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f88c8c4590 msgr2=0x55f88cc93320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:26:00.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.866+0000 7f28967fc640 1 --2- 172.21.15.69:0/3403886384 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f88c8c4590 0x55f88cc93320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.866+0000 7f28967fc640 1 -- 172.21.15.69:0/3403886384 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f88c879280 msgr2=0x55f88cc96700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:26:00.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.866+0000 7f28967fc640 1 --2- 172.21.15.69:0/3403886384 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f88c879280 0x55f88cc96700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.866+0000 7f28967fc640 1 -- 172.21.15.69:0/3403886384 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f88cc96cc0 con 0x55f88c8e5270 2024-04-25T11:26:00.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.866+0000 7f28977fe640 1 --2- 172.21.15.69:0/3403886384 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f88c879280 0x55f88cc96700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:26:00.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.866+0000 7f28967fc640 1 --2- 172.21.15.69:0/3403886384 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f88c8e5270 0x55f88cc93860 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f288800bc10 tx=0x7f2888005490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c3a86107bc2058 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:26:00.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.866+0000 7f287aff5640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-04-25T11:26:00.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.866+0000 7f287aff5640 1 -- 172.21.15.69:0/3403886384 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2888026020 con 0x55f88c8e5270 2024-04-25T11:26:00.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.866+0000 7f287aff5640 1 -- 172.21.15.69:0/3403886384 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f288800b720 con 0x55f88c8e5270 2024-04-25T11:26:00.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.866+0000 7f28a41a2740 1 -- 172.21.15.69:0/3403886384 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f88cbf31c0 con 0x55f88c8e5270 2024-04-25T11:26:00.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.866+0000 7f287aff5640 1 -- 172.21.15.69:0/3403886384 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2888031050 con 0x55f88c8e5270 2024-04-25T11:26:00.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.866+0000 7f28a41a2740 10 client.15504 fetch_fsmap learned FSMap version 28 2024-04-25T11:26:00.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.866+0000 7f28a41a2740 1 -- 172.21.15.69:0/3403886384 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f88cd07210 con 0x55f88c8e5270 2024-04-25T11:26:00.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.867+0000 7f287aff5640 1 -- 172.21.15.69:0/3403886384 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55f88cbf31c0 con 0x55f88c8e5270 2024-04-25T11:26:00.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.867+0000 7f287aff5640 1 -- 172.21.15.69:0/3403886384 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f28880224a0 con 0x55f88c8e5270 2024-04-25T11:26:00.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.867+0000 7f28a41a2740 10 client.15504 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T11:26:00.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.867+0000 7f28a41a2740 10 client.15504 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T11:26:00.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.867+0000 7f28a41a2740 20 client.15504 populate_metadata read hostname 'smithi069' 2024-04-25T11:26:00.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.867+0000 7f28a41a2740 1 --2- 172.21.15.69:0/3403886384 >> [v2:172.21.15.146:6838/2013283448,v1:172.21.15.146:6839/2013283448] conn(0x55f88cd07f90 0x55f88cd085e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:00.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.867+0000 7f28a41a2740 4 client.15504 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:26:00.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.867+0000 7f28a41a2740 1 -- 172.21.15.69:0/3403886384 --> [v2:172.21.15.146:6838/2013283448,v1:172.21.15.146:6839/2013283448] -- command(tid 0: {"prefix": "status"}) -- 0x55f88cb6e9a0 con 0x55f88cd07f90 2024-04-25T11:26:00.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.867+0000 7f2896ffd640 1 --2- 172.21.15.69:0/3403886384 >> [v2:172.21.15.146:6838/2013283448,v1:172.21.15.146:6839/2013283448] conn(0x55f88cd07f90 0x55f88cd085e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:00.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.867+0000 7f287aff5640 10 client.15504 ms_handle_connect on v2:172.21.15.146:6838/2013283448 2024-04-25T11:26:00.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.867+0000 7f2896ffd640 1 --2- 172.21.15.69:0/3403886384 >> [v2:172.21.15.146:6838/2013283448,v1:172.21.15.146:6839/2013283448] conn(0x55f88cd07f90 0x55f88cd085e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7783bea9f6a7ddfa server_cookie=82af6668881d654d in_seq=0 out_seq=0 2024-04-25T11:26:00.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.868+0000 7f287aff5640 1 -- 172.21.15.69:0/3403886384 <== mds.0 v2:172.21.15.146:6838/2013283448 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f88cb6e9a0 con 0x55f88cd07f90 2024-04-25T11:26:00.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.868+0000 7f287aff5640 10 client.15504 handle_command_reply: tid=0 2024-04-25T11:26:00.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.868+0000 7f28a41a2740 1 client.15504 shutdown 2024-04-25T11:26:00.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.868+0000 7f28a41a2740 1 -- 172.21.15.69:0/3403886384 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f88c8e5270 msgr2=0x55f88cc93860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:00.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.868+0000 7f28a41a2740 1 --2- 172.21.15.69:0/3403886384 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f88c8e5270 0x55f88cc93860 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f288800bc10 tx=0x7f2888005490 comp rx=0 tx=0).stop 2024-04-25T11:26:00.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.868+0000 7f28a41a2740 1 -- 172.21.15.69:0/3403886384 shutdown_connections 2024-04-25T11:26:00.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.868+0000 7f28a41a2740 1 --2- 172.21.15.69:0/3403886384 >> [v2:172.21.15.146:6838/2013283448,v1:172.21.15.146:6839/2013283448] conn(0x55f88cd07f90 0x55f88cd085e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.868+0000 7f28a41a2740 1 --2- 172.21.15.69:0/3403886384 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f88c879280 0x55f88cc96700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.868+0000 7f28a41a2740 1 --2- 172.21.15.69:0/3403886384 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f88c8e5270 0x55f88cc93860 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.868+0000 7f287aff5640 0 client.15504 ms_handle_reset on v2:172.21.15.146:6838/2013283448 2024-04-25T11:26:00.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.868+0000 7f28a41a2740 1 --2- 172.21.15.69:0/3403886384 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f88c8c4590 0x55f88cc93320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.868+0000 7f28a41a2740 1 -- 172.21.15.69:0/3403886384 >> 172.21.15.69:0/3403886384 conn(0x55f88c8d93c0 msgr2=0x55f88cc8e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:26:00.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.868+0000 7f28a41a2740 1 -- 172.21.15.69:0/3403886384 shutdown_connections 2024-04-25T11:26:00.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.868+0000 7f28a41a2740 1 -- 172.21.15.69:0/3403886384 wait complete. 2024-04-25T11:26:00.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.868+0000 7f28a41a2740 20 client.15504 trim_cache size 0 max 16384 2024-04-25T11:26:00.868 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-25T11:26:00.868 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "f7ed73dc-02f4-11ef-bc93-c7b262605968", 2024-04-25T11:26:00.868 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 0, 2024-04-25T11:26:00.868 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 24515, 2024-04-25T11:26:00.868 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-04-25T11:26:00.868 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-04-25T11:26:00.868 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-04-25T11:26:00.868 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 139.249199713, 2024-04-25T11:26:00.869 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 28, 2024-04-25T11:26:00.869 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 84, 2024-04-25T11:26:00.869 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 80, 2024-04-25T11:26:00.869 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 146.19027829999999 2024-04-25T11:26:00.869 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-25T11:26:00.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.868+0000 7f289d2f2640 1 -- 172.21.15.69:0/1771533510 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f28800afaf0 msgr2=0x7f28800b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:00.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.868+0000 7f289d2f2640 1 --2- 172.21.15.69:0/1771533510 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f28800afaf0 0x7f28800b2000 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2890006010 tx=0x7f28900095c0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.868+0000 7f289d2f2640 1 -- 172.21.15.69:0/1771533510 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28981029e0 msgr2=0x7f2898198fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:00.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.868+0000 7f289d2f2640 1 --2- 172.21.15.69:0/1771533510 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28981029e0 0x7f2898198fd0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f288c005d00 tx=0x7f288c00dde0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.869+0000 7f289d2f2640 1 -- 172.21.15.69:0/1771533510 shutdown_connections 2024-04-25T11:26:00.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.869+0000 7f289d2f2640 1 --2- 172.21.15.69:0/1771533510 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f28800afaf0 0x7f28800b2000 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.869+0000 7f289d2f2640 1 --2- 172.21.15.69:0/1771533510 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2898108920 0x7f28981a04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.869+0000 7f289d2f2640 1 --2- 172.21.15.69:0/1771533510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2898103c20 0x7f2898199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.869+0000 7f289d2f2640 1 --2- 172.21.15.69:0/1771533510 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28981029e0 0x7f2898198fd0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:00.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.869+0000 7f289d2f2640 1 -- 172.21.15.69:0/1771533510 >> 172.21.15.69:0/1771533510 conn(0x7f28980fdbd0 msgr2=0x7f2898104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:26:00.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.869+0000 7f289d2f2640 1 -- 172.21.15.69:0/1771533510 shutdown_connections 2024-04-25T11:26:00.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:00.869+0000 7f289d2f2640 1 -- 172.21.15.69:0/1771533510 wait complete. 2024-04-25T11:26:00.872 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T11:26:00.883 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T11:26:00.883 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T11:26:00.883 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T11:26:00.883 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-25T11:26:00.894 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-25T11:26:00.894 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-04-25T11:26:00.894 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-04-25T11:26:00.895 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T11:26:00.895 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T11:26:00.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:26:00.895 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list 2024-04-25T11:26:00.910 INFO:teuthology.orchestra.run.smithi069.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-25T11:26:00.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:26:00.911 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T11:26:00.970 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:00 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/5448446' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:26:00.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:26:00.974 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link delete ceph-brx 2024-04-25T11:26:01.067 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-25T11:26:01.067 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-25T11:26:01.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:00 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/5448446' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:26:01.305 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:26:01.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:00 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/5448446' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:26:02.187 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:01 smithi069 ceph-mon[29188]: pgmap v359: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:26:02.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:01 smithi152 ceph-mon[32701]: pgmap v359: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:26:02.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:01 smithi146 ceph-mon[32821]: pgmap v359: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:26:03.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.070+0000 7ff8b244a700 1 -- 172.21.15.69:0/3534495360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8ac1091a0 msgr2=0x7ff8ac1095b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:03.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.070+0000 7ff8b244a700 1 --2- 172.21.15.69:0/3534495360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8ac1091a0 0x7ff8ac1095b0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff89c009990 tx=0x7ff89c009ca0 comp rx=0 tx=0).stop 2024-04-25T11:26:03.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.070+0000 7ff8b244a700 1 -- 172.21.15.69:0/3534495360 shutdown_connections 2024-04-25T11:26:03.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.070+0000 7ff8b244a700 1 --2- 172.21.15.69:0/3534495360 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8ac102f80 0x7ff8ac111010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:03.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.070+0000 7ff8b244a700 1 --2- 172.21.15.69:0/3534495360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8ac1091a0 0x7ff8ac1095b0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:03.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.070+0000 7ff8b244a700 1 --2- 172.21.15.69:0/3534495360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8ac102630 0x7ff8ac102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:03.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.070+0000 7ff8b244a700 1 -- 172.21.15.69:0/3534495360 >> 172.21.15.69:0/3534495360 conn(0x7ff8ac0fe3f0 msgr2=0x7ff8ac100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:26:03.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.070+0000 7ff8b244a700 1 -- 172.21.15.69:0/3534495360 shutdown_connections 2024-04-25T11:26:03.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.070+0000 7ff8b244a700 1 -- 172.21.15.69:0/3534495360 wait complete. 2024-04-25T11:26:03.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.070+0000 7ff8b244a700 1 Processor -- start 2024-04-25T11:26:03.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.070+0000 7ff8b244a700 1 -- start start 2024-04-25T11:26:03.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.071+0000 7ff8b244a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8ac102630 0x7ff8ac1a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:03.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.071+0000 7ff8b244a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8ac102f80 0x7ff8ac1a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:03.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.071+0000 7ff8b244a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8ac1091a0 0x7ff8ac19aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:03.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.071+0000 7ff8b244a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ac114300 con 0x7ff8ac102f80 2024-04-25T11:26:03.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.071+0000 7ff8b244a700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ac114180 con 0x7ff8ac102630 2024-04-25T11:26:03.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.071+0000 7ff8b244a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ac114480 con 0x7ff8ac1091a0 2024-04-25T11:26:03.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.071+0000 7ff8a3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8ac102f80 0x7ff8ac1a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:03.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.071+0000 7ff8a3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8ac102f80 0x7ff8ac1a0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39858/0 (socket says 172.21.15.69:39858) 2024-04-25T11:26:03.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.071+0000 7ff8a3fff700 1 -- 172.21.15.69:0/49035629 learned_addr learned my addr 172.21.15.69:0/49035629 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:26:03.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.071+0000 7ff8b09e7700 1 --2- 172.21.15.69:0/49035629 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8ac1091a0 0x7ff8ac19aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:03.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.071+0000 7ff8abfff700 1 --2- 172.21.15.69:0/49035629 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8ac102630 0x7ff8ac1a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:03.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.071+0000 7ff8a3fff700 1 -- 172.21.15.69:0/49035629 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8ac1091a0 msgr2=0x7ff8ac19aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:03.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.071+0000 7ff8a3fff700 1 --2- 172.21.15.69:0/49035629 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8ac1091a0 0x7ff8ac19aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:03.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.071+0000 7ff8a3fff700 1 -- 172.21.15.69:0/49035629 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8ac102630 msgr2=0x7ff8ac1a09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:03.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.071+0000 7ff8a3fff700 1 --2- 172.21.15.69:0/49035629 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8ac102630 0x7ff8ac1a09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:03.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.071+0000 7ff8a3fff700 1 -- 172.21.15.69:0/49035629 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8ac19b2d0 con 0x7ff8ac102f80 2024-04-25T11:26:03.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.071+0000 7ff8a3fff700 1 --2- 172.21.15.69:0/49035629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8ac102f80 0x7ff8ac1a0f00 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff89400c8f0 tx=0x7ff89400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:26:03.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.071+0000 7ff8a9ffb700 1 -- 172.21.15.69:0/49035629 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff894007940 con 0x7ff8ac102f80 2024-04-25T11:26:03.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.072+0000 7ff8b244a700 1 -- 172.21.15.69:0/49035629 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8ac19b5c0 con 0x7ff8ac102f80 2024-04-25T11:26:03.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.072+0000 7ff8a9ffb700 1 -- 172.21.15.69:0/49035629 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff89400f450 con 0x7ff8ac102f80 2024-04-25T11:26:03.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.072+0000 7ff8a9ffb700 1 -- 172.21.15.69:0/49035629 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff894018660 con 0x7ff8ac102f80 2024-04-25T11:26:03.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.072+0000 7ff8b244a700 1 -- 172.21.15.69:0/49035629 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8ac103740 con 0x7ff8ac102f80 2024-04-25T11:26:03.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.073+0000 7ff8b244a700 1 -- 172.21.15.69:0/49035629 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8ac051190 con 0x7ff8ac102f80 2024-04-25T11:26:03.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.074+0000 7ff8a9ffb700 1 -- 172.21.15.69:0/49035629 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7ff89400f900 con 0x7ff8ac102f80 2024-04-25T11:26:03.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.075+0000 7ff8a9ffb700 1 --2- 172.21.15.69:0/49035629 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ff8980afb80 0x7ff8980b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:03.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.075+0000 7ff8abfff700 1 --2- 172.21.15.69:0/49035629 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ff8980afb80 0x7ff8980b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:03.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.075+0000 7ff8a9ffb700 1 -- 172.21.15.69:0/49035629 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7ff8940def50 con 0x7ff8ac102f80 2024-04-25T11:26:03.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.076+0000 7ff8abfff700 1 --2- 172.21.15.69:0/49035629 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ff8980afb80 0x7ff8980b2090 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff89c006010 tx=0x7ff89c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:26:03.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.079+0000 7ff8a9ffb700 1 -- 172.21.15.69:0/49035629 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff894014030 con 0x7ff8ac102f80 2024-04-25T11:26:03.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.318+0000 7ff8b244a700 1 -- 172.21.15.69:0/49035629 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff8ac02e510 con 0x7ff8ac102f80 2024-04-25T11:26:03.311 INFO:teuthology.orchestra.run.smithi069.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T11:26:03.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.319+0000 7ff8a9ffb700 1 -- 172.21.15.69:0/49035629 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) ==== 53+0+88 (secure 0 0 0) 0x7ff8940a77f0 con 0x7ff8ac102f80 2024-04-25T11:26:03.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.321+0000 7ff8b244a700 1 -- 172.21.15.69:0/49035629 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ff8980afb80 msgr2=0x7ff8980b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:03.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.321+0000 7ff8b244a700 1 --2- 172.21.15.69:0/49035629 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ff8980afb80 0x7ff8980b2090 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff89c006010 tx=0x7ff89c00b540 comp rx=0 tx=0).stop 2024-04-25T11:26:03.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.321+0000 7ff8b244a700 1 -- 172.21.15.69:0/49035629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8ac102f80 msgr2=0x7ff8ac1a0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:03.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.321+0000 7ff8b244a700 1 --2- 172.21.15.69:0/49035629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8ac102f80 0x7ff8ac1a0f00 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff89400c8f0 tx=0x7ff89400cc00 comp rx=0 tx=0).stop 2024-04-25T11:26:03.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.321+0000 7ff8b244a700 1 -- 172.21.15.69:0/49035629 shutdown_connections 2024-04-25T11:26:03.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.321+0000 7ff8b244a700 1 --2- 172.21.15.69:0/49035629 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ff8980afb80 0x7ff8980b2090 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:03.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.321+0000 7ff8b244a700 1 --2- 172.21.15.69:0/49035629 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8ac1091a0 0x7ff8ac19aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:03.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.321+0000 7ff8b244a700 1 --2- 172.21.15.69:0/49035629 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8ac102630 0x7ff8ac1a09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:03.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.321+0000 7ff8b244a700 1 --2- 172.21.15.69:0/49035629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8ac102f80 0x7ff8ac1a0f00 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:03.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.321+0000 7ff8b244a700 1 -- 172.21.15.69:0/49035629 >> 172.21.15.69:0/49035629 conn(0x7ff8ac0fe3f0 msgr2=0x7ff8ac100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:26:03.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.321+0000 7ff8b244a700 1 -- 172.21.15.69:0/49035629 shutdown_connections 2024-04-25T11:26:03.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:03.321+0000 7ff8b244a700 1 -- 172.21.15.69:0/49035629 wait complete. 2024-04-25T11:26:03.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:03 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/49035629' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T11:26:03.729 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:03 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/49035629' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T11:26:03.848 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:03 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/49035629' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T11:26:04.050 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T11:26:04.050 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:26:04.050 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi069.front.sepia.ceph.com 2024-04-25T11:26:04.050 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T11:26:04.050 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T11:26:04.050 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T11:26:04.050 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T11:26:04.051 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:26:04.051 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:26:04.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:26:04.051 DEBUG:teuthology.orchestra.run.smithi069:> ip addr 2024-04-25T11:26:04.068 INFO:teuthology.orchestra.run.smithi069.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:26:04.068 INFO:teuthology.orchestra.run.smithi069.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:26:04.068 INFO:teuthology.orchestra.run.smithi069.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:26:04.068 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:26:04.068 INFO:teuthology.orchestra.run.smithi069.stdout: inet6 ::1/128 scope host 2024-04-25T11:26:04.068 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:26:04.068 INFO:teuthology.orchestra.run.smithi069.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:26:04.068 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:d8:d9:c2 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:26:04.068 INFO:teuthology.orchestra.run.smithi069.stdout: altname enp7s0f0 2024-04-25T11:26:04.068 INFO:teuthology.orchestra.run.smithi069.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:26:04.069 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:d8:d9:c3 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:26:04.069 INFO:teuthology.orchestra.run.smithi069.stdout: altname enp7s0f1 2024-04-25T11:26:04.069 INFO:teuthology.orchestra.run.smithi069.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:26:04.069 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:8f:d7:36 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:26:04.069 INFO:teuthology.orchestra.run.smithi069.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:26:04.069 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:8f:d7:37 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:26:04.069 INFO:teuthology.orchestra.run.smithi069.stdout: inet 172.21.15.69/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T11:26:04.069 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft 42079sec preferred_lft 42079sec 2024-04-25T11:26:04.069 INFO:teuthology.orchestra.run.smithi069.stdout: inet6 fe80::4637:f1e7:b6c7:4071/64 scope link noprefixroute 2024-04-25T11:26:04.069 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:26:04.069 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T11:26:04.070 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:26:04.070 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-04-25T11:26:04.070 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link add name ceph-brx type bridge 2024-04-25T11:26:04.070 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip addr flush dev ceph-brx 2024-04-25T11:26:04.070 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link set ceph-brx up 2024-04-25T11:26:04.070 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T11:26:04.070 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-04-25T11:26:04.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:26:04.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:26:04.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:26:04.206 DEBUG:teuthology.orchestra.run.smithi069:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:26:04.228 INFO:teuthology.orchestra.run.smithi069.stdout:1 2024-04-25T11:26:04.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:26:04.231 DEBUG:teuthology.orchestra.run.smithi069:> route 2024-04-25T11:26:04.285 INFO:teuthology.orchestra.run.smithi069.stdout:Kernel IP routing table 2024-04-25T11:26:04.286 INFO:teuthology.orchestra.run.smithi069.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:26:04.286 INFO:teuthology.orchestra.run.smithi069.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T11:26:04.286 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T11:26:04.286 INFO:teuthology.orchestra.run.smithi069.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:26:04.286 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:26:04.286 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-04-25T11:26:04.286 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T11:26:04.286 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T11:26:04.286 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T11:26:04.287 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-04-25T11:26:04.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:26:04.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:26:04.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:26:04.411 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list 2024-04-25T11:26:04.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:26:04.427 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list-id 2024-04-25T11:26:04.481 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:26:04.481 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-04-25T11:26:04.481 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:26:04.481 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T11:26:04.482 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-04-25T11:26:04.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:26:04.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:26:04.595 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T11:26:04.595 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:26:04.595 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-04-25T11:26:04.595 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T11:26:04.596 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T11:26:04.596 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T11:26:04.596 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T11:26:04.596 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T11:26:04.596 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-04-25T11:26:04.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:26:04.652 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:04 smithi069 ceph-mon[29188]: pgmap v360: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:26:04.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:26:04.706 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:26:04.706 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-04-25T11:26:04.706 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link set brx.0 up 2024-04-25T11:26:04.706 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T11:26:04.706 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-04-25T11:26:04.729 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:04 smithi152 ceph-mon[32701]: pgmap v360: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:26:04.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:26:04.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:26:04.782 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-04-25T11:26:04.782 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:26:04.782 DEBUG:teuthology.orchestra.run.smithi069:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T11:26:04.796 INFO:teuthology.orchestra.run.smithi069.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:26:04.797 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:26:04.797 DEBUG:teuthology.orchestra.run.smithi069:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:26:04.847 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:04 smithi146 ceph-mon[32821]: pgmap v360: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:26:04.851 DEBUG:teuthology.orchestra.run.smithi069:> sudo modprobe fuse 2024-04-25T11:26:04.915 DEBUG:teuthology.orchestra.run.smithi069:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T11:26:04.970 INFO:teuthology.orchestra.run.smithi069.stdout:/proc 2024-04-25T11:26:04.971 INFO:teuthology.orchestra.run.smithi069.stdout:/sys 2024-04-25T11:26:04.971 INFO:teuthology.orchestra.run.smithi069.stdout:/dev 2024-04-25T11:26:04.971 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/security 2024-04-25T11:26:04.971 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/shm 2024-04-25T11:26:04.971 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/pts 2024-04-25T11:26:04.971 INFO:teuthology.orchestra.run.smithi069.stdout:/run 2024-04-25T11:26:04.971 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/cgroup 2024-04-25T11:26:04.971 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/pstore 2024-04-25T11:26:04.971 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/bpf 2024-04-25T11:26:04.971 INFO:teuthology.orchestra.run.smithi069.stdout:/ 2024-04-25T11:26:04.971 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/selinux 2024-04-25T11:26:04.971 INFO:teuthology.orchestra.run.smithi069.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T11:26:04.972 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/hugepages 2024-04-25T11:26:04.972 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/mqueue 2024-04-25T11:26:04.972 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/debug 2024-04-25T11:26:04.972 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/tracing 2024-04-25T11:26:04.972 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T11:26:04.972 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/fuse/connections 2024-04-25T11:26:04.972 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/config 2024-04-25T11:26:04.972 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T11:26:04.972 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T11:26:04.972 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T11:26:04.972 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T11:26:04.972 INFO:teuthology.orchestra.run.smithi069.stdout:/run/user/1000 2024-04-25T11:26:04.972 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/ceph 2024-04-25T11:26:04.972 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay 2024-04-25T11:26:04.973 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/9f4a46c23d4d86068aa88b4ed1c37628a2eac7a16343feefaeee7fe7577a197a/merged 2024-04-25T11:26:04.973 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/0e9a9a4a9937a1dfd02a7cf62b3058484401334d2dcad2fe35369ee846c671d2/merged 2024-04-25T11:26:04.973 INFO:teuthology.orchestra.run.smithi069.stdout:/run/user/0 2024-04-25T11:26:04.973 INFO:teuthology.orchestra.run.smithi069.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T11:26:04.973 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/2ec9d973deed8ffc94944b1e27e1cb8425417481ba8a54de9056a925de4ebbf6/merged 2024-04-25T11:26:04.973 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/47acf86d54d28602a952d5304647222bc771de57ca486ef4467a1678184c79cc/merged 2024-04-25T11:26:04.973 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/2b9ea3a24270210d404542f52a91f5ff1d1cd981381372eb54c8bc7fba4180be/merged 2024-04-25T11:26:04.973 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/ec425cdf63598fd0b9c9d832b419e769899261ec1b4fc1169beb34d51205c41e/merged 2024-04-25T11:26:04.973 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/1655ce1e6d74b3eb8cd3939568945e34e5f2bec725a5cc6b69244f04a4a8a219/merged 2024-04-25T11:26:04.973 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/6a86092111ee15e201da99074eddf4f6103d966a47d8a3794d659f1ebcc06710/merged 2024-04-25T11:26:04.973 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/5e34351696aeceb6d98047f196043703aec77e26a7575cf28fc317b0b6da0847/merged 2024-04-25T11:26:04.973 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/2b4fca38630f372edbce438286af127f856a1cfc9f37b0f9cfa2542baee1f413/merged 2024-04-25T11:26:04.973 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns 2024-04-25T11:26:04.973 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:26:04.973 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:26:04.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:26:04.974 DEBUG:teuthology.orchestra.run.smithi069:> ls /sys/fs/fuse/connections 2024-04-25T11:26:05.026 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T11:26:05.026 DEBUG:teuthology.orchestra.run.smithi069:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/71b2a7dc-2e75-4036-a954-acca3df9cf14 --client_fs=cephfs) 2024-04-25T11:26:05.068 DEBUG:teuthology.orchestra.run.smithi069:> sudo modprobe fuse 2024-04-25T11:26:05.092 DEBUG:teuthology.orchestra.run.smithi069:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T11:26:05.149 INFO:teuthology.orchestra.run.smithi069.stdout:/proc 2024-04-25T11:26:05.149 INFO:teuthology.orchestra.run.smithi069.stdout:/sys 2024-04-25T11:26:05.149 INFO:teuthology.orchestra.run.smithi069.stdout:/dev 2024-04-25T11:26:05.149 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/security 2024-04-25T11:26:05.149 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/shm 2024-04-25T11:26:05.149 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/pts 2024-04-25T11:26:05.149 INFO:teuthology.orchestra.run.smithi069.stdout:/run 2024-04-25T11:26:05.149 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/cgroup 2024-04-25T11:26:05.149 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/pstore 2024-04-25T11:26:05.150 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/bpf 2024-04-25T11:26:05.150 INFO:teuthology.orchestra.run.smithi069.stdout:/ 2024-04-25T11:26:05.150 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/selinux 2024-04-25T11:26:05.150 INFO:teuthology.orchestra.run.smithi069.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T11:26:05.150 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/hugepages 2024-04-25T11:26:05.150 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/mqueue 2024-04-25T11:26:05.150 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/debug 2024-04-25T11:26:05.150 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/tracing 2024-04-25T11:26:05.150 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T11:26:05.150 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/fuse/connections 2024-04-25T11:26:05.150 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/config 2024-04-25T11:26:05.150 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T11:26:05.150 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T11:26:05.150 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T11:26:05.151 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T11:26:05.151 INFO:teuthology.orchestra.run.smithi069.stdout:/run/user/1000 2024-04-25T11:26:05.151 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/ceph 2024-04-25T11:26:05.151 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay 2024-04-25T11:26:05.151 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/9f4a46c23d4d86068aa88b4ed1c37628a2eac7a16343feefaeee7fe7577a197a/merged 2024-04-25T11:26:05.151 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/0e9a9a4a9937a1dfd02a7cf62b3058484401334d2dcad2fe35369ee846c671d2/merged 2024-04-25T11:26:05.151 INFO:teuthology.orchestra.run.smithi069.stdout:/run/user/0 2024-04-25T11:26:05.151 INFO:teuthology.orchestra.run.smithi069.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T11:26:05.151 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/2ec9d973deed8ffc94944b1e27e1cb8425417481ba8a54de9056a925de4ebbf6/merged 2024-04-25T11:26:05.151 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/47acf86d54d28602a952d5304647222bc771de57ca486ef4467a1678184c79cc/merged 2024-04-25T11:26:05.151 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/2b9ea3a24270210d404542f52a91f5ff1d1cd981381372eb54c8bc7fba4180be/merged 2024-04-25T11:26:05.151 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/ec425cdf63598fd0b9c9d832b419e769899261ec1b4fc1169beb34d51205c41e/merged 2024-04-25T11:26:05.151 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/1655ce1e6d74b3eb8cd3939568945e34e5f2bec725a5cc6b69244f04a4a8a219/merged 2024-04-25T11:26:05.151 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/6a86092111ee15e201da99074eddf4f6103d966a47d8a3794d659f1ebcc06710/merged 2024-04-25T11:26:05.151 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/5e34351696aeceb6d98047f196043703aec77e26a7575cf28fc317b0b6da0847/merged 2024-04-25T11:26:05.152 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/2b4fca38630f372edbce438286af127f856a1cfc9f37b0f9cfa2542baee1f413/merged 2024-04-25T11:26:05.152 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns 2024-04-25T11:26:05.152 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:26:05.152 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:26:05.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:26:05.152 DEBUG:teuthology.orchestra.run.smithi069:> ls /sys/fs/fuse/connections 2024-04-25T11:26:05.158 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi069.stderr:2024-04-25T11:26:05.166+0000 7f373d720480 -1 init, newargv = 0x7f372c0046a0 newargc=15 2024-04-25T11:26:05.159 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi069.stderr:ceph-fuse[83193]: starting ceph client 2024-04-25T11:26:05.183 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi069.stderr:ceph-fuse[83193]: starting fuse 2024-04-25T11:26:06.275 DEBUG:teuthology.orchestra.run.smithi069:> sudo modprobe fuse 2024-04-25T11:26:06.301 DEBUG:teuthology.orchestra.run.smithi069:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T11:26:06.357 INFO:teuthology.orchestra.run.smithi069.stdout:/proc 2024-04-25T11:26:06.357 INFO:teuthology.orchestra.run.smithi069.stdout:/sys 2024-04-25T11:26:06.357 INFO:teuthology.orchestra.run.smithi069.stdout:/dev 2024-04-25T11:26:06.357 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/security 2024-04-25T11:26:06.357 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/shm 2024-04-25T11:26:06.357 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/pts 2024-04-25T11:26:06.357 INFO:teuthology.orchestra.run.smithi069.stdout:/run 2024-04-25T11:26:06.357 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/cgroup 2024-04-25T11:26:06.357 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/pstore 2024-04-25T11:26:06.357 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/bpf 2024-04-25T11:26:06.357 INFO:teuthology.orchestra.run.smithi069.stdout:/ 2024-04-25T11:26:06.357 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/selinux 2024-04-25T11:26:06.357 INFO:teuthology.orchestra.run.smithi069.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T11:26:06.358 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/hugepages 2024-04-25T11:26:06.358 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/mqueue 2024-04-25T11:26:06.358 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/debug 2024-04-25T11:26:06.358 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/tracing 2024-04-25T11:26:06.358 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T11:26:06.358 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/fuse/connections 2024-04-25T11:26:06.358 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/config 2024-04-25T11:26:06.358 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T11:26:06.358 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T11:26:06.358 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T11:26:06.358 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T11:26:06.358 INFO:teuthology.orchestra.run.smithi069.stdout:/run/user/1000 2024-04-25T11:26:06.358 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/ceph 2024-04-25T11:26:06.358 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay 2024-04-25T11:26:06.359 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/9f4a46c23d4d86068aa88b4ed1c37628a2eac7a16343feefaeee7fe7577a197a/merged 2024-04-25T11:26:06.359 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/0e9a9a4a9937a1dfd02a7cf62b3058484401334d2dcad2fe35369ee846c671d2/merged 2024-04-25T11:26:06.359 INFO:teuthology.orchestra.run.smithi069.stdout:/run/user/0 2024-04-25T11:26:06.359 INFO:teuthology.orchestra.run.smithi069.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T11:26:06.359 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/2ec9d973deed8ffc94944b1e27e1cb8425417481ba8a54de9056a925de4ebbf6/merged 2024-04-25T11:26:06.359 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/47acf86d54d28602a952d5304647222bc771de57ca486ef4467a1678184c79cc/merged 2024-04-25T11:26:06.359 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/2b9ea3a24270210d404542f52a91f5ff1d1cd981381372eb54c8bc7fba4180be/merged 2024-04-25T11:26:06.359 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/ec425cdf63598fd0b9c9d832b419e769899261ec1b4fc1169beb34d51205c41e/merged 2024-04-25T11:26:06.359 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/1655ce1e6d74b3eb8cd3939568945e34e5f2bec725a5cc6b69244f04a4a8a219/merged 2024-04-25T11:26:06.359 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/6a86092111ee15e201da99074eddf4f6103d966a47d8a3794d659f1ebcc06710/merged 2024-04-25T11:26:06.359 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/5e34351696aeceb6d98047f196043703aec77e26a7575cf28fc317b0b6da0847/merged 2024-04-25T11:26:06.359 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/2b4fca38630f372edbce438286af127f856a1cfc9f37b0f9cfa2542baee1f413/merged 2024-04-25T11:26:06.359 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns 2024-04-25T11:26:06.359 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:26:06.359 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:26:06.359 INFO:teuthology.orchestra.run.smithi069.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T11:26:06.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:26:06.360 DEBUG:teuthology.orchestra.run.smithi069:> ls /sys/fs/fuse/connections 2024-04-25T11:26:06.412 INFO:teuthology.orchestra.run.smithi069.stdout:115 2024-04-25T11:26:06.412 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [115] 2024-04-25T11:26:06.412 DEBUG:teuthology.orchestra.run.smithi069:> sudo stdin-killer -- python3 -c ' 2024-04-25T11:26:06.412 DEBUG:teuthology.orchestra.run.smithi069:> import glob 2024-04-25T11:26:06.413 DEBUG:teuthology.orchestra.run.smithi069:> import re 2024-04-25T11:26:06.413 DEBUG:teuthology.orchestra.run.smithi069:> import os 2024-04-25T11:26:06.413 DEBUG:teuthology.orchestra.run.smithi069:> import subprocess 2024-04-25T11:26:06.413 DEBUG:teuthology.orchestra.run.smithi069:> 2024-04-25T11:26:06.413 DEBUG:teuthology.orchestra.run.smithi069:> def _find_admin_socket(client_name): 2024-04-25T11:26:06.413 DEBUG:teuthology.orchestra.run.smithi069:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T11:26:06.413 DEBUG:teuthology.orchestra.run.smithi069:> files = glob.glob(asok_path) 2024-04-25T11:26:06.413 DEBUG:teuthology.orchestra.run.smithi069:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T11:26:06.413 DEBUG:teuthology.orchestra.run.smithi069:> 2024-04-25T11:26:06.413 DEBUG:teuthology.orchestra.run.smithi069:> # Given a non-glob path, it better be there 2024-04-25T11:26:06.413 DEBUG:teuthology.orchestra.run.smithi069:> if "*" not in asok_path: 2024-04-25T11:26:06.413 DEBUG:teuthology.orchestra.run.smithi069:> assert(len(files) == 1) 2024-04-25T11:26:06.413 DEBUG:teuthology.orchestra.run.smithi069:> return files[0] 2024-04-25T11:26:06.413 DEBUG:teuthology.orchestra.run.smithi069:> 2024-04-25T11:26:06.413 DEBUG:teuthology.orchestra.run.smithi069:> for f in files: 2024-04-25T11:26:06.413 DEBUG:teuthology.orchestra.run.smithi069:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T11:26:06.414 DEBUG:teuthology.orchestra.run.smithi069:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T11:26:06.414 DEBUG:teuthology.orchestra.run.smithi069:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T11:26:06.414 DEBUG:teuthology.orchestra.run.smithi069:> contents = proc_f.read() 2024-04-25T11:26:06.414 DEBUG:teuthology.orchestra.run.smithi069:> if mountpoint in contents: 2024-04-25T11:26:06.414 DEBUG:teuthology.orchestra.run.smithi069:> return f 2024-04-25T11:26:06.414 DEBUG:teuthology.orchestra.run.smithi069:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T11:26:06.414 DEBUG:teuthology.orchestra.run.smithi069:> 2024-04-25T11:26:06.414 DEBUG:teuthology.orchestra.run.smithi069:> print(_find_admin_socket("client.0")) 2024-04-25T11:26:06.414 DEBUG:teuthology.orchestra.run.smithi069:> ' 2024-04-25T11:26:06.529 INFO:teuthology.orchestra.run.smithi069.stdout:/var/run/ceph/ceph-client.0.83193.asok 2024-04-25T11:26:06.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:26:06.540 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.83193.asok 2024-04-25T11:26:06.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:26:06.540 DEBUG:teuthology.orchestra.run.smithi069:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.83193.asok status 2024-04-25T11:26:06.681 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-25T11:26:06.682 INFO:teuthology.orchestra.run.smithi069.stdout: "metadata": { 2024-04-25T11:26:06.682 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T11:26:06.682 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T11:26:06.682 INFO:teuthology.orchestra.run.smithi069.stdout: "entity_id": "0", 2024-04-25T11:26:06.682 INFO:teuthology.orchestra.run.smithi069.stdout: "hostname": "smithi069", 2024-04-25T11:26:06.682 INFO:teuthology.orchestra.run.smithi069.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T11:26:06.682 INFO:teuthology.orchestra.run.smithi069.stdout: "pid": "83193", 2024-04-25T11:26:06.682 INFO:teuthology.orchestra.run.smithi069.stdout: "root": "/volumes/qa/sv_0/71b2a7dc-2e75-4036-a954-acca3df9cf14" 2024-04-25T11:26:06.682 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:26:06.682 INFO:teuthology.orchestra.run.smithi069.stdout: "dentry_count": 0, 2024-04-25T11:26:06.682 INFO:teuthology.orchestra.run.smithi069.stdout: "dentry_pinned_count": 0, 2024-04-25T11:26:06.682 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 15516, 2024-04-25T11:26:06.683 INFO:teuthology.orchestra.run.smithi069.stdout: "inst": { 2024-04-25T11:26:06.683 INFO:teuthology.orchestra.run.smithi069.stdout: "name": { 2024-04-25T11:26:06.683 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "client", 2024-04-25T11:26:06.683 INFO:teuthology.orchestra.run.smithi069.stdout: "num": 15516 2024-04-25T11:26:06.683 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:26:06.683 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": { 2024-04-25T11:26:06.683 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v1", 2024-04-25T11:26:06.683 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "192.168.0.1:0", 2024-04-25T11:26:06.683 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 4085208641 2024-04-25T11:26:06.683 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:26:06.683 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:26:06.683 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": { 2024-04-25T11:26:06.683 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v1", 2024-04-25T11:26:06.683 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "192.168.0.1:0", 2024-04-25T11:26:06.684 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 4085208641 2024-04-25T11:26:06.684 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:26:06.684 INFO:teuthology.orchestra.run.smithi069.stdout: "inst_str": "client.15516 192.168.0.1:0/4085208641", 2024-04-25T11:26:06.684 INFO:teuthology.orchestra.run.smithi069.stdout: "addr_str": "192.168.0.1:0/4085208641", 2024-04-25T11:26:06.684 INFO:teuthology.orchestra.run.smithi069.stdout: "inode_count": 5, 2024-04-25T11:26:06.684 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_epoch": 28, 2024-04-25T11:26:06.684 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_epoch": 84, 2024-04-25T11:26:06.684 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_epoch_barrier": 0, 2024-04-25T11:26:06.684 INFO:teuthology.orchestra.run.smithi069.stdout: "blocklisted": false, 2024-04-25T11:26:06.684 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs" 2024-04-25T11:26:06.684 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-25T11:26:06.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:26:06.690 DEBUG:teuthology.orchestra.run.smithi069:> stat --file-system '--printf=%T 2024-04-25T11:26:06.690 DEBUG:teuthology.orchestra.run.smithi069:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T11:26:06.706 INFO:teuthology.orchestra.run.smithi069.stdout:fuseblk 2024-04-25T11:26:06.706 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T11:26:06.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:26:06.706 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:26:06.772 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T11:26:06.783 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T11:26:06.783 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi069.front.sepia.ceph.com 2024-04-25T11:26:06.783 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-04-25T11:26:06.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:06.929+0000 7fd9f8092640 1 -- 172.21.15.69:0/580986988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd9f0104d90 msgr2=0x7fd9f01051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:06.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:06.929+0000 7fd9f8092640 1 --2- 172.21.15.69:0/580986988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd9f0104d90 0x7fd9f01051a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0009900 tx=0x7fd9e002a410 comp rx=0 tx=0).stop 2024-04-25T11:26:06.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:06.929+0000 7fd9f8092640 1 -- 172.21.15.69:0/580986988 shutdown_connections 2024-04-25T11:26:06.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:06.929+0000 7fd9f8092640 1 --2- 172.21.15.69:0/580986988 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9f0109a90 0x7fd9f010beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:06.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:06.929+0000 7fd9f8092640 1 --2- 172.21.15.69:0/580986988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd9f0104d90 0x7fd9f01051a0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:06.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:06.929+0000 7fd9f8092640 1 --2- 172.21.15.69:0/580986988 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9f0103b50 0x7fd9f0104850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:06.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:06.929+0000 7fd9f8092640 1 -- 172.21.15.69:0/580986988 >> 172.21.15.69:0/580986988 conn(0x7fd9f00fed20 msgr2=0x7fd9f01011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:26:06.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:06.929+0000 7fd9f8092640 1 -- 172.21.15.69:0/580986988 shutdown_connections 2024-04-25T11:26:06.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:06.929+0000 7fd9f8092640 1 -- 172.21.15.69:0/580986988 wait complete. 2024-04-25T11:26:06.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:06.929+0000 7fd9f8092640 1 Processor -- start 2024-04-25T11:26:06.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:06.929+0000 7fd9f8092640 1 -- start start 2024-04-25T11:26:06.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:06.930+0000 7fd9f8092640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9f0103b50 0x7fd9f019a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:06.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:06.930+0000 7fd9f8092640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9f0104d90 0x7fd9f019a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:06.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:06.930+0000 7fd9f8092640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd9f0109a90 0x7fd9f01a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:06.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:06.930+0000 7fd9f8092640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f010f5f0 con 0x7fd9f0109a90 2024-04-25T11:26:06.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:06.930+0000 7fd9f8092640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f010f470 con 0x7fd9f0104d90 2024-04-25T11:26:06.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:06.930+0000 7fd9f8092640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f010f770 con 0x7fd9f0103b50 2024-04-25T11:26:06.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:06.930+0000 7fd9f662f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd9f0109a90 0x7fd9f01a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:06.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:06.930+0000 7fd9f662f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd9f0109a90 0x7fd9f01a1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39868/0 (socket says 172.21.15.69:39868) 2024-04-25T11:26:06.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:06.930+0000 7fd9f662f640 1 -- 172.21.15.69:0/2034103725 learned_addr learned my addr 172.21.15.69:0/2034103725 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:26:06.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:06.930+0000 7fd9f562d640 1 --2- 172.21.15.69:0/2034103725 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9f0104d90 0x7fd9f019a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:06.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:06.930+0000 7fd9f662f640 1 -- 172.21.15.69:0/2034103725 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9f0103b50 msgr2=0x7fd9f019a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:06.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:06.930+0000 7fd9f662f640 1 --2- 172.21.15.69:0/2034103725 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9f0103b50 0x7fd9f019a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:06.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:06.930+0000 7fd9f662f640 1 -- 172.21.15.69:0/2034103725 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9f0104d90 msgr2=0x7fd9f019a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:06.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:06.930+0000 7fd9f662f640 1 --2- 172.21.15.69:0/2034103725 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9f0104d90 0x7fd9f019a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:06.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:06.930+0000 7fd9f662f640 1 -- 172.21.15.69:0/2034103725 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f01a1d00 con 0x7fd9f0109a90 2024-04-25T11:26:06.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:06.930+0000 7fd9f662f640 1 --2- 172.21.15.69:0/2034103725 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd9f0109a90 0x7fd9f01a1690 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec009fd0 tx=0x7fd9ec00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:26:06.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:06.930+0000 7fd9e6ffd640 1 -- 172.21.15.69:0/2034103725 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec011830 con 0x7fd9f0109a90 2024-04-25T11:26:06.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:06.930+0000 7fd9e6ffd640 1 -- 172.21.15.69:0/2034103725 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd9ec016070 con 0x7fd9f0109a90 2024-04-25T11:26:06.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:06.930+0000 7fd9f8092640 1 -- 172.21.15.69:0/2034103725 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9f01a1ff0 con 0x7fd9f0109a90 2024-04-25T11:26:06.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:06.930+0000 7fd9e6ffd640 1 -- 172.21.15.69:0/2034103725 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec010d20 con 0x7fd9f0109a90 2024-04-25T11:26:06.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:06.930+0000 7fd9f8092640 1 -- 172.21.15.69:0/2034103725 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9f01a2480 con 0x7fd9f0109a90 2024-04-25T11:26:06.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:06.931+0000 7fd9f8092640 1 -- 172.21.15.69:0/2034103725 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9f01088a0 con 0x7fd9f0109a90 2024-04-25T11:26:06.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:06.933+0000 7fd9e6ffd640 1 -- 172.21.15.69:0/2034103725 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7fd9ec0119d0 con 0x7fd9f0109a90 2024-04-25T11:26:06.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:06.933+0000 7fd9e6ffd640 1 --2- 172.21.15.69:0/2034103725 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fd9dc0afae0 0x7fd9dc0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:06.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:06.934+0000 7fd9f5e2e640 1 --2- 172.21.15.69:0/2034103725 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fd9dc0afae0 0x7fd9dc0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:06.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:06.934+0000 7fd9e6ffd640 1 -- 172.21.15.69:0/2034103725 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7fd9ec0ded20 con 0x7fd9f0109a90 2024-04-25T11:26:06.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:06.934+0000 7fd9f5e2e640 1 --2- 172.21.15.69:0/2034103725 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fd9dc0afae0 0x7fd9dc0b1ff0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0104750 tx=0x7fd9e80093c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:26:06.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:06.937+0000 7fd9e6ffd640 1 -- 172.21.15.69:0/2034103725 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd9ec0a7530 con 0x7fd9f0109a90 2024-04-25T11:26:07.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:07.154+0000 7fd9f8092640 1 -- 172.21.15.69:0/2034103725 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7fd9f002db50 con 0x7fd9f0109a90 2024-04-25T11:26:07.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:06 smithi069 ceph-mon[29188]: pgmap v361: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:26:07.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:06 smithi152 ceph-mon[32701]: pgmap v361: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:26:07.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:06 smithi146 ceph-mon[32821]: pgmap v361: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:26:07.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:07.882+0000 7fd9e6ffd640 1 -- 172.21.15.69:0/2034103725 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v29) ==== 118+0+0 (secure 0 0 0) 0x7fd9ec0a6e30 con 0x7fd9f0109a90 2024-04-25T11:26:07.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:07.886+0000 7fd9f8092640 1 -- 172.21.15.69:0/2034103725 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fd9dc0afae0 msgr2=0x7fd9dc0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:07.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:07.886+0000 7fd9f8092640 1 --2- 172.21.15.69:0/2034103725 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fd9dc0afae0 0x7fd9dc0b1ff0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0104750 tx=0x7fd9e80093c0 comp rx=0 tx=0).stop 2024-04-25T11:26:07.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:07.886+0000 7fd9f8092640 1 -- 172.21.15.69:0/2034103725 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd9f0109a90 msgr2=0x7fd9f01a1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:07.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:07.886+0000 7fd9f8092640 1 --2- 172.21.15.69:0/2034103725 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd9f0109a90 0x7fd9f01a1690 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec009fd0 tx=0x7fd9ec00b8e0 comp rx=0 tx=0).stop 2024-04-25T11:26:07.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:07.887+0000 7fd9f8092640 1 -- 172.21.15.69:0/2034103725 shutdown_connections 2024-04-25T11:26:07.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:07.887+0000 7fd9f8092640 1 --2- 172.21.15.69:0/2034103725 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fd9dc0afae0 0x7fd9dc0b1ff0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:07.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:07.887+0000 7fd9f8092640 1 --2- 172.21.15.69:0/2034103725 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd9f0109a90 0x7fd9f01a1690 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:07.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:07.887+0000 7fd9f8092640 1 --2- 172.21.15.69:0/2034103725 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9f0104d90 0x7fd9f019a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:07.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:07.887+0000 7fd9f8092640 1 --2- 172.21.15.69:0/2034103725 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9f0103b50 0x7fd9f019a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:07.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:07.887+0000 7fd9f8092640 1 -- 172.21.15.69:0/2034103725 >> 172.21.15.69:0/2034103725 conn(0x7fd9f00fed20 msgr2=0x7fd9f0105a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:26:07.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:07.887+0000 7fd9f8092640 1 -- 172.21.15.69:0/2034103725 shutdown_connections 2024-04-25T11:26:07.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:07.887+0000 7fd9f8092640 1 -- 172.21.15.69:0/2034103725 wait complete. 2024-04-25T11:26:07.893 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-04-25T11:26:08.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.016+0000 7f6ac12ca640 1 -- 172.21.15.69:0/1582681106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6abc102fa0 msgr2=0x7f6abc1033b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:08.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.016+0000 7f6ac12ca640 1 --2- 172.21.15.69:0/1582681106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6abc102fa0 0x7f6abc1033b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f6aac009900 tx=0x7f6aac02a410 comp rx=0 tx=0).stop 2024-04-25T11:26:08.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.016+0000 7f6ac12ca640 1 -- 172.21.15.69:0/1582681106 shutdown_connections 2024-04-25T11:26:08.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.016+0000 7f6ac12ca640 1 --2- 172.21.15.69:0/1582681106 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6abc1044c0 0x7f6abc10ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:08.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.016+0000 7f6ac12ca640 1 --2- 172.21.15.69:0/1582681106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6abc103980 0x7f6abc103e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:08.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.016+0000 7f6ac12ca640 1 --2- 172.21.15.69:0/1582681106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6abc102fa0 0x7f6abc1033b0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:08.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.016+0000 7f6ac12ca640 1 -- 172.21.15.69:0/1582681106 >> 172.21.15.69:0/1582681106 conn(0x7f6abc0fd3c0 msgr2=0x7f6abc0ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:26:08.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.016+0000 7f6ac12ca640 1 -- 172.21.15.69:0/1582681106 shutdown_connections 2024-04-25T11:26:08.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.016+0000 7f6ac12ca640 1 -- 172.21.15.69:0/1582681106 wait complete. 2024-04-25T11:26:08.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.017+0000 7f6ac12ca640 1 Processor -- start 2024-04-25T11:26:08.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.017+0000 7f6ac12ca640 1 -- start start 2024-04-25T11:26:08.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.017+0000 7f6ac12ca640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6abc102fa0 0x7f6abc199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:08.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.017+0000 7f6ac12ca640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6abc103980 0x7f6abc1995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:08.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.017+0000 7f6ac12ca640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6abc1044c0 0x7f6abc1a0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:08.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.017+0000 7f6ac12ca640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6abc10e400 con 0x7f6abc102fa0 2024-04-25T11:26:08.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.017+0000 7f6ac12ca640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6abc10e280 con 0x7f6abc1044c0 2024-04-25T11:26:08.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.017+0000 7f6ac12ca640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6abc10e580 con 0x7f6abc103980 2024-04-25T11:26:08.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.017+0000 7f6abaffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6abc102fa0 0x7f6abc199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:08.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.017+0000 7f6abaffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6abc102fa0 0x7f6abc199060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39896/0 (socket says 172.21.15.69:39896) 2024-04-25T11:26:08.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.017+0000 7f6abaffd640 1 -- 172.21.15.69:0/3692518672 learned_addr learned my addr 172.21.15.69:0/3692518672 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:26:08.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.017+0000 7f6abb7fe640 1 --2- 172.21.15.69:0/3692518672 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6abc1044c0 0x7f6abc1a0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:08.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.017+0000 7f6aba7fc640 1 --2- 172.21.15.69:0/3692518672 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6abc103980 0x7f6abc1995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:08.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.018+0000 7f6abaffd640 1 -- 172.21.15.69:0/3692518672 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6abc103980 msgr2=0x7f6abc1995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:08.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.018+0000 7f6abaffd640 1 --2- 172.21.15.69:0/3692518672 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6abc103980 0x7f6abc1995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:08.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.018+0000 7f6abaffd640 1 -- 172.21.15.69:0/3692518672 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6abc1044c0 msgr2=0x7f6abc1a0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:08.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.018+0000 7f6abaffd640 1 --2- 172.21.15.69:0/3692518672 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6abc1044c0 0x7f6abc1a0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:08.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.018+0000 7f6abaffd640 1 -- 172.21.15.69:0/3692518672 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6abc1a0ba0 con 0x7f6abc102fa0 2024-04-25T11:26:08.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.018+0000 7f6abaffd640 1 --2- 172.21.15.69:0/3692518672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6abc102fa0 0x7f6abc199060 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f6aac005760 tx=0x7f6aac004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:26:08.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.018+0000 7f6aa3fff640 1 -- 172.21.15.69:0/3692518672 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aac03f030 con 0x7f6abc102fa0 2024-04-25T11:26:08.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.018+0000 7f6aa3fff640 1 -- 172.21.15.69:0/3692518672 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6aac004440 con 0x7f6abc102fa0 2024-04-25T11:26:08.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.018+0000 7f6ac12ca640 1 -- 172.21.15.69:0/3692518672 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6abc1a0e30 con 0x7f6abc102fa0 2024-04-25T11:26:08.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.018+0000 7f6aa3fff640 1 -- 172.21.15.69:0/3692518672 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aac035070 con 0x7f6abc102fa0 2024-04-25T11:26:08.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.018+0000 7f6ac12ca640 1 -- 172.21.15.69:0/3692518672 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6abc1a1240 con 0x7f6abc102fa0 2024-04-25T11:26:08.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.019+0000 7f6ac12ca640 1 -- 172.21.15.69:0/3692518672 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6abc107940 con 0x7f6abc102fa0 2024-04-25T11:26:08.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.020+0000 7f6aa3fff640 1 -- 172.21.15.69:0/3692518672 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) ==== 147520+0+0 (secure 0 0 0) 0x7f6aac042030 con 0x7f6abc102fa0 2024-04-25T11:26:08.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.021+0000 7f6aa3fff640 1 --2- 172.21.15.69:0/3692518672 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f6aa40afae0 0x7f6aa40b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:26:08.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.021+0000 7f6aba7fc640 1 --2- 172.21.15.69:0/3692518672 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f6aa40afae0 0x7f6aa40b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:26:08.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.021+0000 7f6aa3fff640 1 -- 172.21.15.69:0/3692518672 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f6aac0ff750 con 0x7f6abc102fa0 2024-04-25T11:26:08.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.021+0000 7f6aba7fc640 1 --2- 172.21.15.69:0/3692518672 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f6aa40afae0 0x7f6aa40b1ff0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6abc19a100 tx=0x7f6ab0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:26:08.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.025+0000 7f6aa3fff640 1 -- 172.21.15.69:0/3692518672 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6aac0c7910 con 0x7f6abc102fa0 2024-04-25T11:26:08.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:07 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2034103725' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T11:26:08.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.230+0000 7f6ac12ca640 1 -- 172.21.15.69:0/3692518672 --> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f6abc063140 con 0x7f6aa40afae0 2024-04-25T11:26:08.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:07 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/2034103725' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T11:26:08.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.251+0000 7f6aa3fff640 1 -- 172.21.15.69:0/3692518672 <== mgr.14152 v2:172.21.15.69:6800/4220065409 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f6abc063140 con 0x7f6aa40afae0 2024-04-25T11:26:08.243 INFO:teuthology.orchestra.run.smithi069.stdout:{} 2024-04-25T11:26:08.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.253+0000 7f6ac12ca640 1 -- 172.21.15.69:0/3692518672 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f6aa40afae0 msgr2=0x7f6aa40b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:08.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.253+0000 7f6ac12ca640 1 --2- 172.21.15.69:0/3692518672 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f6aa40afae0 0x7f6aa40b1ff0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6abc19a100 tx=0x7f6ab0005f90 comp rx=0 tx=0).stop 2024-04-25T11:26:08.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.253+0000 7f6ac12ca640 1 -- 172.21.15.69:0/3692518672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6abc102fa0 msgr2=0x7f6abc199060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:26:08.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.253+0000 7f6ac12ca640 1 --2- 172.21.15.69:0/3692518672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6abc102fa0 0x7f6abc199060 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f6aac005760 tx=0x7f6aac004040 comp rx=0 tx=0).stop 2024-04-25T11:26:08.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.253+0000 7f6ac12ca640 1 -- 172.21.15.69:0/3692518672 shutdown_connections 2024-04-25T11:26:08.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.253+0000 7f6ac12ca640 1 --2- 172.21.15.69:0/3692518672 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f6aa40afae0 0x7f6aa40b1ff0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:08.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.253+0000 7f6ac12ca640 1 --2- 172.21.15.69:0/3692518672 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6abc1044c0 0x7f6abc1a0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:08.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.253+0000 7f6ac12ca640 1 --2- 172.21.15.69:0/3692518672 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6abc103980 0x7f6abc1995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:08.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.253+0000 7f6ac12ca640 1 --2- 172.21.15.69:0/3692518672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6abc102fa0 0x7f6abc199060 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:26:08.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.253+0000 7f6ac12ca640 1 -- 172.21.15.69:0/3692518672 >> 172.21.15.69:0/3692518672 conn(0x7f6abc0fd3c0 msgr2=0x7f6abc104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:26:08.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.253+0000 7f6ac12ca640 1 -- 172.21.15.69:0/3692518672 shutdown_connections 2024-04-25T11:26:08.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:26:08.253+0000 7f6ac12ca640 1 -- 172.21.15.69:0/3692518672 wait complete. 2024-04-25T11:26:08.257 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-25T11:26:08.268 INFO:teuthology.run_tasks:Running task workunit... 2024-04-25T11:26:08.279 INFO:tasks.workunit:Pulling workunits from ref 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T11:26:08.279 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-25T11:26:08.279 INFO:tasks.workunit:timeout=3h 2024-04-25T11:26:08.279 INFO:tasks.workunit:cleanup=True 2024-04-25T11:26:08.280 DEBUG:teuthology.orchestra.run.smithi069:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-25T11:26:08.295 INFO:teuthology.orchestra.run.smithi069.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-25T11:26:08.295 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-04-25T11:26:08.295 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 73h/115d Inode: 1 Links: 2 2024-04-25T11:26:08.295 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:26:08.295 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:fusefs_t:s0 2024-04-25T11:26:08.295 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2024-04-25 11:24:08.630201637 +0000 2024-04-25T11:26:08.295 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2024-04-25 11:24:08.630201637 +0000 2024-04-25T11:26:08.295 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2024-04-25 11:26:06.777404639 +0000 2024-04-25T11:26:08.295 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: - 2024-04-25T11:26:08.295 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-25T11:26:08.296 DEBUG:teuthology.orchestra.run.smithi069:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-25T11:26:08.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:07 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/2034103725' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T11:26:08.364 DEBUG:teuthology.orchestra.run.smithi069:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T11:26:08.419 INFO:tasks.workunit.client.0.smithi069.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-25T11:26:08.692 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:26:08 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:26:08.234+0000 7f2672441700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T11:26:08.692 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:26:08 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:26:08.234+0000 7f2672441700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T11:26:08.692 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:26:08 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:26:08.234+0000 7f2672441700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T11:26:08.692 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:26:08 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:26:08.234+0000 7f2672441700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T11:26:08.692 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:26:08 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:26:08.234+0000 7f2672441700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T11:26:09.192 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:08 smithi069 ceph-mon[29188]: pgmap v362: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-25T11:26:09.192 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:08 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2034103725' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-25T11:26:09.192 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:08 smithi069 ceph-mon[29188]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T11:26:09.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:08 smithi152 ceph-mon[32701]: pgmap v362: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-25T11:26:09.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:08 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/2034103725' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-25T11:26:09.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:08 smithi152 ceph-mon[32701]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T11:26:09.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:08 smithi146 ceph-mon[32821]: pgmap v362: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-25T11:26:09.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:08 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/2034103725' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-25T11:26:09.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:08 smithi146 ceph-mon[32821]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T11:26:10.192 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:09 smithi069 ceph-mon[29188]: from='client.15528 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:26:10.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:09 smithi152 ceph-mon[32701]: from='client.15528 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:26:10.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:09 smithi146 ceph-mon[32821]: from='client.15528 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:26:11.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:10 smithi069 ceph-mon[29188]: pgmap v363: 129 pgs: 129 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.3 KiB/s wr, 14 op/s 2024-04-25T11:26:11.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:10 smithi152 ceph-mon[32701]: pgmap v363: 129 pgs: 129 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.3 KiB/s wr, 14 op/s 2024-04-25T11:26:11.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:10 smithi146 ceph-mon[32821]: pgmap v363: 129 pgs: 129 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.3 KiB/s wr, 14 op/s 2024-04-25T11:26:12.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:11 smithi069 ceph-mon[29188]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-25T11:26:12.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:11 smithi152 ceph-mon[32701]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-25T11:26:12.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:11 smithi146 ceph-mon[32821]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-25T11:26:13.192 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:12 smithi069 ceph-mon[29188]: pgmap v364: 129 pgs: 129 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.6 KiB/s wr, 12 op/s 2024-04-25T11:26:13.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:12 smithi152 ceph-mon[32701]: pgmap v364: 129 pgs: 129 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.6 KiB/s wr, 12 op/s 2024-04-25T11:26:13.249 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:12 smithi146 ceph-mon[32821]: pgmap v364: 129 pgs: 129 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.6 KiB/s wr, 12 op/s 2024-04-25T11:26:15.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:14 smithi069 ceph-mon[29188]: pgmap v365: 129 pgs: 129 active+clean; 659 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-04-25T11:26:15.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:14 smithi152 ceph-mon[32701]: pgmap v365: 129 pgs: 129 active+clean; 659 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-04-25T11:26:15.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:14 smithi146 ceph-mon[32821]: pgmap v365: 129 pgs: 129 active+clean; 659 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-04-25T11:26:17.192 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:16 smithi069 ceph-mon[29188]: pgmap v366: 129 pgs: 129 active+clean; 659 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.7 KiB/s wr, 14 op/s 2024-04-25T11:26:17.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:16 smithi152 ceph-mon[32701]: pgmap v366: 129 pgs: 129 active+clean; 659 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.7 KiB/s wr, 14 op/s 2024-04-25T11:26:17.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:16 smithi146 ceph-mon[32821]: pgmap v366: 129 pgs: 129 active+clean; 659 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.7 KiB/s wr, 14 op/s 2024-04-25T11:26:19.192 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:18 smithi069 ceph-mon[29188]: pgmap v367: 129 pgs: 129 active+clean; 659 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-04-25T11:26:19.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:18 smithi152 ceph-mon[32701]: pgmap v367: 129 pgs: 129 active+clean; 659 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-04-25T11:26:19.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:18 smithi146 ceph-mon[32821]: pgmap v367: 129 pgs: 129 active+clean; 659 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-04-25T11:26:21.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:20 smithi069 ceph-mon[29188]: pgmap v368: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 3.2 KiB/s wr, 14 op/s 2024-04-25T11:26:21.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:20 smithi152 ceph-mon[32701]: pgmap v368: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 3.2 KiB/s wr, 14 op/s 2024-04-25T11:26:21.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:20 smithi146 ceph-mon[32821]: pgmap v368: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 3.2 KiB/s wr, 14 op/s 2024-04-25T11:26:22.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:21 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:26:22.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:21 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:26:22.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:21 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:26:23.185 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:22 smithi069 ceph-mon[29188]: pgmap v369: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-04-25T11:26:23.186 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:22 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:26:23.186 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:22 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:26:23.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:22 smithi152 ceph-mon[32701]: pgmap v369: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-04-25T11:26:23.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:22 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:26:23.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:22 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:26:23.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:22 smithi146 ceph-mon[32821]: pgmap v369: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-04-25T11:26:23.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:22 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:26:23.348 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:22 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:26:24.598 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:24 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:26:24.598 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:24 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:26:24.598 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:24 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:26:24.598 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:24 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:26:24.598 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:24 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:26:24.598 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:24 smithi146 ceph-mon[32821]: pgmap v370: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 853 B/s wr, 9 op/s 2024-04-25T11:26:24.598 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:24 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:26:24.691 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:24 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:26:24.692 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:24 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:26:24.692 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:24 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:26:24.692 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:24 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:26:24.692 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:24 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:26:24.692 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:24 smithi069 ceph-mon[29188]: pgmap v370: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 853 B/s wr, 9 op/s 2024-04-25T11:26:24.692 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:24 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:26:24.729 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:24 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:26:24.729 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:24 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:26:24.729 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:24 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:26:24.729 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:24 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:26:24.729 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:24 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:26:24.729 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:24 smithi152 ceph-mon[32701]: pgmap v370: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 853 B/s wr, 9 op/s 2024-04-25T11:26:24.729 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:24 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:26:25.597 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:25 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:26:25.598 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:25 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:26:25.598 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:25 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:26:25.692 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:25 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:26:25.692 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:25 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:26:25.692 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:25 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:26:25.729 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:25 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:26:25.729 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:25 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:26:25.729 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:25 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:26:26.598 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:26 smithi146 ceph-mon[32821]: pgmap v371: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 767 B/s wr, 12 op/s 2024-04-25T11:26:26.691 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:26 smithi069 ceph-mon[29188]: pgmap v371: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 767 B/s wr, 12 op/s 2024-04-25T11:26:26.729 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:26 smithi152 ceph-mon[32701]: pgmap v371: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 767 B/s wr, 12 op/s 2024-04-25T11:26:29.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:28 smithi069 ceph-mon[29188]: pgmap v372: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-25T11:26:29.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:28 smithi152 ceph-mon[32701]: pgmap v372: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-25T11:26:29.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:28 smithi146 ceph-mon[32821]: pgmap v372: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-25T11:26:31.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:30 smithi069 ceph-mon[29188]: pgmap v373: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 767 B/s wr, 12 op/s 2024-04-25T11:26:31.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:30 smithi152 ceph-mon[32701]: pgmap v373: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 767 B/s wr, 12 op/s 2024-04-25T11:26:31.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:30 smithi146 ceph-mon[32821]: pgmap v373: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 767 B/s wr, 12 op/s 2024-04-25T11:26:33.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:32 smithi069 ceph-mon[29188]: pgmap v374: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:26:33.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:32 smithi152 ceph-mon[32701]: pgmap v374: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:26:33.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:32 smithi146 ceph-mon[32821]: pgmap v374: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:26:35.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:34 smithi069 ceph-mon[29188]: pgmap v375: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:26:35.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:34 smithi152 ceph-mon[32701]: pgmap v375: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:26:35.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:34 smithi146 ceph-mon[32821]: pgmap v375: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:26:37.194 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:36 smithi069 ceph-mon[29188]: pgmap v376: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:26:37.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:36 smithi152 ceph-mon[32701]: pgmap v376: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:26:37.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:36 smithi146 ceph-mon[32821]: pgmap v376: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:26:39.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:38 smithi069 ceph-mon[29188]: pgmap v377: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:26:39.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:38 smithi152 ceph-mon[32701]: pgmap v377: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:26:39.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:38 smithi146 ceph-mon[32821]: pgmap v377: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:26:41.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:40 smithi069 ceph-mon[29188]: pgmap v378: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:26:41.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:40 smithi152 ceph-mon[32701]: pgmap v378: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:26:41.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:40 smithi146 ceph-mon[32821]: pgmap v378: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:26:43.149 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:42 smithi146 ceph-mon[32821]: pgmap v379: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:26:43.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:42 smithi069 ceph-mon[29188]: pgmap v379: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:26:43.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:42 smithi152 ceph-mon[32701]: pgmap v379: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:26:45.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:44 smithi069 ceph-mon[29188]: pgmap v380: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:26:45.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:44 smithi152 ceph-mon[32701]: pgmap v380: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:26:45.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:44 smithi146 ceph-mon[32821]: pgmap v380: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:26:47.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:46 smithi069 ceph-mon[29188]: pgmap v381: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:26:47.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:46 smithi152 ceph-mon[32701]: pgmap v381: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:26:47.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:46 smithi146 ceph-mon[32821]: pgmap v381: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:26:49.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:48 smithi069 ceph-mon[29188]: pgmap v382: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:26:49.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:48 smithi152 ceph-mon[32701]: pgmap v382: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:26:49.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:48 smithi146 ceph-mon[32821]: pgmap v382: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:26:51.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:50 smithi069 ceph-mon[29188]: pgmap v383: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:26:51.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:50 smithi152 ceph-mon[32701]: pgmap v383: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:26:51.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:50 smithi146 ceph-mon[32821]: pgmap v383: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:26:53.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:52 smithi069 ceph-mon[29188]: pgmap v384: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:26:53.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:52 smithi152 ceph-mon[32701]: pgmap v384: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:26:53.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:52 smithi146 ceph-mon[32821]: pgmap v384: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:26:55.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:54 smithi069 ceph-mon[29188]: pgmap v385: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:26:55.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:54 smithi152 ceph-mon[32701]: pgmap v385: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:26:55.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:54 smithi146 ceph-mon[32821]: pgmap v385: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:26:57.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:56 smithi069 ceph-mon[29188]: pgmap v386: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:26:57.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:56 smithi152 ceph-mon[32701]: pgmap v386: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:26:57.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:56 smithi146 ceph-mon[32821]: pgmap v386: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:26:59.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:26:58 smithi069 ceph-mon[29188]: pgmap v387: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:26:59.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:26:58 smithi152 ceph-mon[32701]: pgmap v387: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:26:59.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:26:58 smithi146 ceph-mon[32821]: pgmap v387: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:27:01.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:27:00 smithi069 ceph-mon[29188]: pgmap v388: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:27:01.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:27:00 smithi152 ceph-mon[32701]: pgmap v388: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:27:01.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:27:00 smithi146 ceph-mon[32821]: pgmap v388: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:27:03.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:27:02 smithi069 ceph-mon[29188]: pgmap v389: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:27:03.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:27:02 smithi152 ceph-mon[32701]: pgmap v389: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:27:03.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:27:02 smithi146 ceph-mon[32821]: pgmap v389: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:27:05.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:27:04 smithi069 ceph-mon[29188]: pgmap v390: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:27:05.229 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:27:04 smithi152 ceph-mon[32701]: pgmap v390: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:27:05.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:27:04 smithi146 ceph-mon[32821]: pgmap v390: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:27:07.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:27:06 smithi069 ceph-mon[29188]: pgmap v391: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:27:07.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:27:06 smithi152 ceph-mon[32701]: pgmap v391: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:27:07.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:27:06 smithi146 ceph-mon[32821]: pgmap v391: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:27:08.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:27:07 smithi146 ceph-mon[32821]: pgmap v392: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:27:08.441 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:27:07 smithi069 ceph-mon[29188]: pgmap v392: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:27:08.478 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:27:07 smithi152 ceph-mon[32701]: pgmap v392: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:27:11.182 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:27:10 smithi152 ceph-mon[32701]: pgmap v393: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:27:11.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:27:10 smithi069 ceph-mon[29188]: pgmap v393: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:27:11.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:27:10 smithi146 ceph-mon[32821]: pgmap v393: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:27:13.149 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:27:12 smithi146 ceph-mon[32821]: pgmap v394: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:27:13.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:27:12 smithi069 ceph-mon[29188]: pgmap v394: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:27:13.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:27:12 smithi152 ceph-mon[32701]: pgmap v394: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:27:15.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:27:14 smithi069 ceph-mon[29188]: pgmap v395: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:27:15.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:27:14 smithi152 ceph-mon[32701]: pgmap v395: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:27:15.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:27:14 smithi146 ceph-mon[32821]: pgmap v395: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:27:17.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:27:16 smithi069 ceph-mon[29188]: pgmap v396: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:27:17.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:27:16 smithi152 ceph-mon[32701]: pgmap v396: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:27:17.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:27:16 smithi146 ceph-mon[32821]: pgmap v396: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:27:19.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:27:18 smithi069 ceph-mon[29188]: pgmap v397: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:27:19.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:27:18 smithi152 ceph-mon[32701]: pgmap v397: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:27:19.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:27:18 smithi146 ceph-mon[32821]: pgmap v397: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:27:21.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:27:20 smithi069 ceph-mon[29188]: pgmap v398: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:27:21.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:27:20 smithi152 ceph-mon[32701]: pgmap v398: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:27:21.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:27:20 smithi146 ceph-mon[32821]: pgmap v398: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:27:23.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:27:22 smithi069 ceph-mon[29188]: pgmap v399: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:27:23.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:27:22 smithi152 ceph-mon[32701]: pgmap v399: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:27:23.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:27:22 smithi146 ceph-mon[32821]: pgmap v399: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:27:25.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:27:24 smithi069 ceph-mon[29188]: pgmap v400: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:27:25.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:27:24 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:27:25.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:27:24 smithi152 ceph-mon[32701]: pgmap v400: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:27:25.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:27:24 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:27:25.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:27:24 smithi146 ceph-mon[32821]: pgmap v400: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:27:25.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:27:24 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:27:26.441 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:27:26 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:27:26.441 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:27:26 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:27:26.441 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:27:26 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:27:26.441 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:27:26 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:27:26.441 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:27:26 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:27:26.441 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:27:26 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:27:26.441 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:27:26 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:27:26.441 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:27:26 smithi069 ceph-mon[29188]: pgmap v401: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:27:26.478 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:27:26 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:27:26.478 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:27:26 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:27:26.478 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:27:26 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:27:26.478 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:27:26 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:27:26.479 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:27:26 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:27:26.479 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:27:26 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:27:26.479 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:27:26 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:27:26.479 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:27:26 smithi152 ceph-mon[32701]: pgmap v401: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:27:26.597 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:27:26 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:27:26.597 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:27:26 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:27:26.597 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:27:26 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:27:26.597 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:27:26 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:27:26.597 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:27:26 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:27:26.597 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:27:26 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:27:26.598 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:27:26 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:27:26.598 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:27:26 smithi146 ceph-mon[32821]: pgmap v401: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:27:29.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:27:28 smithi069 ceph-mon[29188]: pgmap v402: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:27:29.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:27:28 smithi152 ceph-mon[32701]: pgmap v402: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:27:29.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:27:28 smithi146 ceph-mon[32821]: pgmap v402: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:27:31.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:27:30 smithi069 ceph-mon[29188]: pgmap v403: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:27:31.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:27:30 smithi152 ceph-mon[32701]: pgmap v403: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:27:31.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:27:30 smithi146 ceph-mon[32821]: pgmap v403: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-25T11:27:31.759 INFO:tasks.workunit.client.0.smithi069.stderr:Note: switching to '994e2f20102a20fa9a27966c14b625cc01884df8'. 2024-04-25T11:27:31.759 INFO:tasks.workunit.client.0.smithi069.stderr: 2024-04-25T11:27:31.759 INFO:tasks.workunit.client.0.smithi069.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-25T11:27:31.760 INFO:tasks.workunit.client.0.smithi069.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-25T11:27:31.760 INFO:tasks.workunit.client.0.smithi069.stderr:state without impacting any branches by switching back to a branch. 2024-04-25T11:27:31.760 INFO:tasks.workunit.client.0.smithi069.stderr: 2024-04-25T11:27:31.760 INFO:tasks.workunit.client.0.smithi069.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-25T11:27:31.760 INFO:tasks.workunit.client.0.smithi069.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-25T11:27:31.760 INFO:tasks.workunit.client.0.smithi069.stderr: 2024-04-25T11:27:31.760 INFO:tasks.workunit.client.0.smithi069.stderr: git switch -c 2024-04-25T11:27:31.760 INFO:tasks.workunit.client.0.smithi069.stderr: 2024-04-25T11:27:31.760 INFO:tasks.workunit.client.0.smithi069.stderr:Or undo this operation with: 2024-04-25T11:27:31.760 INFO:tasks.workunit.client.0.smithi069.stderr: 2024-04-25T11:27:31.760 INFO:tasks.workunit.client.0.smithi069.stderr: git switch - 2024-04-25T11:27:31.760 INFO:tasks.workunit.client.0.smithi069.stderr: 2024-04-25T11:27:31.760 INFO:tasks.workunit.client.0.smithi069.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-25T11:27:31.760 INFO:tasks.workunit.client.0.smithi069.stderr: 2024-04-25T11:27:31.761 INFO:tasks.workunit.client.0.smithi069.stderr:HEAD is now at 994e2f20102 Merge PR #56810 into wip-pdonnell-testing-20240425.015853-debug 2024-04-25T11:27:31.766 DEBUG:teuthology.orchestra.run.smithi069:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-25T11:27:31.829 INFO:tasks.workunit.client.0.smithi069.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-25T11:27:31.831 INFO:tasks.workunit.client.0.smithi069.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T11:27:31.831 INFO:tasks.workunit.client.0.smithi069.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-25T11:27:32.430 INFO:tasks.workunit.client.0.smithi069.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-25T11:27:32.483 INFO:tasks.workunit.client.0.smithi069.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-25T11:27:32.523 INFO:tasks.workunit.client.0.smithi069.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T11:27:32.525 INFO:tasks.workunit.client.0.smithi069.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T11:27:32.525 INFO:tasks.workunit.client.0.smithi069.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-25T11:27:32.563 INFO:tasks.workunit.client.0.smithi069.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T11:27:32.566 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T11:27:32.566 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-25T11:27:32.622 INFO:tasks.workunit:Running workunits matching fs/test_o_trunc.sh on client.0... 2024-04-25T11:27:32.623 INFO:tasks.workunit:Running workunit fs/test_o_trunc.sh... 2024-04-25T11:27:32.624 DEBUG:teuthology.orchestra.run.smithi069:workunit test fs/test_o_trunc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=994e2f20102a20fa9a27966c14b625cc01884df8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-04-25T11:27:32.687 INFO:tasks.workunit.client.0.smithi069.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-04-25T11:27:32.688 INFO:tasks.workunit.client.0.smithi069.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs 2024-04-25T11:27:32.688 INFO:tasks.workunit.client.0.smithi069.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc trunc.foo 600 2024-04-25T11:27:33.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:27:32 smithi069 ceph-mon[29188]: pgmap v404: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:27:33.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:27:32 smithi152 ceph-mon[32701]: pgmap v404: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:27:33.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:27:32 smithi146 ceph-mon[32821]: pgmap v404: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:27:34.190 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:27:33 smithi069 ceph-mon[29188]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-04-25T11:27:34.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:27:33 smithi152 ceph-mon[32701]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-04-25T11:27:34.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:27:33 smithi146 ceph-mon[32821]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-04-25T11:27:35.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:27:34 smithi069 ceph-mon[29188]: pgmap v405: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T11:27:35.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:27:34 smithi152 ceph-mon[32701]: pgmap v405: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T11:27:35.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:27:34 smithi146 ceph-mon[32821]: pgmap v405: 129 pgs: 129 active+clean; 666 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T11:27:37.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:27:36 smithi069 ceph-mon[29188]: pgmap v406: 129 pgs: 129 active+clean; 685 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2024-04-25T11:27:37.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:27:36 smithi152 ceph-mon[32701]: pgmap v406: 129 pgs: 129 active+clean; 685 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2024-04-25T11:27:37.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:27:36 smithi146 ceph-mon[32821]: pgmap v406: 129 pgs: 129 active+clean; 685 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2024-04-25T11:27:39.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:27:38 smithi069 ceph-mon[29188]: pgmap v407: 129 pgs: 129 active+clean; 685 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.4 KiB/s wr, 10 op/s 2024-04-25T11:27:39.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:27:38 smithi152 ceph-mon[32701]: pgmap v407: 129 pgs: 129 active+clean; 685 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.4 KiB/s wr, 10 op/s 2024-04-25T11:27:39.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:27:38 smithi146 ceph-mon[32821]: pgmap v407: 129 pgs: 129 active+clean; 685 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.4 KiB/s wr, 10 op/s 2024-04-25T11:27:41.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:27:40 smithi069 ceph-mon[29188]: pgmap v408: 129 pgs: 129 active+clean; 738 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.5 KiB/s wr, 16 op/s 2024-04-25T11:27:41.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:27:40 smithi152 ceph-mon[32701]: pgmap v408: 129 pgs: 129 active+clean; 738 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.5 KiB/s wr, 16 op/s 2024-04-25T11:27:41.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:27:40 smithi146 ceph-mon[32821]: pgmap v408: 129 pgs: 129 active+clean; 738 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.5 KiB/s wr, 16 op/s 2024-04-25T11:27:43.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:27:42 smithi069 ceph-mon[29188]: pgmap v409: 129 pgs: 129 active+clean; 738 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.5 KiB/s wr, 13 op/s 2024-04-25T11:27:43.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:27:42 smithi152 ceph-mon[32701]: pgmap v409: 129 pgs: 129 active+clean; 738 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.5 KiB/s wr, 13 op/s 2024-04-25T11:27:43.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:27:42 smithi146 ceph-mon[32821]: pgmap v409: 129 pgs: 129 active+clean; 738 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.5 KiB/s wr, 13 op/s 2024-04-25T11:27:45.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:27:44 smithi069 ceph-mon[29188]: pgmap v410: 129 pgs: 129 active+clean; 738 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.9 KiB/s wr, 14 op/s 2024-04-25T11:27:45.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:27:44 smithi152 ceph-mon[32701]: pgmap v410: 129 pgs: 129 active+clean; 738 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.9 KiB/s wr, 14 op/s 2024-04-25T11:27:45.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:27:44 smithi146 ceph-mon[32821]: pgmap v410: 129 pgs: 129 active+clean; 738 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.9 KiB/s wr, 14 op/s 2024-04-25T11:27:47.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:27:46 smithi069 ceph-mon[29188]: pgmap v411: 129 pgs: 129 active+clean; 789 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-04-25T11:27:47.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:27:46 smithi152 ceph-mon[32701]: pgmap v411: 129 pgs: 129 active+clean; 789 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-04-25T11:27:47.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:27:46 smithi146 ceph-mon[32821]: pgmap v411: 129 pgs: 129 active+clean; 789 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-04-25T11:27:49.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:27:48 smithi069 ceph-mon[29188]: pgmap v412: 129 pgs: 129 active+clean; 789 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:27:49.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:27:48 smithi152 ceph-mon[32701]: pgmap v412: 129 pgs: 129 active+clean; 789 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:27:49.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:27:48 smithi146 ceph-mon[32821]: pgmap v412: 129 pgs: 129 active+clean; 789 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:27:50.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:27:49 smithi069 ceph-mon[29188]: pgmap v413: 129 pgs: 129 active+clean; 842 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:27:50.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:27:49 smithi152 ceph-mon[32701]: pgmap v413: 129 pgs: 129 active+clean; 842 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:27:50.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:27:49 smithi146 ceph-mon[32821]: pgmap v413: 129 pgs: 129 active+clean; 842 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:27:53.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:27:52 smithi069 ceph-mon[29188]: pgmap v414: 129 pgs: 129 active+clean; 842 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:27:53.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:27:52 smithi152 ceph-mon[32701]: pgmap v414: 129 pgs: 129 active+clean; 842 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:27:53.350 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:27:52 smithi146 ceph-mon[32821]: pgmap v414: 129 pgs: 129 active+clean; 842 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:27:55.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:27:54 smithi069 ceph-mon[29188]: pgmap v415: 129 pgs: 129 active+clean; 842 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:27:55.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:27:54 smithi152 ceph-mon[32701]: pgmap v415: 129 pgs: 129 active+clean; 842 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:27:55.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:27:54 smithi146 ceph-mon[32821]: pgmap v415: 129 pgs: 129 active+clean; 842 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:27:57.190 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:27:56 smithi069 ceph-mon[29188]: pgmap v416: 129 pgs: 129 active+clean; 888 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-04-25T11:27:57.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:27:56 smithi152 ceph-mon[32701]: pgmap v416: 129 pgs: 129 active+clean; 888 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-04-25T11:27:57.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:27:56 smithi146 ceph-mon[32821]: pgmap v416: 129 pgs: 129 active+clean; 888 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-04-25T11:27:59.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:27:58 smithi069 ceph-mon[29188]: pgmap v417: 129 pgs: 129 active+clean; 888 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.6 KiB/s wr, 13 op/s 2024-04-25T11:27:59.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:27:58 smithi152 ceph-mon[32701]: pgmap v417: 129 pgs: 129 active+clean; 888 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.6 KiB/s wr, 13 op/s 2024-04-25T11:27:59.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:27:58 smithi146 ceph-mon[32821]: pgmap v417: 129 pgs: 129 active+clean; 888 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.6 KiB/s wr, 13 op/s 2024-04-25T11:28:01.190 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:28:00 smithi069 ceph-mon[29188]: pgmap v418: 129 pgs: 129 active+clean; 941 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:28:01.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:28:00 smithi152 ceph-mon[32701]: pgmap v418: 129 pgs: 129 active+clean; 941 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:28:01.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:28:00 smithi146 ceph-mon[32821]: pgmap v418: 129 pgs: 129 active+clean; 941 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:28:03.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:28:02 smithi069 ceph-mon[29188]: pgmap v419: 129 pgs: 129 active+clean; 941 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.5 KiB/s wr, 14 op/s 2024-04-25T11:28:03.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:28:02 smithi152 ceph-mon[32701]: pgmap v419: 129 pgs: 129 active+clean; 941 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.5 KiB/s wr, 14 op/s 2024-04-25T11:28:03.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:28:02 smithi146 ceph-mon[32821]: pgmap v419: 129 pgs: 129 active+clean; 941 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.5 KiB/s wr, 14 op/s 2024-04-25T11:28:05.190 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:28:04 smithi069 ceph-mon[29188]: pgmap v420: 129 pgs: 129 active+clean; 941 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2024-04-25T11:28:05.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:28:04 smithi152 ceph-mon[32701]: pgmap v420: 129 pgs: 129 active+clean; 941 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2024-04-25T11:28:05.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:28:04 smithi146 ceph-mon[32821]: pgmap v420: 129 pgs: 129 active+clean; 941 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2024-04-25T11:28:07.190 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:28:06 smithi069 ceph-mon[29188]: pgmap v421: 129 pgs: 129 active+clean; 998 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-04-25T11:28:07.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:28:06 smithi152 ceph-mon[32701]: pgmap v421: 129 pgs: 129 active+clean; 998 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-04-25T11:28:07.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:28:06 smithi146 ceph-mon[32821]: pgmap v421: 129 pgs: 129 active+clean; 998 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-04-25T11:28:09.190 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:28:08 smithi069 ceph-mon[29188]: pgmap v422: 129 pgs: 129 active+clean; 998 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-04-25T11:28:09.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:28:08 smithi152 ceph-mon[32701]: pgmap v422: 129 pgs: 129 active+clean; 998 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-04-25T11:28:09.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:28:08 smithi146 ceph-mon[32821]: pgmap v422: 129 pgs: 129 active+clean; 998 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-04-25T11:28:11.190 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:28:10 smithi069 ceph-mon[29188]: pgmap v423: 129 pgs: 129 active+clean; 1.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:28:11.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:28:10 smithi152 ceph-mon[32701]: pgmap v423: 129 pgs: 129 active+clean; 1.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:28:11.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:28:10 smithi146 ceph-mon[32821]: pgmap v423: 129 pgs: 129 active+clean; 1.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:28:13.190 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:28:12 smithi069 ceph-mon[29188]: pgmap v424: 129 pgs: 129 active+clean; 1.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:28:13.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:28:12 smithi152 ceph-mon[32701]: pgmap v424: 129 pgs: 129 active+clean; 1.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:28:13.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:28:12 smithi146 ceph-mon[32821]: pgmap v424: 129 pgs: 129 active+clean; 1.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:28:15.190 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:28:14 smithi069 ceph-mon[29188]: pgmap v425: 129 pgs: 129 active+clean; 1.0 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:28:15.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:28:14 smithi152 ceph-mon[32701]: pgmap v425: 129 pgs: 129 active+clean; 1.0 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:28:15.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:28:14 smithi146 ceph-mon[32821]: pgmap v425: 129 pgs: 129 active+clean; 1.0 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:28:17.190 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:28:16 smithi069 ceph-mon[29188]: pgmap v426: 129 pgs: 129 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 KiB/s wr, 18 op/s 2024-04-25T11:28:17.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:28:16 smithi152 ceph-mon[32701]: pgmap v426: 129 pgs: 129 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 KiB/s wr, 18 op/s 2024-04-25T11:28:17.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:28:16 smithi146 ceph-mon[32821]: pgmap v426: 129 pgs: 129 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 KiB/s wr, 18 op/s 2024-04-25T11:28:18.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:28:17 smithi152 ceph-mon[32701]: pgmap v427: 129 pgs: 129 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-04-25T11:28:18.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:28:17 smithi146 ceph-mon[32821]: pgmap v427: 129 pgs: 129 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-04-25T11:28:18.440 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:28:17 smithi069 ceph-mon[29188]: pgmap v427: 129 pgs: 129 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-04-25T11:28:21.190 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:28:20 smithi069 ceph-mon[29188]: pgmap v428: 129 pgs: 129 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:28:21.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:28:20 smithi152 ceph-mon[32701]: pgmap v428: 129 pgs: 129 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:28:21.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:28:20 smithi146 ceph-mon[32821]: pgmap v428: 129 pgs: 129 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:28:23.190 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:28:22 smithi069 ceph-mon[29188]: pgmap v429: 129 pgs: 129 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:28:23.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:28:22 smithi152 ceph-mon[32701]: pgmap v429: 129 pgs: 129 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:28:23.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:28:22 smithi146 ceph-mon[32821]: pgmap v429: 129 pgs: 129 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:28:25.190 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:28:24 smithi069 ceph-mon[29188]: pgmap v430: 129 pgs: 129 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:28:25.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:28:24 smithi152 ceph-mon[32701]: pgmap v430: 129 pgs: 129 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:28:25.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:28:24 smithi146 ceph-mon[32821]: pgmap v430: 129 pgs: 129 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:28:27.190 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:28:26 smithi069 ceph-mon[29188]: pgmap v431: 129 pgs: 129 active+clean; 1.2 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 18 op/s 2024-04-25T11:28:27.190 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:28:26 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:28:27.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:28:26 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:28:27.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:28:26 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:28:27.191 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:28:26 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:28:27.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:28:26 smithi152 ceph-mon[32701]: pgmap v431: 129 pgs: 129 active+clean; 1.2 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 18 op/s 2024-04-25T11:28:27.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:28:26 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:28:27.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:28:26 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:28:27.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:28:26 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:28:27.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:28:26 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:28:27.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:28:26 smithi146 ceph-mon[32821]: pgmap v431: 129 pgs: 129 active+clean; 1.2 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 18 op/s 2024-04-25T11:28:27.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:28:26 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:28:27.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:28:26 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:28:27.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:28:26 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:28:27.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:28:26 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:28:29.190 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:28:28 smithi069 ceph-mon[29188]: pgmap v432: 129 pgs: 129 active+clean; 1.2 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:28:29.227 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:28:28 smithi152 ceph-mon[32701]: pgmap v432: 129 pgs: 129 active+clean; 1.2 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:28:29.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:28:28 smithi146 ceph-mon[32821]: pgmap v432: 129 pgs: 129 active+clean; 1.2 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:28:31.190 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:28:30 smithi069 ceph-mon[29188]: pgmap v433: 129 pgs: 129 active+clean; 1.2 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:28:31.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:28:30 smithi152 ceph-mon[32701]: pgmap v433: 129 pgs: 129 active+clean; 1.2 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:28:31.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:28:30 smithi146 ceph-mon[32821]: pgmap v433: 129 pgs: 129 active+clean; 1.2 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:28:33.190 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:28:32 smithi069 ceph-mon[29188]: pgmap v434: 129 pgs: 129 active+clean; 1.2 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:28:33.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:28:32 smithi152 ceph-mon[32701]: pgmap v434: 129 pgs: 129 active+clean; 1.2 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:28:33.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:28:32 smithi146 ceph-mon[32821]: pgmap v434: 129 pgs: 129 active+clean; 1.2 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:28:35.190 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:28:34 smithi069 ceph-mon[29188]: pgmap v435: 129 pgs: 129 active+clean; 1.2 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T11:28:35.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:28:34 smithi152 ceph-mon[32701]: pgmap v435: 129 pgs: 129 active+clean; 1.2 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T11:28:35.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:28:34 smithi146 ceph-mon[32821]: pgmap v435: 129 pgs: 129 active+clean; 1.2 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T11:28:36.277 INFO:tasks.workunit.client.0.smithi069.stdout:1/600: open fd = 3 2024-04-25T11:28:36.277 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.277 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.277 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.277 INFO:tasks.workunit.client.0.smithi069.stdout:2/600: open fd = 3 2024-04-25T11:28:36.277 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.278 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.278 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.278 INFO:tasks.workunit.client.0.smithi069.stdout:3/600: open fd = 3 2024-04-25T11:28:36.278 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.278 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.278 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.278 INFO:tasks.workunit.client.0.smithi069.stdout:4/600: open fd = 3 2024-04-25T11:28:36.278 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.278 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.278 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.278 INFO:tasks.workunit.client.0.smithi069.stdout:5/600: open fd = 3 2024-04-25T11:28:36.278 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.278 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.278 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.279 INFO:tasks.workunit.client.0.smithi069.stdout:6/600: open fd = 3 2024-04-25T11:28:36.279 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.279 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.279 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.279 INFO:tasks.workunit.client.0.smithi069.stdout:7/600: open fd = 3 2024-04-25T11:28:36.279 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.279 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.279 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.279 INFO:tasks.workunit.client.0.smithi069.stdout:8/600: open fd = 3 2024-04-25T11:28:36.279 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.279 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.279 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.279 INFO:tasks.workunit.client.0.smithi069.stdout:9/600: open fd = 3 2024-04-25T11:28:36.279 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.280 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.280 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.280 INFO:tasks.workunit.client.0.smithi069.stdout:10/600: open fd = 3 2024-04-25T11:28:36.280 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.280 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.280 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.280 INFO:tasks.workunit.client.0.smithi069.stdout:11/600: open fd = 3 2024-04-25T11:28:36.280 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.280 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.280 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.280 INFO:tasks.workunit.client.0.smithi069.stdout:12/600: open fd = 3 2024-04-25T11:28:36.280 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.280 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.280 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.281 INFO:tasks.workunit.client.0.smithi069.stdout:13/600: open fd = 3 2024-04-25T11:28:36.281 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.281 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.281 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.281 INFO:tasks.workunit.client.0.smithi069.stdout:14/600: open fd = 3 2024-04-25T11:28:36.281 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.281 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.281 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.281 INFO:tasks.workunit.client.0.smithi069.stdout:15/600: open fd = 3 2024-04-25T11:28:36.281 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.281 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.281 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.281 INFO:tasks.workunit.client.0.smithi069.stdout:16/600: open fd = 3 2024-04-25T11:28:36.281 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.281 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.281 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.282 INFO:tasks.workunit.client.0.smithi069.stdout:17/600: open fd = 3 2024-04-25T11:28:36.282 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.282 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.282 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.282 INFO:tasks.workunit.client.0.smithi069.stdout:18/600: open fd = 3 2024-04-25T11:28:36.282 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.282 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.282 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.282 INFO:tasks.workunit.client.0.smithi069.stdout:19/600: open fd = 3 2024-04-25T11:28:36.282 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.282 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.282 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.282 INFO:tasks.workunit.client.0.smithi069.stdout:20/600: open fd = 3 2024-04-25T11:28:36.283 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.283 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.283 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.283 INFO:tasks.workunit.client.0.smithi069.stdout:21/600: open fd = 3 2024-04-25T11:28:36.283 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.283 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.283 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.283 INFO:tasks.workunit.client.0.smithi069.stdout:22/600: open fd = 3 2024-04-25T11:28:36.283 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.283 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.283 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.283 INFO:tasks.workunit.client.0.smithi069.stdout:23/600: open fd = 3 2024-04-25T11:28:36.283 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.283 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.284 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.284 INFO:tasks.workunit.client.0.smithi069.stdout:24/600: open fd = 3 2024-04-25T11:28:36.284 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.284 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.284 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.284 INFO:tasks.workunit.client.0.smithi069.stdout:25/600: open fd = 3 2024-04-25T11:28:36.284 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.284 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.284 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.284 INFO:tasks.workunit.client.0.smithi069.stdout:26/600: open fd = 3 2024-04-25T11:28:36.284 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.284 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.285 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.285 INFO:tasks.workunit.client.0.smithi069.stdout:27/600: open fd = 3 2024-04-25T11:28:36.285 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.285 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.285 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.285 INFO:tasks.workunit.client.0.smithi069.stdout:28/600: open fd = 3 2024-04-25T11:28:36.285 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.285 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.285 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.285 INFO:tasks.workunit.client.0.smithi069.stdout:29/600: open fd = 3 2024-04-25T11:28:36.285 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.285 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.285 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.285 INFO:tasks.workunit.client.0.smithi069.stdout:30/600: open fd = 3 2024-04-25T11:28:36.285 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.286 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.286 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.286 INFO:tasks.workunit.client.0.smithi069.stdout:31/600: open fd = 3 2024-04-25T11:28:36.286 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.286 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.286 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.286 INFO:tasks.workunit.client.0.smithi069.stdout:32/600: open fd = 3 2024-04-25T11:28:36.286 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.286 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.286 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.286 INFO:tasks.workunit.client.0.smithi069.stdout:33/600: open fd = 3 2024-04-25T11:28:36.286 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.286 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.286 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.287 INFO:tasks.workunit.client.0.smithi069.stdout:34/600: open fd = 3 2024-04-25T11:28:36.287 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.287 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.287 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.287 INFO:tasks.workunit.client.0.smithi069.stdout:35/600: open fd = 3 2024-04-25T11:28:36.287 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.287 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.287 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.287 INFO:tasks.workunit.client.0.smithi069.stdout:36/600: open fd = 3 2024-04-25T11:28:36.287 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.287 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.287 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.287 INFO:tasks.workunit.client.0.smithi069.stdout:37/600: open fd = 3 2024-04-25T11:28:36.287 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.287 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.288 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.288 INFO:tasks.workunit.client.0.smithi069.stdout:38/600: open fd = 3 2024-04-25T11:28:36.288 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.288 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.288 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.288 INFO:tasks.workunit.client.0.smithi069.stdout:39/600: open fd = 3 2024-04-25T11:28:36.288 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.288 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.288 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.288 INFO:tasks.workunit.client.0.smithi069.stdout:40/600: open fd = 3 2024-04-25T11:28:36.288 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.288 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.288 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.288 INFO:tasks.workunit.client.0.smithi069.stdout:41/600: open fd = 3 2024-04-25T11:28:36.288 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.289 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.289 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.289 INFO:tasks.workunit.client.0.smithi069.stdout:42/600: open fd = 3 2024-04-25T11:28:36.289 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.289 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.289 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.289 INFO:tasks.workunit.client.0.smithi069.stdout:43/600: open fd = 3 2024-04-25T11:28:36.289 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.289 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.289 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.289 INFO:tasks.workunit.client.0.smithi069.stdout:44/600: open fd = 3 2024-04-25T11:28:36.289 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.289 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.289 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.289 INFO:tasks.workunit.client.0.smithi069.stdout:45/600: open fd = 3 2024-04-25T11:28:36.290 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.290 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.290 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.290 INFO:tasks.workunit.client.0.smithi069.stdout:46/600: open fd = 3 2024-04-25T11:28:36.290 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.290 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.290 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.290 INFO:tasks.workunit.client.0.smithi069.stdout:47/600: open fd = 3 2024-04-25T11:28:36.290 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.290 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.290 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.290 INFO:tasks.workunit.client.0.smithi069.stdout:48/600: open fd = 3 2024-04-25T11:28:36.290 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.290 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.290 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.291 INFO:tasks.workunit.client.0.smithi069.stdout:49/600: open fd = 3 2024-04-25T11:28:36.291 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.291 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.291 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.291 INFO:tasks.workunit.client.0.smithi069.stdout:50/600: open fd = 3 2024-04-25T11:28:36.291 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.291 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.291 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.291 INFO:tasks.workunit.client.0.smithi069.stdout:51/600: open fd = 3 2024-04-25T11:28:36.291 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.291 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.291 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.291 INFO:tasks.workunit.client.0.smithi069.stdout:52/600: open fd = 3 2024-04-25T11:28:36.291 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.292 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.292 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.292 INFO:tasks.workunit.client.0.smithi069.stdout:53/600: open fd = 3 2024-04-25T11:28:36.292 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.292 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.292 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.292 INFO:tasks.workunit.client.0.smithi069.stdout:54/600: open fd = 3 2024-04-25T11:28:36.292 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.292 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.292 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.292 INFO:tasks.workunit.client.0.smithi069.stdout:55/600: open fd = 3 2024-04-25T11:28:36.292 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.292 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.292 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.292 INFO:tasks.workunit.client.0.smithi069.stdout:56/600: open fd = 3 2024-04-25T11:28:36.293 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.293 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.293 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.293 INFO:tasks.workunit.client.0.smithi069.stdout:57/600: open fd = 3 2024-04-25T11:28:36.293 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.293 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.293 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.293 INFO:tasks.workunit.client.0.smithi069.stdout:58/600: open fd = 3 2024-04-25T11:28:36.293 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.293 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.293 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.293 INFO:tasks.workunit.client.0.smithi069.stdout:59/600: open fd = 3 2024-04-25T11:28:36.293 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.293 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.293 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.294 INFO:tasks.workunit.client.0.smithi069.stdout:60/600: open fd = 3 2024-04-25T11:28:36.294 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.294 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.294 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.294 INFO:tasks.workunit.client.0.smithi069.stdout:61/600: open fd = 3 2024-04-25T11:28:36.294 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.294 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.294 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.294 INFO:tasks.workunit.client.0.smithi069.stdout:62/600: open fd = 3 2024-04-25T11:28:36.294 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.294 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.294 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:36.294 INFO:tasks.workunit.client.0.smithi069.stdout:63/600: open fd = 3 2024-04-25T11:28:36.294 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.295 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:28:36.295 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:28:37.190 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:28:36 smithi069 ceph-mon[29188]: pgmap v436: 129 pgs: 129 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:28:37.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:28:36 smithi152 ceph-mon[32701]: pgmap v436: 129 pgs: 129 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:28:37.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:28:36 smithi146 ceph-mon[32821]: pgmap v436: 129 pgs: 129 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:28:39.190 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:28:38 smithi069 ceph-mon[29188]: pgmap v437: 129 pgs: 129 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:28:39.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:28:38 smithi152 ceph-mon[32701]: pgmap v437: 129 pgs: 129 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:28:39.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:28:38 smithi146 ceph-mon[32821]: pgmap v437: 129 pgs: 129 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:28:41.227 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:28:40 smithi152 ceph-mon[32701]: pgmap v438: 129 pgs: 129 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:28:41.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:28:40 smithi146 ceph-mon[32821]: pgmap v438: 129 pgs: 129 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:28:41.440 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:28:40 smithi069 ceph-mon[29188]: pgmap v438: 129 pgs: 129 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:28:42.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:28:41 smithi152 ceph-mon[32701]: pgmap v439: 129 pgs: 129 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:28:42.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:28:41 smithi146 ceph-mon[32821]: pgmap v439: 129 pgs: 129 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:28:42.440 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:28:41 smithi069 ceph-mon[29188]: pgmap v439: 129 pgs: 129 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:28:45.190 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:28:44 smithi069 ceph-mon[29188]: pgmap v440: 129 pgs: 129 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T11:28:45.227 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:28:44 smithi152 ceph-mon[32701]: pgmap v440: 129 pgs: 129 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T11:28:45.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:28:44 smithi146 ceph-mon[32821]: pgmap v440: 129 pgs: 129 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T11:28:47.190 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:28:46 smithi069 ceph-mon[29188]: pgmap v441: 129 pgs: 129 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:28:47.227 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:28:46 smithi152 ceph-mon[32701]: pgmap v441: 129 pgs: 129 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:28:47.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:28:46 smithi146 ceph-mon[32821]: pgmap v441: 129 pgs: 129 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:28:49.190 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:28:48 smithi069 ceph-mon[29188]: pgmap v442: 129 pgs: 129 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:28:49.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:28:48 smithi152 ceph-mon[32701]: pgmap v442: 129 pgs: 129 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:28:49.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:28:48 smithi146 ceph-mon[32821]: pgmap v442: 129 pgs: 129 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:28:51.190 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:28:50 smithi069 ceph-mon[29188]: pgmap v443: 129 pgs: 129 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:28:51.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:28:50 smithi152 ceph-mon[32701]: pgmap v443: 129 pgs: 129 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:28:51.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:28:50 smithi146 ceph-mon[32821]: pgmap v443: 129 pgs: 129 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:28:53.190 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:28:52 smithi069 ceph-mon[29188]: pgmap v444: 129 pgs: 129 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:28:53.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:28:52 smithi152 ceph-mon[32701]: pgmap v444: 129 pgs: 129 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:28:53.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:28:52 smithi146 ceph-mon[32821]: pgmap v444: 129 pgs: 129 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:28:55.190 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:28:54 smithi069 ceph-mon[29188]: pgmap v445: 129 pgs: 129 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:28:55.227 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:28:54 smithi152 ceph-mon[32701]: pgmap v445: 129 pgs: 129 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:28:55.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:28:54 smithi146 ceph-mon[32821]: pgmap v445: 129 pgs: 129 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:28:57.190 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:28:56 smithi069 ceph-mon[29188]: pgmap v446: 129 pgs: 129 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:28:57.227 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:28:56 smithi152 ceph-mon[32701]: pgmap v446: 129 pgs: 129 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:28:57.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:28:56 smithi146 ceph-mon[32821]: pgmap v446: 129 pgs: 129 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:28:59.228 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:28:58 smithi152 ceph-mon[32701]: pgmap v447: 129 pgs: 129 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:28:59.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:28:58 smithi146 ceph-mon[32821]: pgmap v447: 129 pgs: 129 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:28:59.440 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:28:58 smithi069 ceph-mon[29188]: pgmap v447: 129 pgs: 129 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:29:01.227 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:29:00 smithi152 ceph-mon[32701]: pgmap v448: 129 pgs: 129 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:29:01.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:29:00 smithi146 ceph-mon[32821]: pgmap v448: 129 pgs: 129 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:29:01.440 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:29:00 smithi069 ceph-mon[29188]: pgmap v448: 129 pgs: 129 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:29:02.227 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:29:01 smithi152 ceph-mon[32701]: pgmap v449: 129 pgs: 129 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:29:02.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:29:01 smithi146 ceph-mon[32821]: pgmap v449: 129 pgs: 129 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:29:02.440 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:29:01 smithi069 ceph-mon[29188]: pgmap v449: 129 pgs: 129 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:29:05.190 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:29:04 smithi069 ceph-mon[29188]: pgmap v450: 129 pgs: 129 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:29:05.227 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:29:04 smithi152 ceph-mon[32701]: pgmap v450: 129 pgs: 129 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:29:05.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:29:04 smithi146 ceph-mon[32821]: pgmap v450: 129 pgs: 129 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:29:07.190 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:29:06 smithi069 ceph-mon[29188]: pgmap v451: 129 pgs: 129 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:29:07.227 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:29:06 smithi152 ceph-mon[32701]: pgmap v451: 129 pgs: 129 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:29:07.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:29:06 smithi146 ceph-mon[32821]: pgmap v451: 129 pgs: 129 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:29:09.190 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:29:08 smithi069 ceph-mon[29188]: pgmap v452: 129 pgs: 129 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:29:09.227 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:29:08 smithi152 ceph-mon[32701]: pgmap v452: 129 pgs: 129 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:29:09.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:29:08 smithi146 ceph-mon[32821]: pgmap v452: 129 pgs: 129 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:29:11.190 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:29:10 smithi069 ceph-mon[29188]: pgmap v453: 129 pgs: 129 active+clean; 1.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:29:11.227 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:29:10 smithi152 ceph-mon[32701]: pgmap v453: 129 pgs: 129 active+clean; 1.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:29:11.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:29:10 smithi146 ceph-mon[32821]: pgmap v453: 129 pgs: 129 active+clean; 1.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:29:13.190 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:29:12 smithi069 ceph-mon[29188]: pgmap v454: 129 pgs: 129 active+clean; 1.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:29:13.227 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:29:12 smithi152 ceph-mon[32701]: pgmap v454: 129 pgs: 129 active+clean; 1.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:29:13.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:29:12 smithi146 ceph-mon[32821]: pgmap v454: 129 pgs: 129 active+clean; 1.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:29:15.227 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:29:14 smithi152 ceph-mon[32701]: pgmap v455: 129 pgs: 129 active+clean; 1.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:29:15.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:29:14 smithi146 ceph-mon[32821]: pgmap v455: 129 pgs: 129 active+clean; 1.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:29:15.440 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:29:14 smithi069 ceph-mon[29188]: pgmap v455: 129 pgs: 129 active+clean; 1.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:29:17.227 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:29:16 smithi152 ceph-mon[32701]: pgmap v456: 129 pgs: 129 active+clean; 1.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:29:17.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:29:16 smithi146 ceph-mon[32821]: pgmap v456: 129 pgs: 129 active+clean; 1.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:29:17.440 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:29:16 smithi069 ceph-mon[29188]: pgmap v456: 129 pgs: 129 active+clean; 1.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:29:18.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:29:18 smithi146 ceph-mon[32821]: pgmap v457: 129 pgs: 129 active+clean; 1.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:29:18.440 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:29:18 smithi069 ceph-mon[29188]: pgmap v457: 129 pgs: 129 active+clean; 1.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:29:18.477 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:29:18 smithi152 ceph-mon[32701]: pgmap v457: 129 pgs: 129 active+clean; 1.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:29:21.190 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:29:20 smithi069 ceph-mon[29188]: pgmap v458: 129 pgs: 129 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:29:21.227 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:29:20 smithi152 ceph-mon[32701]: pgmap v458: 129 pgs: 129 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:29:21.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:29:20 smithi146 ceph-mon[32821]: pgmap v458: 129 pgs: 129 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:29:23.190 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:29:22 smithi069 ceph-mon[29188]: pgmap v459: 129 pgs: 129 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:29:23.227 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:29:22 smithi152 ceph-mon[32701]: pgmap v459: 129 pgs: 129 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:29:23.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:29:22 smithi146 ceph-mon[32821]: pgmap v459: 129 pgs: 129 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:29:25.190 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:29:24 smithi069 ceph-mon[29188]: pgmap v460: 129 pgs: 129 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:29:25.227 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:29:24 smithi152 ceph-mon[32701]: pgmap v460: 129 pgs: 129 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:29:25.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:29:24 smithi146 ceph-mon[32821]: pgmap v460: 129 pgs: 129 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:29:27.227 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:29:26 smithi152 ceph-mon[32701]: pgmap v461: 129 pgs: 129 active+clean; 1.8 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:29:27.227 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:29:26 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:29:27.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:29:26 smithi146 ceph-mon[32821]: pgmap v461: 129 pgs: 129 active+clean; 1.8 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:29:27.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:29:26 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:29:27.440 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:29:26 smithi069 ceph-mon[29188]: pgmap v461: 129 pgs: 129 active+clean; 1.8 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:29:27.440 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:29:26 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:29:28.227 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:29:27 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:29:28.227 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:29:27 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:29:28.227 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:29:27 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:29:28.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:29:27 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:29:28.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:29:27 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:29:28.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:29:27 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:29:28.440 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:29:27 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:29:28.440 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:29:27 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:29:28.440 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:29:27 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:29:29.227 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:29:28 smithi152 ceph-mon[32701]: pgmap v462: 129 pgs: 129 active+clean; 1.8 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:29:29.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:29:28 smithi146 ceph-mon[32821]: pgmap v462: 129 pgs: 129 active+clean; 1.8 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:29:29.440 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:29:28 smithi069 ceph-mon[29188]: pgmap v462: 129 pgs: 129 active+clean; 1.8 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:29:31.227 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:29:30 smithi152 ceph-mon[32701]: pgmap v463: 129 pgs: 129 active+clean; 1.9 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:29:31.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:29:30 smithi146 ceph-mon[32821]: pgmap v463: 129 pgs: 129 active+clean; 1.9 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:29:31.440 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:29:30 smithi069 ceph-mon[29188]: pgmap v463: 129 pgs: 129 active+clean; 1.9 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:29:32.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:29:32 smithi146 ceph-mon[32821]: pgmap v464: 129 pgs: 129 active+clean; 1.9 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:29:32.440 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:29:32 smithi069 ceph-mon[29188]: pgmap v464: 129 pgs: 129 active+clean; 1.9 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:29:32.477 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:29:32 smithi152 ceph-mon[32701]: pgmap v464: 129 pgs: 129 active+clean; 1.9 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:29:35.190 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:29:34 smithi069 ceph-mon[29188]: pgmap v465: 129 pgs: 129 active+clean; 1.9 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:29:35.227 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:29:34 smithi152 ceph-mon[32701]: pgmap v465: 129 pgs: 129 active+clean; 1.9 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:29:35.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:29:34 smithi146 ceph-mon[32821]: pgmap v465: 129 pgs: 129 active+clean; 1.9 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:29:37.227 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:29:36 smithi152 ceph-mon[32701]: pgmap v466: 129 pgs: 129 active+clean; 1.9 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:29:37.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:29:36 smithi146 ceph-mon[32821]: pgmap v466: 129 pgs: 129 active+clean; 1.9 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:29:37.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:29:36 smithi069 ceph-mon[29188]: pgmap v466: 129 pgs: 129 active+clean; 1.9 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:29:39.227 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:29:38 smithi152 ceph-mon[32701]: pgmap v467: 129 pgs: 129 active+clean; 1.9 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:29:39.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:29:38 smithi146 ceph-mon[32821]: pgmap v467: 129 pgs: 129 active+clean; 1.9 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:29:39.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:29:38 smithi069 ceph-mon[29188]: pgmap v467: 129 pgs: 129 active+clean; 1.9 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:29:39.846 INFO:tasks.workunit.client.0.smithi069.stdout:64/600: open fd = 3 2024-04-25T11:29:39.847 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.847 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.847 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.847 INFO:tasks.workunit.client.0.smithi069.stdout:65/600: open fd = 3 2024-04-25T11:29:39.847 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.847 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.847 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.847 INFO:tasks.workunit.client.0.smithi069.stdout:66/600: open fd = 3 2024-04-25T11:29:39.847 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.847 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.847 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.848 INFO:tasks.workunit.client.0.smithi069.stdout:67/600: open fd = 3 2024-04-25T11:29:39.848 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.848 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.848 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.848 INFO:tasks.workunit.client.0.smithi069.stdout:68/600: open fd = 3 2024-04-25T11:29:39.848 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.848 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.848 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.848 INFO:tasks.workunit.client.0.smithi069.stdout:69/600: open fd = 3 2024-04-25T11:29:39.848 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.848 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.848 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.848 INFO:tasks.workunit.client.0.smithi069.stdout:70/600: open fd = 3 2024-04-25T11:29:39.848 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.849 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.849 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.849 INFO:tasks.workunit.client.0.smithi069.stdout:71/600: open fd = 3 2024-04-25T11:29:39.849 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.849 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.849 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.849 INFO:tasks.workunit.client.0.smithi069.stdout:72/600: open fd = 3 2024-04-25T11:29:39.849 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.849 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.849 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.849 INFO:tasks.workunit.client.0.smithi069.stdout:73/600: open fd = 3 2024-04-25T11:29:39.849 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.849 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.850 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.850 INFO:tasks.workunit.client.0.smithi069.stdout:74/600: open fd = 3 2024-04-25T11:29:39.850 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.850 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.850 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.850 INFO:tasks.workunit.client.0.smithi069.stdout:75/600: open fd = 3 2024-04-25T11:29:39.850 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.850 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.850 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.850 INFO:tasks.workunit.client.0.smithi069.stdout:76/600: open fd = 3 2024-04-25T11:29:39.850 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.850 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.850 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.850 INFO:tasks.workunit.client.0.smithi069.stdout:77/600: open fd = 3 2024-04-25T11:29:39.851 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.851 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.851 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.851 INFO:tasks.workunit.client.0.smithi069.stdout:78/600: open fd = 3 2024-04-25T11:29:39.851 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.851 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.851 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.851 INFO:tasks.workunit.client.0.smithi069.stdout:79/600: open fd = 3 2024-04-25T11:29:39.851 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.851 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.851 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.851 INFO:tasks.workunit.client.0.smithi069.stdout:80/600: open fd = 3 2024-04-25T11:29:39.851 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.851 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.851 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.852 INFO:tasks.workunit.client.0.smithi069.stdout:81/600: open fd = 3 2024-04-25T11:29:39.852 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.852 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.852 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.852 INFO:tasks.workunit.client.0.smithi069.stdout:82/600: open fd = 3 2024-04-25T11:29:39.852 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.852 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.852 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.852 INFO:tasks.workunit.client.0.smithi069.stdout:83/600: open fd = 3 2024-04-25T11:29:39.852 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.852 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.852 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.852 INFO:tasks.workunit.client.0.smithi069.stdout:84/600: open fd = 3 2024-04-25T11:29:39.852 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.852 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.853 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.853 INFO:tasks.workunit.client.0.smithi069.stdout:85/600: open fd = 3 2024-04-25T11:29:39.853 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.853 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.853 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.853 INFO:tasks.workunit.client.0.smithi069.stdout:86/600: open fd = 3 2024-04-25T11:29:39.853 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.853 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.853 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.853 INFO:tasks.workunit.client.0.smithi069.stdout:87/600: open fd = 3 2024-04-25T11:29:39.853 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.853 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.853 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.853 INFO:tasks.workunit.client.0.smithi069.stdout:88/600: open fd = 3 2024-04-25T11:29:39.853 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.854 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.854 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.854 INFO:tasks.workunit.client.0.smithi069.stdout:89/600: open fd = 3 2024-04-25T11:29:39.854 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.854 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.854 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.854 INFO:tasks.workunit.client.0.smithi069.stdout:90/600: open fd = 3 2024-04-25T11:29:39.854 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.854 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.854 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.854 INFO:tasks.workunit.client.0.smithi069.stdout:91/600: open fd = 3 2024-04-25T11:29:39.854 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.854 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.854 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.854 INFO:tasks.workunit.client.0.smithi069.stdout:92/600: open fd = 3 2024-04-25T11:29:39.855 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.855 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.855 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.855 INFO:tasks.workunit.client.0.smithi069.stdout:93/600: open fd = 3 2024-04-25T11:29:39.855 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.855 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.855 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.855 INFO:tasks.workunit.client.0.smithi069.stdout:94/600: open fd = 3 2024-04-25T11:29:39.855 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.855 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.855 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.855 INFO:tasks.workunit.client.0.smithi069.stdout:95/600: open fd = 3 2024-04-25T11:29:39.855 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.856 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.856 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.856 INFO:tasks.workunit.client.0.smithi069.stdout:96/600: open fd = 3 2024-04-25T11:29:39.856 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.856 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.856 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.856 INFO:tasks.workunit.client.0.smithi069.stdout:97/600: open fd = 3 2024-04-25T11:29:39.856 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.856 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.856 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.856 INFO:tasks.workunit.client.0.smithi069.stdout:98/600: open fd = 3 2024-04-25T11:29:39.856 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.856 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.856 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.856 INFO:tasks.workunit.client.0.smithi069.stdout:99/600: open fd = 3 2024-04-25T11:29:39.857 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.857 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.857 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.857 INFO:tasks.workunit.client.0.smithi069.stdout:100/600: open fd = 3 2024-04-25T11:29:39.857 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.857 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.857 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.857 INFO:tasks.workunit.client.0.smithi069.stdout:101/600: open fd = 3 2024-04-25T11:29:39.857 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.857 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.857 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.857 INFO:tasks.workunit.client.0.smithi069.stdout:102/600: open fd = 3 2024-04-25T11:29:39.857 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.857 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.857 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.858 INFO:tasks.workunit.client.0.smithi069.stdout:103/600: open fd = 3 2024-04-25T11:29:39.858 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.858 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.858 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.858 INFO:tasks.workunit.client.0.smithi069.stdout:104/600: open fd = 3 2024-04-25T11:29:39.858 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.858 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.858 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.858 INFO:tasks.workunit.client.0.smithi069.stdout:105/600: open fd = 3 2024-04-25T11:29:39.858 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.858 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.858 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.858 INFO:tasks.workunit.client.0.smithi069.stdout:106/600: open fd = 3 2024-04-25T11:29:39.858 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.859 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.859 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.859 INFO:tasks.workunit.client.0.smithi069.stdout:107/600: open fd = 3 2024-04-25T11:29:39.859 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.859 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.859 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.859 INFO:tasks.workunit.client.0.smithi069.stdout:108/600: open fd = 3 2024-04-25T11:29:39.859 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.859 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.859 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.859 INFO:tasks.workunit.client.0.smithi069.stdout:109/600: open fd = 3 2024-04-25T11:29:39.859 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.859 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.859 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.859 INFO:tasks.workunit.client.0.smithi069.stdout:110/600: open fd = 3 2024-04-25T11:29:39.860 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.860 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.860 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.860 INFO:tasks.workunit.client.0.smithi069.stdout:111/600: open fd = 3 2024-04-25T11:29:39.860 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.860 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.860 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.860 INFO:tasks.workunit.client.0.smithi069.stdout:112/600: open fd = 3 2024-04-25T11:29:39.860 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.860 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.860 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.860 INFO:tasks.workunit.client.0.smithi069.stdout:113/600: open fd = 3 2024-04-25T11:29:39.860 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.860 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.860 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.861 INFO:tasks.workunit.client.0.smithi069.stdout:114/600: open fd = 3 2024-04-25T11:29:39.861 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.861 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.861 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.861 INFO:tasks.workunit.client.0.smithi069.stdout:115/600: open fd = 3 2024-04-25T11:29:39.861 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.861 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.861 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.861 INFO:tasks.workunit.client.0.smithi069.stdout:116/600: open fd = 3 2024-04-25T11:29:39.861 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.861 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.861 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.861 INFO:tasks.workunit.client.0.smithi069.stdout:117/600: open fd = 3 2024-04-25T11:29:39.861 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.861 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.862 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.862 INFO:tasks.workunit.client.0.smithi069.stdout:118/600: open fd = 3 2024-04-25T11:29:39.862 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.862 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.862 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.862 INFO:tasks.workunit.client.0.smithi069.stdout:119/600: open fd = 3 2024-04-25T11:29:39.862 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.862 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.862 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.862 INFO:tasks.workunit.client.0.smithi069.stdout:120/600: open fd = 3 2024-04-25T11:29:39.862 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.862 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.862 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.862 INFO:tasks.workunit.client.0.smithi069.stdout:121/600: open fd = 3 2024-04-25T11:29:39.862 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.863 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.863 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.863 INFO:tasks.workunit.client.0.smithi069.stdout:122/600: open fd = 3 2024-04-25T11:29:39.863 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.863 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.863 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.863 INFO:tasks.workunit.client.0.smithi069.stdout:123/600: open fd = 3 2024-04-25T11:29:39.863 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.863 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.863 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.863 INFO:tasks.workunit.client.0.smithi069.stdout:124/600: open fd = 3 2024-04-25T11:29:39.863 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.863 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.863 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.864 INFO:tasks.workunit.client.0.smithi069.stdout:125/600: open fd = 3 2024-04-25T11:29:39.864 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.864 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:39.864 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:29:39.864 INFO:tasks.workunit.client.0.smithi069.stdout:126/600: open fd = 3 2024-04-25T11:29:39.864 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:29:41.227 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:29:40 smithi152 ceph-mon[32701]: pgmap v468: 129 pgs: 129 active+clean; 2.0 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:29:41.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:29:40 smithi146 ceph-mon[32821]: pgmap v468: 129 pgs: 129 active+clean; 2.0 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:29:41.440 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:29:40 smithi069 ceph-mon[29188]: pgmap v468: 129 pgs: 129 active+clean; 2.0 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:29:42.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:29:42 smithi146 ceph-mon[32821]: pgmap v469: 129 pgs: 129 active+clean; 2.0 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:29:42.440 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:29:42 smithi069 ceph-mon[29188]: pgmap v469: 129 pgs: 129 active+clean; 2.0 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:29:42.477 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:29:42 smithi152 ceph-mon[32701]: pgmap v469: 129 pgs: 129 active+clean; 2.0 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:29:45.227 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:29:44 smithi152 ceph-mon[32701]: pgmap v470: 129 pgs: 129 active+clean; 2.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 18 op/s 2024-04-25T11:29:45.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:29:44 smithi146 ceph-mon[32821]: pgmap v470: 129 pgs: 129 active+clean; 2.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 18 op/s 2024-04-25T11:29:45.440 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:29:44 smithi069 ceph-mon[29188]: pgmap v470: 129 pgs: 129 active+clean; 2.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 18 op/s 2024-04-25T11:29:47.227 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:29:46 smithi152 ceph-mon[32701]: pgmap v471: 129 pgs: 129 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.6 KiB/s wr, 14 op/s 2024-04-25T11:29:47.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:29:46 smithi146 ceph-mon[32821]: pgmap v471: 129 pgs: 129 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.6 KiB/s wr, 14 op/s 2024-04-25T11:29:47.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:29:46 smithi069 ceph-mon[29188]: pgmap v471: 129 pgs: 129 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.6 KiB/s wr, 14 op/s 2024-04-25T11:29:49.227 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:29:48 smithi152 ceph-mon[32701]: pgmap v472: 129 pgs: 129 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.6 KiB/s wr, 13 op/s 2024-04-25T11:29:49.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:29:48 smithi146 ceph-mon[32821]: pgmap v472: 129 pgs: 129 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.6 KiB/s wr, 13 op/s 2024-04-25T11:29:49.440 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:29:48 smithi069 ceph-mon[29188]: pgmap v472: 129 pgs: 129 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.6 KiB/s wr, 13 op/s 2024-04-25T11:29:51.227 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:29:50 smithi152 ceph-mon[32701]: pgmap v473: 129 pgs: 129 active+clean; 2.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:29:51.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:29:50 smithi146 ceph-mon[32821]: pgmap v473: 129 pgs: 129 active+clean; 2.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:29:51.440 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:29:50 smithi069 ceph-mon[29188]: pgmap v473: 129 pgs: 129 active+clean; 2.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:29:52.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:29:52 smithi146 ceph-mon[32821]: pgmap v474: 129 pgs: 129 active+clean; 2.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 14 op/s 2024-04-25T11:29:52.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:29:52 smithi069 ceph-mon[29188]: pgmap v474: 129 pgs: 129 active+clean; 2.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 14 op/s 2024-04-25T11:29:52.477 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:29:52 smithi152 ceph-mon[32701]: pgmap v474: 129 pgs: 129 active+clean; 2.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 14 op/s 2024-04-25T11:29:55.227 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:29:54 smithi152 ceph-mon[32701]: pgmap v475: 129 pgs: 129 active+clean; 2.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 18 op/s 2024-04-25T11:29:55.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:29:54 smithi146 ceph-mon[32821]: pgmap v475: 129 pgs: 129 active+clean; 2.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 18 op/s 2024-04-25T11:29:55.440 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:29:54 smithi069 ceph-mon[29188]: pgmap v475: 129 pgs: 129 active+clean; 2.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 18 op/s 2024-04-25T11:29:57.227 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:29:56 smithi152 ceph-mon[32701]: pgmap v476: 129 pgs: 129 active+clean; 2.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:29:57.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:29:56 smithi146 ceph-mon[32821]: pgmap v476: 129 pgs: 129 active+clean; 2.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:29:57.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:29:56 smithi069 ceph-mon[29188]: pgmap v476: 129 pgs: 129 active+clean; 2.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:29:59.227 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:29:58 smithi152 ceph-mon[32701]: pgmap v477: 129 pgs: 129 active+clean; 2.1 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-04-25T11:29:59.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:29:58 smithi146 ceph-mon[32821]: pgmap v477: 129 pgs: 129 active+clean; 2.1 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-04-25T11:29:59.441 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:29:58 smithi069 ceph-mon[29188]: pgmap v477: 129 pgs: 129 active+clean; 2.1 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-04-25T11:30:00.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:30:00 smithi146 ceph-mon[32821]: pgmap v478: 129 pgs: 129 active+clean; 2.2 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:30:00.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:30:00 smithi146 ceph-mon[32821]: overall HEALTH_OK 2024-04-25T11:30:00.440 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:30:00 smithi069 ceph-mon[29188]: pgmap v478: 129 pgs: 129 active+clean; 2.2 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:30:00.440 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:30:00 smithi069 ceph-mon[29188]: overall HEALTH_OK 2024-04-25T11:30:00.477 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:30:00 smithi152 ceph-mon[32701]: pgmap v478: 129 pgs: 129 active+clean; 2.2 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:30:00.477 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:30:00 smithi152 ceph-mon[32701]: overall HEALTH_OK 2024-04-25T11:30:03.227 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:30:02 smithi152 ceph-mon[32701]: pgmap v479: 129 pgs: 129 active+clean; 2.2 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:30:03.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:30:02 smithi146 ceph-mon[32821]: pgmap v479: 129 pgs: 129 active+clean; 2.2 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:30:03.440 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:30:02 smithi069 ceph-mon[29188]: pgmap v479: 129 pgs: 129 active+clean; 2.2 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:30:05.227 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:30:04 smithi152 ceph-mon[32701]: pgmap v480: 129 pgs: 129 active+clean; 2.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:30:05.242 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:30:04 smithi069 ceph-mon[29188]: pgmap v480: 129 pgs: 129 active+clean; 2.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:30:05.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:30:04 smithi146 ceph-mon[32821]: pgmap v480: 129 pgs: 129 active+clean; 2.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:30:06.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:30:06 smithi146 ceph-mon[32821]: pgmap v481: 129 pgs: 129 active+clean; 2.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:30:06.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:30:06 smithi069 ceph-mon[29188]: pgmap v481: 129 pgs: 129 active+clean; 2.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:30:06.477 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:30:06 smithi152 ceph-mon[32701]: pgmap v481: 129 pgs: 129 active+clean; 2.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:30:09.227 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:30:08 smithi152 ceph-mon[32701]: pgmap v482: 129 pgs: 129 active+clean; 2.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:30:09.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:30:08 smithi146 ceph-mon[32821]: pgmap v482: 129 pgs: 129 active+clean; 2.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:30:09.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:30:08 smithi069 ceph-mon[29188]: pgmap v482: 129 pgs: 129 active+clean; 2.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:30:11.227 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:30:10 smithi152 ceph-mon[32701]: pgmap v483: 129 pgs: 129 active+clean; 2.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:30:11.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:30:10 smithi146 ceph-mon[32821]: pgmap v483: 129 pgs: 129 active+clean; 2.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:30:11.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:30:10 smithi069 ceph-mon[29188]: pgmap v483: 129 pgs: 129 active+clean; 2.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:30:12.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:30:12 smithi146 ceph-mon[32821]: pgmap v484: 129 pgs: 129 active+clean; 2.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:30:12.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:30:12 smithi069 ceph-mon[29188]: pgmap v484: 129 pgs: 129 active+clean; 2.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:30:12.477 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:30:12 smithi152 ceph-mon[32701]: pgmap v484: 129 pgs: 129 active+clean; 2.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:30:15.227 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:30:14 smithi152 ceph-mon[32701]: pgmap v485: 129 pgs: 129 active+clean; 2.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:30:15.322 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:30:14 smithi146 ceph-mon[32821]: pgmap v485: 129 pgs: 129 active+clean; 2.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:30:15.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:30:14 smithi069 ceph-mon[29188]: pgmap v485: 129 pgs: 129 active+clean; 2.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:30:16.254 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:30:16 smithi146 ceph-mon[32821]: pgmap v486: 129 pgs: 129 active+clean; 2.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:30:16.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:30:16 smithi069 ceph-mon[29188]: pgmap v486: 129 pgs: 129 active+clean; 2.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:30:16.477 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:30:16 smithi152 ceph-mon[32701]: pgmap v486: 129 pgs: 129 active+clean; 2.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:30:19.227 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:30:18 smithi152 ceph-mon[32701]: pgmap v487: 129 pgs: 129 active+clean; 2.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:30:19.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:30:18 smithi146 ceph-mon[32821]: pgmap v487: 129 pgs: 129 active+clean; 2.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:30:19.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:30:18 smithi069 ceph-mon[29188]: pgmap v487: 129 pgs: 129 active+clean; 2.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:30:20.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:30:20 smithi146 ceph-mon[32821]: pgmap v488: 129 pgs: 129 active+clean; 2.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:30:20.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:30:20 smithi069 ceph-mon[29188]: pgmap v488: 129 pgs: 129 active+clean; 2.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:30:20.477 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:30:20 smithi152 ceph-mon[32701]: pgmap v488: 129 pgs: 129 active+clean; 2.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:30:23.227 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:30:22 smithi152 ceph-mon[32701]: pgmap v489: 129 pgs: 129 active+clean; 2.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:30:23.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:30:22 smithi146 ceph-mon[32821]: pgmap v489: 129 pgs: 129 active+clean; 2.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:30:23.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:30:22 smithi069 ceph-mon[29188]: pgmap v489: 129 pgs: 129 active+clean; 2.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:30:24.227 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:30:23 smithi152 ceph-mon[32701]: pgmap v490: 129 pgs: 129 active+clean; 2.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:30:24.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:30:23 smithi146 ceph-mon[32821]: pgmap v490: 129 pgs: 129 active+clean; 2.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:30:24.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:30:23 smithi069 ceph-mon[29188]: pgmap v490: 129 pgs: 129 active+clean; 2.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:30:26.329 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:30:26 smithi146 ceph-mon[32821]: pgmap v491: 129 pgs: 129 active+clean; 2.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:30:26.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:30:26 smithi069 ceph-mon[29188]: pgmap v491: 129 pgs: 129 active+clean; 2.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:30:26.477 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:30:26 smithi152 ceph-mon[32701]: pgmap v491: 129 pgs: 129 active+clean; 2.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:30:27.939 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:30:27 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:30:27.976 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:30:27 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:30:28.095 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:30:27 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:30:28.939 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:30:28 smithi069 ceph-mon[29188]: pgmap v492: 129 pgs: 129 active+clean; 2.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:30:28.939 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:30:28 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:30:28.939 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:30:28 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:30:28.940 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:30:28 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:30:28.976 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:30:28 smithi152 ceph-mon[32701]: pgmap v492: 129 pgs: 129 active+clean; 2.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:30:28.977 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:30:28 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:30:28.977 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:30:28 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:30:28.977 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:30:28 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:30:29.095 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:30:28 smithi146 ceph-mon[32821]: pgmap v492: 129 pgs: 129 active+clean; 2.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:30:29.096 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:30:28 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:30:29.096 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:30:28 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:30:29.096 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:30:28 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:30:30.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:30:30 smithi146 ceph-mon[32821]: pgmap v493: 129 pgs: 129 active+clean; 2.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:30:30.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:30:30 smithi069 ceph-mon[29188]: pgmap v493: 129 pgs: 129 active+clean; 2.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:30:30.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:30:30 smithi152 ceph-mon[32701]: pgmap v493: 129 pgs: 129 active+clean; 2.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:30:32.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:30:32 smithi146 ceph-mon[32821]: pgmap v494: 129 pgs: 129 active+clean; 2.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:30:32.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:30:32 smithi069 ceph-mon[29188]: pgmap v494: 129 pgs: 129 active+clean; 2.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:30:32.477 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:30:32 smithi152 ceph-mon[32701]: pgmap v494: 129 pgs: 129 active+clean; 2.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:30:34.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:30:34 smithi146 ceph-mon[32821]: pgmap v495: 129 pgs: 129 active+clean; 2.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:30:34.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:30:34 smithi069 ceph-mon[29188]: pgmap v495: 129 pgs: 129 active+clean; 2.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:30:34.477 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:30:34 smithi152 ceph-mon[32701]: pgmap v495: 129 pgs: 129 active+clean; 2.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:30:36.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:30:36 smithi146 ceph-mon[32821]: pgmap v496: 129 pgs: 129 active+clean; 2.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:30:36.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:30:36 smithi069 ceph-mon[29188]: pgmap v496: 129 pgs: 129 active+clean; 2.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:30:36.477 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:30:36 smithi152 ceph-mon[32701]: pgmap v496: 129 pgs: 129 active+clean; 2.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:30:38.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:30:38 smithi146 ceph-mon[32821]: pgmap v497: 129 pgs: 129 active+clean; 2.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:30:38.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:30:38 smithi069 ceph-mon[29188]: pgmap v497: 129 pgs: 129 active+clean; 2.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:30:38.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:30:38 smithi152 ceph-mon[32701]: pgmap v497: 129 pgs: 129 active+clean; 2.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:30:40.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:30:40 smithi146 ceph-mon[32821]: pgmap v498: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:30:40.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:30:40 smithi069 ceph-mon[29188]: pgmap v498: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:30:40.477 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:30:40 smithi152 ceph-mon[32701]: pgmap v498: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:30:42.279 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:30:42 smithi152 ceph-mon[32701]: pgmap v499: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:30:42.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:30:42 smithi146 ceph-mon[32821]: pgmap v499: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:30:42.436 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.436 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.436 INFO:tasks.workunit.client.0.smithi069.stdout:127/600: open fd = 3 2024-04-25T11:30:42.437 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.437 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.437 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.437 INFO:tasks.workunit.client.0.smithi069.stdout:128/600: open fd = 3 2024-04-25T11:30:42.437 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.437 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.437 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.437 INFO:tasks.workunit.client.0.smithi069.stdout:129/600: open fd = 3 2024-04-25T11:30:42.437 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.437 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.437 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.437 INFO:tasks.workunit.client.0.smithi069.stdout:130/600: open fd = 3 2024-04-25T11:30:42.437 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.438 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.438 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.438 INFO:tasks.workunit.client.0.smithi069.stdout:131/600: open fd = 3 2024-04-25T11:30:42.438 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.438 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.438 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.438 INFO:tasks.workunit.client.0.smithi069.stdout:132/600: open fd = 3 2024-04-25T11:30:42.438 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.438 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.438 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.438 INFO:tasks.workunit.client.0.smithi069.stdout:133/600: open fd = 3 2024-04-25T11:30:42.438 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.438 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.439 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.439 INFO:tasks.workunit.client.0.smithi069.stdout:134/600: open fd = 3 2024-04-25T11:30:42.439 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.439 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.439 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.439 INFO:tasks.workunit.client.0.smithi069.stdout:135/600: open fd = 3 2024-04-25T11:30:42.439 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.439 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.439 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.439 INFO:tasks.workunit.client.0.smithi069.stdout:136/600: open fd = 3 2024-04-25T11:30:42.439 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.439 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.439 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.439 INFO:tasks.workunit.client.0.smithi069.stdout:137/600: open fd = 3 2024-04-25T11:30:42.440 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.440 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.440 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.440 INFO:tasks.workunit.client.0.smithi069.stdout:138/600: open fd = 3 2024-04-25T11:30:42.440 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.440 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.440 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.440 INFO:tasks.workunit.client.0.smithi069.stdout:139/600: open fd = 3 2024-04-25T11:30:42.440 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.440 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.440 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.440 INFO:tasks.workunit.client.0.smithi069.stdout:140/600: open fd = 3 2024-04-25T11:30:42.440 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.440 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.441 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.441 INFO:tasks.workunit.client.0.smithi069.stdout:141/600: open fd = 3 2024-04-25T11:30:42.441 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.441 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.441 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.441 INFO:tasks.workunit.client.0.smithi069.stdout:142/600: open fd = 3 2024-04-25T11:30:42.441 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.441 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.441 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.441 INFO:tasks.workunit.client.0.smithi069.stdout:143/600: open fd = 3 2024-04-25T11:30:42.441 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.441 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.441 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.441 INFO:tasks.workunit.client.0.smithi069.stdout:144/600: open fd = 3 2024-04-25T11:30:42.442 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.442 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.442 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.442 INFO:tasks.workunit.client.0.smithi069.stdout:145/600: open fd = 3 2024-04-25T11:30:42.442 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.442 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.442 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.442 INFO:tasks.workunit.client.0.smithi069.stdout:146/600: open fd = 3 2024-04-25T11:30:42.442 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.442 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.442 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.442 INFO:tasks.workunit.client.0.smithi069.stdout:147/600: open fd = 3 2024-04-25T11:30:42.442 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.443 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.443 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.443 INFO:tasks.workunit.client.0.smithi069.stdout:148/600: open fd = 3 2024-04-25T11:30:42.443 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.443 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.443 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.443 INFO:tasks.workunit.client.0.smithi069.stdout:149/600: open fd = 3 2024-04-25T11:30:42.443 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.443 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.443 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.443 INFO:tasks.workunit.client.0.smithi069.stdout:150/600: open fd = 3 2024-04-25T11:30:42.443 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.443 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.443 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.443 INFO:tasks.workunit.client.0.smithi069.stdout:151/600: open fd = 3 2024-04-25T11:30:42.444 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.444 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.444 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.444 INFO:tasks.workunit.client.0.smithi069.stdout:152/600: open fd = 3 2024-04-25T11:30:42.444 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.444 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.444 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.444 INFO:tasks.workunit.client.0.smithi069.stdout:153/600: open fd = 3 2024-04-25T11:30:42.444 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.444 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.444 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.444 INFO:tasks.workunit.client.0.smithi069.stdout:154/600: open fd = 3 2024-04-25T11:30:42.444 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.444 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.445 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.445 INFO:tasks.workunit.client.0.smithi069.stdout:155/600: open fd = 3 2024-04-25T11:30:42.445 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.445 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.445 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.445 INFO:tasks.workunit.client.0.smithi069.stdout:156/600: open fd = 3 2024-04-25T11:30:42.445 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.445 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.445 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.445 INFO:tasks.workunit.client.0.smithi069.stdout:157/600: open fd = 3 2024-04-25T11:30:42.445 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.445 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.445 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.445 INFO:tasks.workunit.client.0.smithi069.stdout:158/600: open fd = 3 2024-04-25T11:30:42.445 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.446 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.446 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.446 INFO:tasks.workunit.client.0.smithi069.stdout:159/600: open fd = 3 2024-04-25T11:30:42.446 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.446 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.446 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.446 INFO:tasks.workunit.client.0.smithi069.stdout:160/600: open fd = 3 2024-04-25T11:30:42.446 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.446 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.446 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.446 INFO:tasks.workunit.client.0.smithi069.stdout:161/600: open fd = 3 2024-04-25T11:30:42.446 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.446 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.446 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.447 INFO:tasks.workunit.client.0.smithi069.stdout:162/600: open fd = 3 2024-04-25T11:30:42.447 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.447 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.447 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.447 INFO:tasks.workunit.client.0.smithi069.stdout:163/600: open fd = 3 2024-04-25T11:30:42.447 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.447 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.447 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.447 INFO:tasks.workunit.client.0.smithi069.stdout:164/600: open fd = 3 2024-04-25T11:30:42.447 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.447 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.447 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.447 INFO:tasks.workunit.client.0.smithi069.stdout:165/600: open fd = 3 2024-04-25T11:30:42.447 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.447 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.448 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.448 INFO:tasks.workunit.client.0.smithi069.stdout:166/600: open fd = 3 2024-04-25T11:30:42.448 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.448 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.448 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.448 INFO:tasks.workunit.client.0.smithi069.stdout:167/600: open fd = 3 2024-04-25T11:30:42.448 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.448 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.448 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.448 INFO:tasks.workunit.client.0.smithi069.stdout:168/600: open fd = 3 2024-04-25T11:30:42.448 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.448 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.448 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.448 INFO:tasks.workunit.client.0.smithi069.stdout:169/600: open fd = 3 2024-04-25T11:30:42.448 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.449 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.449 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.449 INFO:tasks.workunit.client.0.smithi069.stdout:170/600: open fd = 3 2024-04-25T11:30:42.449 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.449 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.449 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.449 INFO:tasks.workunit.client.0.smithi069.stdout:171/600: open fd = 3 2024-04-25T11:30:42.449 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.449 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.449 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.449 INFO:tasks.workunit.client.0.smithi069.stdout:172/600: open fd = 3 2024-04-25T11:30:42.449 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.449 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.449 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.449 INFO:tasks.workunit.client.0.smithi069.stdout:173/600: open fd = 3 2024-04-25T11:30:42.449 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.449 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.450 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.450 INFO:tasks.workunit.client.0.smithi069.stdout:174/600: open fd = 3 2024-04-25T11:30:42.450 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.450 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.450 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.450 INFO:tasks.workunit.client.0.smithi069.stdout:175/600: open fd = 3 2024-04-25T11:30:42.450 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.450 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.450 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.450 INFO:tasks.workunit.client.0.smithi069.stdout:176/600: open fd = 3 2024-04-25T11:30:42.450 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.450 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.450 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.450 INFO:tasks.workunit.client.0.smithi069.stdout:177/600: open fd = 3 2024-04-25T11:30:42.451 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.451 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.451 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.451 INFO:tasks.workunit.client.0.smithi069.stdout:178/600: open fd = 3 2024-04-25T11:30:42.451 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.451 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.451 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.451 INFO:tasks.workunit.client.0.smithi069.stdout:179/600: open fd = 3 2024-04-25T11:30:42.451 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.451 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.451 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.451 INFO:tasks.workunit.client.0.smithi069.stdout:180/600: open fd = 3 2024-04-25T11:30:42.451 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.451 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.452 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.452 INFO:tasks.workunit.client.0.smithi069.stdout:181/600: open fd = 3 2024-04-25T11:30:42.452 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.452 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.452 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.452 INFO:tasks.workunit.client.0.smithi069.stdout:182/600: open fd = 3 2024-04-25T11:30:42.452 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.452 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.452 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.452 INFO:tasks.workunit.client.0.smithi069.stdout:183/600: open fd = 3 2024-04-25T11:30:42.452 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.452 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.452 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.453 INFO:tasks.workunit.client.0.smithi069.stdout:184/600: open fd = 3 2024-04-25T11:30:42.453 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.453 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.453 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.453 INFO:tasks.workunit.client.0.smithi069.stdout:185/600: open fd = 3 2024-04-25T11:30:42.453 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.453 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.453 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.453 INFO:tasks.workunit.client.0.smithi069.stdout:186/600: open fd = 3 2024-04-25T11:30:42.453 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.453 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.453 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.453 INFO:tasks.workunit.client.0.smithi069.stdout:187/600: open fd = 3 2024-04-25T11:30:42.453 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.453 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.453 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:30:42.453 INFO:tasks.workunit.client.0.smithi069.stdout:188/600: open fd = 3 2024-04-25T11:30:42.454 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.454 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:30:42.454 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:30:42 smithi069 ceph-mon[29188]: pgmap v499: 129 pgs: 129 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:30:44.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:30:44 smithi146 ceph-mon[32821]: pgmap v500: 129 pgs: 129 active+clean; 2.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:30:44.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:30:44 smithi069 ceph-mon[29188]: pgmap v500: 129 pgs: 129 active+clean; 2.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:30:44.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:30:44 smithi152 ceph-mon[32701]: pgmap v500: 129 pgs: 129 active+clean; 2.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:30:46.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:30:46 smithi146 ceph-mon[32821]: pgmap v501: 129 pgs: 129 active+clean; 2.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:30:46.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:30:46 smithi069 ceph-mon[29188]: pgmap v501: 129 pgs: 129 active+clean; 2.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:30:46.477 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:30:46 smithi152 ceph-mon[32701]: pgmap v501: 129 pgs: 129 active+clean; 2.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:30:48.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:30:48 smithi146 ceph-mon[32821]: pgmap v502: 129 pgs: 129 active+clean; 2.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-04-25T11:30:48.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:30:48 smithi069 ceph-mon[29188]: pgmap v502: 129 pgs: 129 active+clean; 2.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-04-25T11:30:48.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:30:48 smithi152 ceph-mon[32701]: pgmap v502: 129 pgs: 129 active+clean; 2.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-04-25T11:30:50.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:30:50 smithi146 ceph-mon[32821]: pgmap v503: 129 pgs: 129 active+clean; 2.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:30:50.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:30:50 smithi069 ceph-mon[29188]: pgmap v503: 129 pgs: 129 active+clean; 2.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:30:50.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:30:50 smithi152 ceph-mon[32701]: pgmap v503: 129 pgs: 129 active+clean; 2.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:30:52.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:30:52 smithi146 ceph-mon[32821]: pgmap v504: 129 pgs: 129 active+clean; 2.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:30:52.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:30:52 smithi069 ceph-mon[29188]: pgmap v504: 129 pgs: 129 active+clean; 2.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:30:52.477 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:30:52 smithi152 ceph-mon[32701]: pgmap v504: 129 pgs: 129 active+clean; 2.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:30:54.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:30:54 smithi146 ceph-mon[32821]: pgmap v505: 129 pgs: 129 active+clean; 2.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:30:54.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:30:54 smithi069 ceph-mon[29188]: pgmap v505: 129 pgs: 129 active+clean; 2.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:30:54.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:30:54 smithi152 ceph-mon[32701]: pgmap v505: 129 pgs: 129 active+clean; 2.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:30:56.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:30:56 smithi146 ceph-mon[32821]: pgmap v506: 129 pgs: 129 active+clean; 2.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:30:56.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:30:56 smithi069 ceph-mon[29188]: pgmap v506: 129 pgs: 129 active+clean; 2.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:30:56.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:30:56 smithi152 ceph-mon[32701]: pgmap v506: 129 pgs: 129 active+clean; 2.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:30:58.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:30:58 smithi146 ceph-mon[32821]: pgmap v507: 129 pgs: 129 active+clean; 2.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:30:58.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:30:58 smithi069 ceph-mon[29188]: pgmap v507: 129 pgs: 129 active+clean; 2.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:30:58.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:30:58 smithi152 ceph-mon[32701]: pgmap v507: 129 pgs: 129 active+clean; 2.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:31:00.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:31:00 smithi146 ceph-mon[32821]: pgmap v508: 129 pgs: 129 active+clean; 2.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:31:00.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:31:00 smithi069 ceph-mon[29188]: pgmap v508: 129 pgs: 129 active+clean; 2.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:31:00.477 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:31:00 smithi152 ceph-mon[32701]: pgmap v508: 129 pgs: 129 active+clean; 2.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:31:02.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:31:02 smithi146 ceph-mon[32821]: pgmap v509: 129 pgs: 129 active+clean; 2.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:31:02.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:31:02 smithi069 ceph-mon[29188]: pgmap v509: 129 pgs: 129 active+clean; 2.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:31:02.477 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:31:02 smithi152 ceph-mon[32701]: pgmap v509: 129 pgs: 129 active+clean; 2.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:31:04.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:31:04 smithi146 ceph-mon[32821]: pgmap v510: 129 pgs: 129 active+clean; 2.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:31:04.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:31:04 smithi069 ceph-mon[29188]: pgmap v510: 129 pgs: 129 active+clean; 2.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:31:04.477 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:31:04 smithi152 ceph-mon[32701]: pgmap v510: 129 pgs: 129 active+clean; 2.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:31:06.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:31:06 smithi146 ceph-mon[32821]: pgmap v511: 129 pgs: 129 active+clean; 2.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:31:06.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:31:06 smithi069 ceph-mon[29188]: pgmap v511: 129 pgs: 129 active+clean; 2.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:31:06.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:31:06 smithi152 ceph-mon[32701]: pgmap v511: 129 pgs: 129 active+clean; 2.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:31:08.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:31:08 smithi146 ceph-mon[32821]: pgmap v512: 129 pgs: 129 active+clean; 2.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:31:08.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:31:08 smithi069 ceph-mon[29188]: pgmap v512: 129 pgs: 129 active+clean; 2.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:31:08.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:31:08 smithi152 ceph-mon[32701]: pgmap v512: 129 pgs: 129 active+clean; 2.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:31:10.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:31:10 smithi146 ceph-mon[32821]: pgmap v513: 129 pgs: 129 active+clean; 2.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:31:10.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:31:10 smithi069 ceph-mon[29188]: pgmap v513: 129 pgs: 129 active+clean; 2.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:31:10.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:31:10 smithi152 ceph-mon[32701]: pgmap v513: 129 pgs: 129 active+clean; 2.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:31:12.279 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:31:12 smithi152 ceph-mon[32701]: pgmap v514: 129 pgs: 129 active+clean; 2.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:31:12.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:31:12 smithi146 ceph-mon[32821]: pgmap v514: 129 pgs: 129 active+clean; 2.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:31:12.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:31:12 smithi069 ceph-mon[29188]: pgmap v514: 129 pgs: 129 active+clean; 2.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:31:14.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:31:14 smithi146 ceph-mon[32821]: pgmap v515: 129 pgs: 129 active+clean; 3.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:31:14.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:31:14 smithi069 ceph-mon[29188]: pgmap v515: 129 pgs: 129 active+clean; 3.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:31:14.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:31:14 smithi152 ceph-mon[32701]: pgmap v515: 129 pgs: 129 active+clean; 3.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:31:16.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:31:16 smithi146 ceph-mon[32821]: pgmap v516: 129 pgs: 129 active+clean; 3.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:31:16.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:31:16 smithi069 ceph-mon[29188]: pgmap v516: 129 pgs: 129 active+clean; 3.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:31:16.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:31:16 smithi152 ceph-mon[32701]: pgmap v516: 129 pgs: 129 active+clean; 3.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:31:18.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:31:18 smithi146 ceph-mon[32821]: pgmap v517: 129 pgs: 129 active+clean; 3.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-04-25T11:31:18.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:31:18 smithi069 ceph-mon[29188]: pgmap v517: 129 pgs: 129 active+clean; 3.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-04-25T11:31:18.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:31:18 smithi152 ceph-mon[32701]: pgmap v517: 129 pgs: 129 active+clean; 3.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-04-25T11:31:20.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:31:20 smithi146 ceph-mon[32821]: pgmap v518: 129 pgs: 129 active+clean; 3.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:31:20.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:31:20 smithi069 ceph-mon[29188]: pgmap v518: 129 pgs: 129 active+clean; 3.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:31:20.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:31:20 smithi152 ceph-mon[32701]: pgmap v518: 129 pgs: 129 active+clean; 3.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:31:22.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:31:22 smithi146 ceph-mon[32821]: pgmap v519: 129 pgs: 129 active+clean; 3.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:31:22.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:31:22 smithi069 ceph-mon[29188]: pgmap v519: 129 pgs: 129 active+clean; 3.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:31:22.477 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:31:22 smithi152 ceph-mon[32701]: pgmap v519: 129 pgs: 129 active+clean; 3.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:31:24.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:31:24 smithi146 ceph-mon[32821]: pgmap v520: 129 pgs: 129 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:31:24.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:31:24 smithi069 ceph-mon[29188]: pgmap v520: 129 pgs: 129 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:31:24.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:31:24 smithi152 ceph-mon[32701]: pgmap v520: 129 pgs: 129 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:31:26.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:31:26 smithi146 ceph-mon[32821]: pgmap v521: 129 pgs: 129 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:31:26.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:31:26 smithi069 ceph-mon[29188]: pgmap v521: 129 pgs: 129 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:31:26.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:31:26 smithi152 ceph-mon[32701]: pgmap v521: 129 pgs: 129 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:31:28.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:31:28 smithi146 ceph-mon[32821]: pgmap v522: 129 pgs: 129 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:31:28.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:31:28 smithi069 ceph-mon[29188]: pgmap v522: 129 pgs: 129 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:31:28.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:31:28 smithi152 ceph-mon[32701]: pgmap v522: 129 pgs: 129 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:31:29.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:31:29 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:31:29.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:31:29 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:31:29.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:31:29 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:31:30.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:31:30 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:31:30.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:31:30 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:31:30.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:31:30 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:31:30.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:31:30 smithi146 ceph-mon[32821]: pgmap v523: 129 pgs: 129 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-04-25T11:31:30.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:31:30 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:31:30.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:31:30 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:31:30.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:31:30 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:31:30.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:31:30 smithi069 ceph-mon[29188]: pgmap v523: 129 pgs: 129 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-04-25T11:31:30.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:31:30 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:31:30.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:31:30 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:31:30.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:31:30 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:31:30.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:31:30 smithi152 ceph-mon[32701]: pgmap v523: 129 pgs: 129 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-04-25T11:31:32.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:31:32 smithi146 ceph-mon[32821]: pgmap v524: 129 pgs: 129 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.4 KiB/s wr, 14 op/s 2024-04-25T11:31:32.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:31:32 smithi069 ceph-mon[29188]: pgmap v524: 129 pgs: 129 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.4 KiB/s wr, 14 op/s 2024-04-25T11:31:32.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:31:32 smithi152 ceph-mon[32701]: pgmap v524: 129 pgs: 129 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.4 KiB/s wr, 14 op/s 2024-04-25T11:31:34.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:31:34 smithi146 ceph-mon[32821]: pgmap v525: 129 pgs: 129 active+clean; 3.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 18 op/s 2024-04-25T11:31:34.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:31:34 smithi069 ceph-mon[29188]: pgmap v525: 129 pgs: 129 active+clean; 3.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 18 op/s 2024-04-25T11:31:34.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:31:34 smithi152 ceph-mon[32701]: pgmap v525: 129 pgs: 129 active+clean; 3.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 18 op/s 2024-04-25T11:31:36.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:31:36 smithi146 ceph-mon[32821]: pgmap v526: 129 pgs: 129 active+clean; 3.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 14 op/s 2024-04-25T11:31:36.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:31:36 smithi069 ceph-mon[29188]: pgmap v526: 129 pgs: 129 active+clean; 3.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 14 op/s 2024-04-25T11:31:36.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:31:36 smithi152 ceph-mon[32701]: pgmap v526: 129 pgs: 129 active+clean; 3.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 14 op/s 2024-04-25T11:31:38.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:31:38 smithi146 ceph-mon[32821]: pgmap v527: 129 pgs: 129 active+clean; 3.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 13 op/s 2024-04-25T11:31:38.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:31:38 smithi069 ceph-mon[29188]: pgmap v527: 129 pgs: 129 active+clean; 3.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 13 op/s 2024-04-25T11:31:38.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:31:38 smithi152 ceph-mon[32701]: pgmap v527: 129 pgs: 129 active+clean; 3.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 13 op/s 2024-04-25T11:31:40.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:31:40 smithi146 ceph-mon[32821]: pgmap v528: 129 pgs: 129 active+clean; 3.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:31:40.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:31:40 smithi069 ceph-mon[29188]: pgmap v528: 129 pgs: 129 active+clean; 3.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:31:40.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:31:40 smithi152 ceph-mon[32701]: pgmap v528: 129 pgs: 129 active+clean; 3.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:31:42.279 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:31:42 smithi152 ceph-mon[32701]: pgmap v529: 129 pgs: 129 active+clean; 3.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:31:42.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:31:42 smithi146 ceph-mon[32821]: pgmap v529: 129 pgs: 129 active+clean; 3.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:31:42.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:31:42 smithi069 ceph-mon[29188]: pgmap v529: 129 pgs: 129 active+clean; 3.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:31:44.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:31:44 smithi146 ceph-mon[32821]: pgmap v530: 129 pgs: 129 active+clean; 3.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:31:44.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:31:44 smithi069 ceph-mon[29188]: pgmap v530: 129 pgs: 129 active+clean; 3.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:31:44.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:31:44 smithi152 ceph-mon[32701]: pgmap v530: 129 pgs: 129 active+clean; 3.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:31:45.042 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.042 INFO:tasks.workunit.client.0.smithi069.stdout:189/600: open fd = 3 2024-04-25T11:31:45.042 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.042 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.042 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.042 INFO:tasks.workunit.client.0.smithi069.stdout:190/600: open fd = 3 2024-04-25T11:31:45.042 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.042 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.042 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.043 INFO:tasks.workunit.client.0.smithi069.stdout:191/600: open fd = 3 2024-04-25T11:31:45.043 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.043 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.043 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.043 INFO:tasks.workunit.client.0.smithi069.stdout:192/600: open fd = 3 2024-04-25T11:31:45.043 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.043 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.043 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.043 INFO:tasks.workunit.client.0.smithi069.stdout:193/600: open fd = 3 2024-04-25T11:31:45.043 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.043 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.043 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.043 INFO:tasks.workunit.client.0.smithi069.stdout:194/600: open fd = 3 2024-04-25T11:31:45.043 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.043 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.044 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.044 INFO:tasks.workunit.client.0.smithi069.stdout:195/600: open fd = 3 2024-04-25T11:31:45.044 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.044 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.044 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.044 INFO:tasks.workunit.client.0.smithi069.stdout:196/600: open fd = 3 2024-04-25T11:31:45.044 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.044 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.044 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.044 INFO:tasks.workunit.client.0.smithi069.stdout:197/600: open fd = 3 2024-04-25T11:31:45.044 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.044 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.044 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.044 INFO:tasks.workunit.client.0.smithi069.stdout:198/600: open fd = 3 2024-04-25T11:31:45.044 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.044 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.045 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.045 INFO:tasks.workunit.client.0.smithi069.stdout:199/600: open fd = 3 2024-04-25T11:31:45.045 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.045 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.045 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.045 INFO:tasks.workunit.client.0.smithi069.stdout:200/600: open fd = 3 2024-04-25T11:31:45.045 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.045 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.045 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.045 INFO:tasks.workunit.client.0.smithi069.stdout:201/600: open fd = 3 2024-04-25T11:31:45.045 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.045 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.045 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.045 INFO:tasks.workunit.client.0.smithi069.stdout:202/600: open fd = 3 2024-04-25T11:31:45.046 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.046 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.046 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.046 INFO:tasks.workunit.client.0.smithi069.stdout:203/600: open fd = 3 2024-04-25T11:31:45.046 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.046 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.046 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.046 INFO:tasks.workunit.client.0.smithi069.stdout:204/600: open fd = 3 2024-04-25T11:31:45.046 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.046 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.046 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.046 INFO:tasks.workunit.client.0.smithi069.stdout:205/600: open fd = 3 2024-04-25T11:31:45.046 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.046 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.047 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.047 INFO:tasks.workunit.client.0.smithi069.stdout:206/600: open fd = 3 2024-04-25T11:31:45.047 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.047 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.047 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.047 INFO:tasks.workunit.client.0.smithi069.stdout:207/600: open fd = 3 2024-04-25T11:31:45.047 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.047 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.047 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.047 INFO:tasks.workunit.client.0.smithi069.stdout:208/600: open fd = 3 2024-04-25T11:31:45.047 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.047 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.047 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.047 INFO:tasks.workunit.client.0.smithi069.stdout:209/600: open fd = 3 2024-04-25T11:31:45.048 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.048 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.048 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.048 INFO:tasks.workunit.client.0.smithi069.stdout:210/600: open fd = 3 2024-04-25T11:31:45.048 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.048 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.048 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.048 INFO:tasks.workunit.client.0.smithi069.stdout:211/600: open fd = 3 2024-04-25T11:31:45.048 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.048 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.048 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.048 INFO:tasks.workunit.client.0.smithi069.stdout:212/600: open fd = 3 2024-04-25T11:31:45.048 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.048 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.049 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.049 INFO:tasks.workunit.client.0.smithi069.stdout:213/600: open fd = 3 2024-04-25T11:31:45.049 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.049 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.049 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.049 INFO:tasks.workunit.client.0.smithi069.stdout:214/600: open fd = 3 2024-04-25T11:31:45.049 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.049 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.049 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.049 INFO:tasks.workunit.client.0.smithi069.stdout:215/600: open fd = 3 2024-04-25T11:31:45.049 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.049 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.049 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.049 INFO:tasks.workunit.client.0.smithi069.stdout:216/600: open fd = 3 2024-04-25T11:31:45.049 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.050 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.050 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.050 INFO:tasks.workunit.client.0.smithi069.stdout:217/600: open fd = 3 2024-04-25T11:31:45.050 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.050 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.050 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.050 INFO:tasks.workunit.client.0.smithi069.stdout:218/600: open fd = 3 2024-04-25T11:31:45.050 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.050 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.050 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.050 INFO:tasks.workunit.client.0.smithi069.stdout:219/600: open fd = 3 2024-04-25T11:31:45.050 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.050 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.050 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.050 INFO:tasks.workunit.client.0.smithi069.stdout:220/600: open fd = 3 2024-04-25T11:31:45.051 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.051 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.051 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.051 INFO:tasks.workunit.client.0.smithi069.stdout:221/600: open fd = 3 2024-04-25T11:31:45.051 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.051 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.051 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.051 INFO:tasks.workunit.client.0.smithi069.stdout:222/600: open fd = 3 2024-04-25T11:31:45.051 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.051 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.051 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.051 INFO:tasks.workunit.client.0.smithi069.stdout:223/600: open fd = 3 2024-04-25T11:31:45.051 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.051 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.051 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.052 INFO:tasks.workunit.client.0.smithi069.stdout:224/600: open fd = 3 2024-04-25T11:31:45.052 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.052 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.052 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.052 INFO:tasks.workunit.client.0.smithi069.stdout:225/600: open fd = 3 2024-04-25T11:31:45.052 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.052 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.052 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.052 INFO:tasks.workunit.client.0.smithi069.stdout:226/600: open fd = 3 2024-04-25T11:31:45.052 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.052 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.052 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.052 INFO:tasks.workunit.client.0.smithi069.stdout:227/600: open fd = 3 2024-04-25T11:31:45.052 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.052 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.053 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.053 INFO:tasks.workunit.client.0.smithi069.stdout:228/600: open fd = 3 2024-04-25T11:31:45.053 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.053 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.053 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.053 INFO:tasks.workunit.client.0.smithi069.stdout:229/600: open fd = 3 2024-04-25T11:31:45.053 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.053 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.053 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.053 INFO:tasks.workunit.client.0.smithi069.stdout:230/600: open fd = 3 2024-04-25T11:31:45.053 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.053 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.053 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.053 INFO:tasks.workunit.client.0.smithi069.stdout:231/600: open fd = 3 2024-04-25T11:31:45.054 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.054 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.054 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.054 INFO:tasks.workunit.client.0.smithi069.stdout:232/600: open fd = 3 2024-04-25T11:31:45.054 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.054 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.054 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.054 INFO:tasks.workunit.client.0.smithi069.stdout:233/600: open fd = 3 2024-04-25T11:31:45.054 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.054 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.054 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.054 INFO:tasks.workunit.client.0.smithi069.stdout:234/600: open fd = 3 2024-04-25T11:31:45.054 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.054 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.054 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.055 INFO:tasks.workunit.client.0.smithi069.stdout:235/600: open fd = 3 2024-04-25T11:31:45.055 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.055 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.055 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.055 INFO:tasks.workunit.client.0.smithi069.stdout:236/600: open fd = 3 2024-04-25T11:31:45.055 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.055 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.055 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.055 INFO:tasks.workunit.client.0.smithi069.stdout:237/600: open fd = 3 2024-04-25T11:31:45.055 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.055 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.055 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.055 INFO:tasks.workunit.client.0.smithi069.stdout:238/600: open fd = 3 2024-04-25T11:31:45.055 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.055 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.055 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.056 INFO:tasks.workunit.client.0.smithi069.stdout:239/600: open fd = 3 2024-04-25T11:31:45.056 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.056 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.056 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.056 INFO:tasks.workunit.client.0.smithi069.stdout:240/600: open fd = 3 2024-04-25T11:31:45.056 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.056 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.056 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.056 INFO:tasks.workunit.client.0.smithi069.stdout:241/600: open fd = 3 2024-04-25T11:31:45.056 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.056 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.056 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.056 INFO:tasks.workunit.client.0.smithi069.stdout:242/600: open fd = 3 2024-04-25T11:31:45.056 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.056 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.057 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.057 INFO:tasks.workunit.client.0.smithi069.stdout:243/600: open fd = 3 2024-04-25T11:31:45.057 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.057 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.057 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.057 INFO:tasks.workunit.client.0.smithi069.stdout:244/600: open fd = 3 2024-04-25T11:31:45.057 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.057 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.057 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.057 INFO:tasks.workunit.client.0.smithi069.stdout:245/600: open fd = 3 2024-04-25T11:31:45.057 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.057 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.057 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.058 INFO:tasks.workunit.client.0.smithi069.stdout:246/600: open fd = 3 2024-04-25T11:31:45.058 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.058 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.058 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.058 INFO:tasks.workunit.client.0.smithi069.stdout:247/600: open fd = 3 2024-04-25T11:31:45.058 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.058 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.058 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.058 INFO:tasks.workunit.client.0.smithi069.stdout:248/600: open fd = 3 2024-04-25T11:31:45.058 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.058 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.058 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.058 INFO:tasks.workunit.client.0.smithi069.stdout:249/600: open fd = 3 2024-04-25T11:31:45.058 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.058 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.059 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:31:45.059 INFO:tasks.workunit.client.0.smithi069.stdout:250/600: open fd = 3 2024-04-25T11:31:45.059 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:45.059 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:31:46.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:31:46 smithi146 ceph-mon[32821]: pgmap v531: 129 pgs: 129 active+clean; 3.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:31:46.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:31:46 smithi069 ceph-mon[29188]: pgmap v531: 129 pgs: 129 active+clean; 3.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:31:46.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:31:46 smithi152 ceph-mon[32701]: pgmap v531: 129 pgs: 129 active+clean; 3.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:31:48.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:31:48 smithi146 ceph-mon[32821]: pgmap v532: 129 pgs: 129 active+clean; 3.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-04-25T11:31:48.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:31:48 smithi069 ceph-mon[29188]: pgmap v532: 129 pgs: 129 active+clean; 3.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-04-25T11:31:48.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:31:48 smithi152 ceph-mon[32701]: pgmap v532: 129 pgs: 129 active+clean; 3.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-04-25T11:31:50.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:31:50 smithi146 ceph-mon[32821]: pgmap v533: 129 pgs: 129 active+clean; 3.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:31:50.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:31:50 smithi069 ceph-mon[29188]: pgmap v533: 129 pgs: 129 active+clean; 3.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:31:50.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:31:50 smithi152 ceph-mon[32701]: pgmap v533: 129 pgs: 129 active+clean; 3.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:31:52.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:31:52 smithi146 ceph-mon[32821]: pgmap v534: 129 pgs: 129 active+clean; 3.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:31:52.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:31:52 smithi069 ceph-mon[29188]: pgmap v534: 129 pgs: 129 active+clean; 3.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:31:52.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:31:52 smithi152 ceph-mon[32701]: pgmap v534: 129 pgs: 129 active+clean; 3.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:31:54.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:31:54 smithi146 ceph-mon[32821]: pgmap v535: 129 pgs: 129 active+clean; 3.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:31:54.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:31:54 smithi069 ceph-mon[29188]: pgmap v535: 129 pgs: 129 active+clean; 3.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:31:54.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:31:54 smithi152 ceph-mon[32701]: pgmap v535: 129 pgs: 129 active+clean; 3.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:31:56.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:31:56 smithi146 ceph-mon[32821]: pgmap v536: 129 pgs: 129 active+clean; 3.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:31:56.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:31:56 smithi069 ceph-mon[29188]: pgmap v536: 129 pgs: 129 active+clean; 3.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:31:56.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:31:56 smithi152 ceph-mon[32701]: pgmap v536: 129 pgs: 129 active+clean; 3.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:31:58.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:31:58 smithi146 ceph-mon[32821]: pgmap v537: 129 pgs: 129 active+clean; 3.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-04-25T11:31:58.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:31:58 smithi069 ceph-mon[29188]: pgmap v537: 129 pgs: 129 active+clean; 3.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-04-25T11:31:58.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:31:58 smithi152 ceph-mon[32701]: pgmap v537: 129 pgs: 129 active+clean; 3.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-04-25T11:32:00.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:32:00 smithi146 ceph-mon[32821]: pgmap v538: 129 pgs: 129 active+clean; 3.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:32:00.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:32:00 smithi069 ceph-mon[29188]: pgmap v538: 129 pgs: 129 active+clean; 3.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:32:00.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:32:00 smithi152 ceph-mon[32701]: pgmap v538: 129 pgs: 129 active+clean; 3.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:32:02.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:32:02 smithi146 ceph-mon[32821]: pgmap v539: 129 pgs: 129 active+clean; 3.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:32:02.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:32:02 smithi069 ceph-mon[29188]: pgmap v539: 129 pgs: 129 active+clean; 3.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:32:02.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:32:02 smithi152 ceph-mon[32701]: pgmap v539: 129 pgs: 129 active+clean; 3.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:32:04.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:32:04 smithi146 ceph-mon[32821]: pgmap v540: 129 pgs: 129 active+clean; 3.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:32:04.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:32:04 smithi069 ceph-mon[29188]: pgmap v540: 129 pgs: 129 active+clean; 3.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:32:04.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:32:04 smithi152 ceph-mon[32701]: pgmap v540: 129 pgs: 129 active+clean; 3.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:32:06.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:32:06 smithi146 ceph-mon[32821]: pgmap v541: 129 pgs: 129 active+clean; 3.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:32:06.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:32:06 smithi069 ceph-mon[29188]: pgmap v541: 129 pgs: 129 active+clean; 3.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:32:06.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:32:06 smithi152 ceph-mon[32701]: pgmap v541: 129 pgs: 129 active+clean; 3.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:32:08.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:32:08 smithi146 ceph-mon[32821]: pgmap v542: 129 pgs: 129 active+clean; 3.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-04-25T11:32:08.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:32:08 smithi069 ceph-mon[29188]: pgmap v542: 129 pgs: 129 active+clean; 3.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-04-25T11:32:08.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:32:08 smithi152 ceph-mon[32701]: pgmap v542: 129 pgs: 129 active+clean; 3.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-04-25T11:32:10.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:32:10 smithi146 ceph-mon[32821]: pgmap v543: 129 pgs: 129 active+clean; 3.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:32:10.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:32:10 smithi069 ceph-mon[29188]: pgmap v543: 129 pgs: 129 active+clean; 3.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:32:10.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:32:10 smithi152 ceph-mon[32701]: pgmap v543: 129 pgs: 129 active+clean; 3.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:32:12.278 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:32:12 smithi152 ceph-mon[32701]: pgmap v544: 129 pgs: 129 active+clean; 3.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:32:12.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:32:12 smithi146 ceph-mon[32821]: pgmap v544: 129 pgs: 129 active+clean; 3.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:32:12.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:32:12 smithi069 ceph-mon[29188]: pgmap v544: 129 pgs: 129 active+clean; 3.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:32:14.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:32:14 smithi146 ceph-mon[32821]: pgmap v545: 129 pgs: 129 active+clean; 3.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:32:14.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:32:14 smithi069 ceph-mon[29188]: pgmap v545: 129 pgs: 129 active+clean; 3.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:32:14.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:32:14 smithi152 ceph-mon[32701]: pgmap v545: 129 pgs: 129 active+clean; 3.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:32:16.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:32:16 smithi146 ceph-mon[32821]: pgmap v546: 129 pgs: 129 active+clean; 3.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:32:16.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:32:16 smithi069 ceph-mon[29188]: pgmap v546: 129 pgs: 129 active+clean; 3.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:32:16.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:32:16 smithi152 ceph-mon[32701]: pgmap v546: 129 pgs: 129 active+clean; 3.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:32:18.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:32:18 smithi146 ceph-mon[32821]: pgmap v547: 129 pgs: 129 active+clean; 3.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:32:18.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:32:18 smithi069 ceph-mon[29188]: pgmap v547: 129 pgs: 129 active+clean; 3.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:32:18.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:32:18 smithi152 ceph-mon[32701]: pgmap v547: 129 pgs: 129 active+clean; 3.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:32:20.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:32:20 smithi146 ceph-mon[32821]: pgmap v548: 129 pgs: 129 active+clean; 3.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:32:20.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:32:20 smithi069 ceph-mon[29188]: pgmap v548: 129 pgs: 129 active+clean; 3.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:32:20.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:32:20 smithi152 ceph-mon[32701]: pgmap v548: 129 pgs: 129 active+clean; 3.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:32:22.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:32:22 smithi146 ceph-mon[32821]: pgmap v549: 129 pgs: 129 active+clean; 3.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:32:22.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:32:22 smithi069 ceph-mon[29188]: pgmap v549: 129 pgs: 129 active+clean; 3.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:32:22.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:32:22 smithi152 ceph-mon[32701]: pgmap v549: 129 pgs: 129 active+clean; 3.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:32:24.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:32:24 smithi146 ceph-mon[32821]: pgmap v550: 129 pgs: 129 active+clean; 3.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:32:24.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:32:24 smithi069 ceph-mon[29188]: pgmap v550: 129 pgs: 129 active+clean; 3.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:32:24.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:32:24 smithi152 ceph-mon[32701]: pgmap v550: 129 pgs: 129 active+clean; 3.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:32:26.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:32:26 smithi146 ceph-mon[32821]: pgmap v551: 129 pgs: 129 active+clean; 3.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:32:26.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:32:26 smithi069 ceph-mon[29188]: pgmap v551: 129 pgs: 129 active+clean; 3.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:32:26.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:32:26 smithi152 ceph-mon[32701]: pgmap v551: 129 pgs: 129 active+clean; 3.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:32:28.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:32:28 smithi146 ceph-mon[32821]: pgmap v552: 129 pgs: 129 active+clean; 3.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:32:28.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:32:28 smithi069 ceph-mon[29188]: pgmap v552: 129 pgs: 129 active+clean; 3.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:32:28.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:32:28 smithi152 ceph-mon[32701]: pgmap v552: 129 pgs: 129 active+clean; 3.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:32:29.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:32:29 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:32:29.594 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:32:29 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:32:29.688 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:32:29 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:32:30.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:32:30 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:32:30.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:32:30 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:32:30.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:32:30 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:32:30.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:32:30 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:32:30.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:32:30 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:32:30.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:32:30 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:32:30.477 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:32:30 smithi152 ceph-mon[32701]: Adjusting osd_memory_target on smithi146 to 176.2M 2024-04-25T11:32:30.477 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:32:30 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:32:30.477 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:32:30 smithi152 ceph-mon[32701]: Unable to set osd_memory_target on smithi146 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-25T11:32:30.477 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:32:30 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:32:30.477 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:32:30 smithi152 ceph-mon[32701]: Adjusting osd_memory_target on smithi069 to 176.1M 2024-04-25T11:32:30.477 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:32:30 smithi152 ceph-mon[32701]: Unable to set osd_memory_target on smithi069 to 184727244: error parsing value: Value '184727244' is below minimum 939524096 2024-04-25T11:32:30.477 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:32:30 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:32:30.477 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:32:30 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:32:30.477 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:32:30 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:32:30.477 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:32:30 smithi152 ceph-mon[32701]: pgmap v553: 129 pgs: 129 active+clean; 3.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:32:30.594 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:32:30 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:32:30.595 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:32:30 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:32:30.595 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:32:30 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:32:30.595 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:32:30 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:32:30.595 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:32:30 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:32:30.595 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:32:30 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:32:30.595 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:32:30 smithi146 ceph-mon[32821]: Adjusting osd_memory_target on smithi146 to 176.2M 2024-04-25T11:32:30.595 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:32:30 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:32:30.595 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:32:30 smithi146 ceph-mon[32821]: Unable to set osd_memory_target on smithi146 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-25T11:32:30.595 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:32:30 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:32:30.595 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:32:30 smithi146 ceph-mon[32821]: Adjusting osd_memory_target on smithi069 to 176.1M 2024-04-25T11:32:30.595 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:32:30 smithi146 ceph-mon[32821]: Unable to set osd_memory_target on smithi069 to 184727244: error parsing value: Value '184727244' is below minimum 939524096 2024-04-25T11:32:30.596 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:32:30 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:32:30.596 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:32:30 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:32:30.596 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:32:30 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:32:30.596 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:32:30 smithi146 ceph-mon[32821]: pgmap v553: 129 pgs: 129 active+clean; 3.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:32:30.689 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:32:30 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:32:30.689 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:32:30 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:32:30.689 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:32:30 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:32:30.689 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:32:30 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:32:30.689 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:32:30 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:32:30.689 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:32:30 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:32:30.689 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:32:30 smithi069 ceph-mon[29188]: Adjusting osd_memory_target on smithi146 to 176.2M 2024-04-25T11:32:30.689 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:32:30 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:32:30.689 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:32:30 smithi069 ceph-mon[29188]: Unable to set osd_memory_target on smithi146 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-25T11:32:30.689 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:32:30 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:32:30.689 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:32:30 smithi069 ceph-mon[29188]: Adjusting osd_memory_target on smithi069 to 176.1M 2024-04-25T11:32:30.689 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:32:30 smithi069 ceph-mon[29188]: Unable to set osd_memory_target on smithi069 to 184727244: error parsing value: Value '184727244' is below minimum 939524096 2024-04-25T11:32:30.690 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:32:30 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:32:30.690 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:32:30 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:32:30.690 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:32:30 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:32:30.690 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:32:30 smithi069 ceph-mon[29188]: pgmap v553: 129 pgs: 129 active+clean; 3.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:32:32.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:32:32 smithi146 ceph-mon[32821]: pgmap v554: 129 pgs: 129 active+clean; 3.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:32:32.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:32:32 smithi069 ceph-mon[29188]: pgmap v554: 129 pgs: 129 active+clean; 3.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:32:32.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:32:32 smithi152 ceph-mon[32701]: pgmap v554: 129 pgs: 129 active+clean; 3.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:32:34.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:32:34 smithi146 ceph-mon[32821]: pgmap v555: 129 pgs: 129 active+clean; 3.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:32:34.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:32:34 smithi069 ceph-mon[29188]: pgmap v555: 129 pgs: 129 active+clean; 3.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:32:34.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:32:34 smithi152 ceph-mon[32701]: pgmap v555: 129 pgs: 129 active+clean; 3.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:32:36.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:32:36 smithi146 ceph-mon[32821]: pgmap v556: 129 pgs: 129 active+clean; 3.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:32:36.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:32:36 smithi069 ceph-mon[29188]: pgmap v556: 129 pgs: 129 active+clean; 3.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:32:36.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:32:36 smithi152 ceph-mon[32701]: pgmap v556: 129 pgs: 129 active+clean; 3.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:32:38.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:32:38 smithi146 ceph-mon[32821]: pgmap v557: 129 pgs: 129 active+clean; 3.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:32:38.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:32:38 smithi069 ceph-mon[29188]: pgmap v557: 129 pgs: 129 active+clean; 3.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:32:38.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:32:38 smithi152 ceph-mon[32701]: pgmap v557: 129 pgs: 129 active+clean; 3.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:32:40.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:32:40 smithi146 ceph-mon[32821]: pgmap v558: 129 pgs: 129 active+clean; 3.9 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:32:40.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:32:40 smithi069 ceph-mon[29188]: pgmap v558: 129 pgs: 129 active+clean; 3.9 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:32:40.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:32:40 smithi152 ceph-mon[32701]: pgmap v558: 129 pgs: 129 active+clean; 3.9 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:32:42.278 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:32:42 smithi152 ceph-mon[32701]: pgmap v559: 129 pgs: 129 active+clean; 3.9 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:32:42.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:32:42 smithi146 ceph-mon[32821]: pgmap v559: 129 pgs: 129 active+clean; 3.9 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:32:42.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:32:42 smithi069 ceph-mon[29188]: pgmap v559: 129 pgs: 129 active+clean; 3.9 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:32:44.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:32:44 smithi146 ceph-mon[32821]: pgmap v560: 129 pgs: 129 active+clean; 3.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:32:44.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:32:44 smithi069 ceph-mon[29188]: pgmap v560: 129 pgs: 129 active+clean; 3.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:32:44.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:32:44 smithi152 ceph-mon[32701]: pgmap v560: 129 pgs: 129 active+clean; 3.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:32:46.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:32:46 smithi146 ceph-mon[32821]: pgmap v561: 129 pgs: 129 active+clean; 3.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:32:46.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:32:46 smithi069 ceph-mon[29188]: pgmap v561: 129 pgs: 129 active+clean; 3.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:32:46.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:32:46 smithi152 ceph-mon[32701]: pgmap v561: 129 pgs: 129 active+clean; 3.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:32:47.645 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.645 INFO:tasks.workunit.client.0.smithi069.stdout:251/600: open fd = 3 2024-04-25T11:32:47.645 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.645 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.645 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.645 INFO:tasks.workunit.client.0.smithi069.stdout:252/600: open fd = 3 2024-04-25T11:32:47.646 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.646 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.646 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.646 INFO:tasks.workunit.client.0.smithi069.stdout:253/600: open fd = 3 2024-04-25T11:32:47.646 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.646 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.646 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.646 INFO:tasks.workunit.client.0.smithi069.stdout:254/600: open fd = 3 2024-04-25T11:32:47.646 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.646 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.647 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.647 INFO:tasks.workunit.client.0.smithi069.stdout:255/600: open fd = 3 2024-04-25T11:32:47.647 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.647 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.647 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.647 INFO:tasks.workunit.client.0.smithi069.stdout:256/600: open fd = 3 2024-04-25T11:32:47.647 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.647 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.647 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.647 INFO:tasks.workunit.client.0.smithi069.stdout:257/600: open fd = 3 2024-04-25T11:32:47.647 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.647 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.647 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.648 INFO:tasks.workunit.client.0.smithi069.stdout:258/600: open fd = 3 2024-04-25T11:32:47.648 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.648 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.648 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.648 INFO:tasks.workunit.client.0.smithi069.stdout:259/600: open fd = 3 2024-04-25T11:32:47.648 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.648 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.648 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.648 INFO:tasks.workunit.client.0.smithi069.stdout:260/600: open fd = 3 2024-04-25T11:32:47.648 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.648 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.648 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.648 INFO:tasks.workunit.client.0.smithi069.stdout:261/600: open fd = 3 2024-04-25T11:32:47.648 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.649 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.649 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.649 INFO:tasks.workunit.client.0.smithi069.stdout:262/600: open fd = 3 2024-04-25T11:32:47.649 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.649 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.649 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.649 INFO:tasks.workunit.client.0.smithi069.stdout:263/600: open fd = 3 2024-04-25T11:32:47.649 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.649 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.649 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.649 INFO:tasks.workunit.client.0.smithi069.stdout:264/600: open fd = 3 2024-04-25T11:32:47.649 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.649 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.649 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.649 INFO:tasks.workunit.client.0.smithi069.stdout:265/600: open fd = 3 2024-04-25T11:32:47.649 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.650 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.650 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.650 INFO:tasks.workunit.client.0.smithi069.stdout:266/600: open fd = 3 2024-04-25T11:32:47.650 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.650 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.650 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.650 INFO:tasks.workunit.client.0.smithi069.stdout:267/600: open fd = 3 2024-04-25T11:32:47.650 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.650 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.650 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.650 INFO:tasks.workunit.client.0.smithi069.stdout:268/600: open fd = 3 2024-04-25T11:32:47.650 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.650 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.651 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.651 INFO:tasks.workunit.client.0.smithi069.stdout:269/600: open fd = 3 2024-04-25T11:32:47.651 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.651 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.651 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.651 INFO:tasks.workunit.client.0.smithi069.stdout:270/600: open fd = 3 2024-04-25T11:32:47.651 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.651 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.651 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.651 INFO:tasks.workunit.client.0.smithi069.stdout:271/600: open fd = 3 2024-04-25T11:32:47.651 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.651 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.652 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.652 INFO:tasks.workunit.client.0.smithi069.stdout:272/600: open fd = 3 2024-04-25T11:32:47.652 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.652 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.652 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.652 INFO:tasks.workunit.client.0.smithi069.stdout:273/600: open fd = 3 2024-04-25T11:32:47.652 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.652 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.652 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.653 INFO:tasks.workunit.client.0.smithi069.stdout:274/600: open fd = 3 2024-04-25T11:32:47.653 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.653 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.653 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.653 INFO:tasks.workunit.client.0.smithi069.stdout:275/600: open fd = 3 2024-04-25T11:32:47.653 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.653 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.653 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.653 INFO:tasks.workunit.client.0.smithi069.stdout:276/600: open fd = 3 2024-04-25T11:32:47.653 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.653 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.653 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.653 INFO:tasks.workunit.client.0.smithi069.stdout:277/600: open fd = 3 2024-04-25T11:32:47.653 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.653 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.653 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.653 INFO:tasks.workunit.client.0.smithi069.stdout:278/600: open fd = 3 2024-04-25T11:32:47.654 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.654 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.654 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.654 INFO:tasks.workunit.client.0.smithi069.stdout:279/600: open fd = 3 2024-04-25T11:32:47.654 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.654 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.654 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.654 INFO:tasks.workunit.client.0.smithi069.stdout:280/600: open fd = 3 2024-04-25T11:32:47.654 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.654 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.654 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.654 INFO:tasks.workunit.client.0.smithi069.stdout:281/600: open fd = 3 2024-04-25T11:32:47.654 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.654 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.655 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.655 INFO:tasks.workunit.client.0.smithi069.stdout:282/600: open fd = 3 2024-04-25T11:32:47.655 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.655 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.655 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.655 INFO:tasks.workunit.client.0.smithi069.stdout:283/600: open fd = 3 2024-04-25T11:32:47.655 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.655 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.655 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.655 INFO:tasks.workunit.client.0.smithi069.stdout:284/600: open fd = 3 2024-04-25T11:32:47.655 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.655 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.655 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.655 INFO:tasks.workunit.client.0.smithi069.stdout:285/600: open fd = 3 2024-04-25T11:32:47.655 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.656 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.656 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.656 INFO:tasks.workunit.client.0.smithi069.stdout:286/600: open fd = 3 2024-04-25T11:32:47.656 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.656 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.656 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.656 INFO:tasks.workunit.client.0.smithi069.stdout:287/600: open fd = 3 2024-04-25T11:32:47.656 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.656 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.656 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.656 INFO:tasks.workunit.client.0.smithi069.stdout:288/600: open fd = 3 2024-04-25T11:32:47.656 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.656 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.656 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.656 INFO:tasks.workunit.client.0.smithi069.stdout:289/600: open fd = 3 2024-04-25T11:32:47.656 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.657 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.657 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.657 INFO:tasks.workunit.client.0.smithi069.stdout:290/600: open fd = 3 2024-04-25T11:32:47.657 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.657 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.657 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.657 INFO:tasks.workunit.client.0.smithi069.stdout:291/600: open fd = 3 2024-04-25T11:32:47.657 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.657 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.657 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.657 INFO:tasks.workunit.client.0.smithi069.stdout:292/600: open fd = 3 2024-04-25T11:32:47.657 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.657 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.657 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.658 INFO:tasks.workunit.client.0.smithi069.stdout:293/600: open fd = 3 2024-04-25T11:32:47.658 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.658 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.658 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.658 INFO:tasks.workunit.client.0.smithi069.stdout:294/600: open fd = 3 2024-04-25T11:32:47.658 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.658 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.658 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.658 INFO:tasks.workunit.client.0.smithi069.stdout:295/600: open fd = 3 2024-04-25T11:32:47.658 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.658 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.658 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.658 INFO:tasks.workunit.client.0.smithi069.stdout:296/600: open fd = 3 2024-04-25T11:32:47.659 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.659 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.659 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.659 INFO:tasks.workunit.client.0.smithi069.stdout:297/600: open fd = 3 2024-04-25T11:32:47.659 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.659 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.659 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.659 INFO:tasks.workunit.client.0.smithi069.stdout:298/600: open fd = 3 2024-04-25T11:32:47.659 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.659 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.659 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.659 INFO:tasks.workunit.client.0.smithi069.stdout:299/600: open fd = 3 2024-04-25T11:32:47.659 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.659 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.660 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.660 INFO:tasks.workunit.client.0.smithi069.stdout:300/600: open fd = 3 2024-04-25T11:32:47.660 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.660 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.660 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.660 INFO:tasks.workunit.client.0.smithi069.stdout:301/600: open fd = 3 2024-04-25T11:32:47.660 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.660 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.660 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.660 INFO:tasks.workunit.client.0.smithi069.stdout:302/600: open fd = 3 2024-04-25T11:32:47.660 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.660 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.660 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.660 INFO:tasks.workunit.client.0.smithi069.stdout:303/600: open fd = 3 2024-04-25T11:32:47.660 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.661 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.661 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.661 INFO:tasks.workunit.client.0.smithi069.stdout:304/600: open fd = 3 2024-04-25T11:32:47.661 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.661 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.661 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.661 INFO:tasks.workunit.client.0.smithi069.stdout:305/600: open fd = 3 2024-04-25T11:32:47.661 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.661 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.661 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.661 INFO:tasks.workunit.client.0.smithi069.stdout:306/600: open fd = 3 2024-04-25T11:32:47.661 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.661 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.661 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.661 INFO:tasks.workunit.client.0.smithi069.stdout:307/600: open fd = 3 2024-04-25T11:32:47.661 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.662 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.662 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.662 INFO:tasks.workunit.client.0.smithi069.stdout:308/600: open fd = 3 2024-04-25T11:32:47.662 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.662 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.662 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.662 INFO:tasks.workunit.client.0.smithi069.stdout:309/600: open fd = 3 2024-04-25T11:32:47.662 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.662 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.662 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.662 INFO:tasks.workunit.client.0.smithi069.stdout:310/600: open fd = 3 2024-04-25T11:32:47.663 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.663 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.663 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.663 INFO:tasks.workunit.client.0.smithi069.stdout:311/600: open fd = 3 2024-04-25T11:32:47.663 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.663 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.663 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:32:47.663 INFO:tasks.workunit.client.0.smithi069.stdout:312/600: open fd = 3 2024-04-25T11:32:47.663 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:47.663 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:32:48.347 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:32:48 smithi146 ceph-mon[32821]: pgmap v562: 129 pgs: 129 active+clean; 3.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-04-25T11:32:48.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:32:48 smithi069 ceph-mon[29188]: pgmap v562: 129 pgs: 129 active+clean; 3.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-04-25T11:32:48.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:32:48 smithi152 ceph-mon[32701]: pgmap v562: 129 pgs: 129 active+clean; 3.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-04-25T11:32:50.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:32:50 smithi146 ceph-mon[32821]: pgmap v563: 129 pgs: 129 active+clean; 4.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:32:50.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:32:50 smithi069 ceph-mon[29188]: pgmap v563: 129 pgs: 129 active+clean; 4.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:32:50.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:32:50 smithi152 ceph-mon[32701]: pgmap v563: 129 pgs: 129 active+clean; 4.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:32:52.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:32:52 smithi146 ceph-mon[32821]: pgmap v564: 129 pgs: 129 active+clean; 4.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:32:52.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:32:52 smithi069 ceph-mon[29188]: pgmap v564: 129 pgs: 129 active+clean; 4.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:32:52.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:32:52 smithi152 ceph-mon[32701]: pgmap v564: 129 pgs: 129 active+clean; 4.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:32:54.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:32:54 smithi146 ceph-mon[32821]: pgmap v565: 129 pgs: 129 active+clean; 4.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:32:54.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:32:54 smithi069 ceph-mon[29188]: pgmap v565: 129 pgs: 129 active+clean; 4.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:32:54.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:32:54 smithi152 ceph-mon[32701]: pgmap v565: 129 pgs: 129 active+clean; 4.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:32:56.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:32:56 smithi146 ceph-mon[32821]: pgmap v566: 129 pgs: 129 active+clean; 4.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:32:56.441 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:32:56 smithi069 ceph-mon[29188]: pgmap v566: 129 pgs: 129 active+clean; 4.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:32:56.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:32:56 smithi152 ceph-mon[32701]: pgmap v566: 129 pgs: 129 active+clean; 4.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:32:58.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:32:58 smithi146 ceph-mon[32821]: pgmap v567: 129 pgs: 129 active+clean; 4.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:32:58.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:32:58 smithi069 ceph-mon[29188]: pgmap v567: 129 pgs: 129 active+clean; 4.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:32:58.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:32:58 smithi152 ceph-mon[32701]: pgmap v567: 129 pgs: 129 active+clean; 4.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:33:00.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:33:00 smithi146 ceph-mon[32821]: pgmap v568: 129 pgs: 129 active+clean; 4.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:33:00.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:33:00 smithi069 ceph-mon[29188]: pgmap v568: 129 pgs: 129 active+clean; 4.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:33:00.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:33:00 smithi152 ceph-mon[32701]: pgmap v568: 129 pgs: 129 active+clean; 4.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:33:02.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:33:02 smithi146 ceph-mon[32821]: pgmap v569: 129 pgs: 129 active+clean; 4.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:33:02.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:33:02 smithi069 ceph-mon[29188]: pgmap v569: 129 pgs: 129 active+clean; 4.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:33:02.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:33:02 smithi152 ceph-mon[32701]: pgmap v569: 129 pgs: 129 active+clean; 4.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:33:04.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:33:04 smithi146 ceph-mon[32821]: pgmap v570: 129 pgs: 129 active+clean; 4.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:33:04.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:33:04 smithi069 ceph-mon[29188]: pgmap v570: 129 pgs: 129 active+clean; 4.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:33:04.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:33:04 smithi152 ceph-mon[32701]: pgmap v570: 129 pgs: 129 active+clean; 4.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:33:06.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:33:06 smithi146 ceph-mon[32821]: pgmap v571: 129 pgs: 129 active+clean; 4.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:33:06.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:33:06 smithi069 ceph-mon[29188]: pgmap v571: 129 pgs: 129 active+clean; 4.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:33:06.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:33:06 smithi152 ceph-mon[32701]: pgmap v571: 129 pgs: 129 active+clean; 4.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:33:08.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:33:08 smithi146 ceph-mon[32821]: pgmap v572: 129 pgs: 129 active+clean; 4.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-04-25T11:33:08.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:33:08 smithi069 ceph-mon[29188]: pgmap v572: 129 pgs: 129 active+clean; 4.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-04-25T11:33:08.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:33:08 smithi152 ceph-mon[32701]: pgmap v572: 129 pgs: 129 active+clean; 4.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-04-25T11:33:10.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:33:10 smithi146 ceph-mon[32821]: pgmap v573: 129 pgs: 129 active+clean; 4.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:33:10.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:33:10 smithi069 ceph-mon[29188]: pgmap v573: 129 pgs: 129 active+clean; 4.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:33:10.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:33:10 smithi152 ceph-mon[32701]: pgmap v573: 129 pgs: 129 active+clean; 4.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:33:12.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:33:12 smithi146 ceph-mon[32821]: pgmap v574: 129 pgs: 129 active+clean; 4.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:33:12.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:33:12 smithi069 ceph-mon[29188]: pgmap v574: 129 pgs: 129 active+clean; 4.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:33:12.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:33:12 smithi152 ceph-mon[32701]: pgmap v574: 129 pgs: 129 active+clean; 4.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:33:14.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:33:14 smithi146 ceph-mon[32821]: pgmap v575: 129 pgs: 129 active+clean; 4.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 18 op/s 2024-04-25T11:33:14.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:33:14 smithi069 ceph-mon[29188]: pgmap v575: 129 pgs: 129 active+clean; 4.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 18 op/s 2024-04-25T11:33:14.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:33:14 smithi152 ceph-mon[32701]: pgmap v575: 129 pgs: 129 active+clean; 4.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 18 op/s 2024-04-25T11:33:16.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:33:16 smithi146 ceph-mon[32821]: pgmap v576: 129 pgs: 129 active+clean; 4.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 14 op/s 2024-04-25T11:33:16.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:33:16 smithi069 ceph-mon[29188]: pgmap v576: 129 pgs: 129 active+clean; 4.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 14 op/s 2024-04-25T11:33:16.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:33:16 smithi152 ceph-mon[32701]: pgmap v576: 129 pgs: 129 active+clean; 4.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 14 op/s 2024-04-25T11:33:18.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:33:18 smithi146 ceph-mon[32821]: pgmap v577: 129 pgs: 129 active+clean; 4.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 13 op/s 2024-04-25T11:33:18.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:33:18 smithi069 ceph-mon[29188]: pgmap v577: 129 pgs: 129 active+clean; 4.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 13 op/s 2024-04-25T11:33:18.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:33:18 smithi152 ceph-mon[32701]: pgmap v577: 129 pgs: 129 active+clean; 4.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 13 op/s 2024-04-25T11:33:20.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:33:20 smithi146 ceph-mon[32821]: pgmap v578: 129 pgs: 129 active+clean; 4.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:33:20.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:33:20 smithi069 ceph-mon[29188]: pgmap v578: 129 pgs: 129 active+clean; 4.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:33:20.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:33:20 smithi152 ceph-mon[32701]: pgmap v578: 129 pgs: 129 active+clean; 4.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:33:22.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:33:22 smithi146 ceph-mon[32821]: pgmap v579: 129 pgs: 129 active+clean; 4.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 14 op/s 2024-04-25T11:33:22.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:33:22 smithi069 ceph-mon[29188]: pgmap v579: 129 pgs: 129 active+clean; 4.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 14 op/s 2024-04-25T11:33:22.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:33:22 smithi152 ceph-mon[32701]: pgmap v579: 129 pgs: 129 active+clean; 4.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 14 op/s 2024-04-25T11:33:24.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:33:24 smithi146 ceph-mon[32821]: pgmap v580: 129 pgs: 129 active+clean; 4.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 18 op/s 2024-04-25T11:33:24.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:33:24 smithi069 ceph-mon[29188]: pgmap v580: 129 pgs: 129 active+clean; 4.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 18 op/s 2024-04-25T11:33:24.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:33:24 smithi152 ceph-mon[32701]: pgmap v580: 129 pgs: 129 active+clean; 4.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 18 op/s 2024-04-25T11:33:26.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:33:26 smithi146 ceph-mon[32821]: pgmap v581: 129 pgs: 129 active+clean; 4.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:33:26.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:33:26 smithi069 ceph-mon[29188]: pgmap v581: 129 pgs: 129 active+clean; 4.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:33:26.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:33:26 smithi152 ceph-mon[32701]: pgmap v581: 129 pgs: 129 active+clean; 4.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:33:28.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:33:28 smithi146 ceph-mon[32821]: pgmap v582: 129 pgs: 129 active+clean; 4.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-04-25T11:33:28.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:33:28 smithi069 ceph-mon[29188]: pgmap v582: 129 pgs: 129 active+clean; 4.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-04-25T11:33:28.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:33:28 smithi152 ceph-mon[32701]: pgmap v582: 129 pgs: 129 active+clean; 4.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-04-25T11:33:30.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:33:30 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:33:30.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:33:30 smithi146 ceph-mon[32821]: pgmap v583: 129 pgs: 129 active+clean; 4.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:33:30.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:33:30 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:33:30.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:33:30 smithi069 ceph-mon[29188]: pgmap v583: 129 pgs: 129 active+clean; 4.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:33:30.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:33:30 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:33:30.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:33:30 smithi152 ceph-mon[32701]: pgmap v583: 129 pgs: 129 active+clean; 4.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 19 op/s 2024-04-25T11:33:31.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:33:31 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:33:31.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:33:31 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:33:31.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:33:31 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:33:31.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:33:31 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:33:31.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:33:31 smithi146 ceph-mon[32821]: Adjusting osd_memory_target on smithi152 to 176.2M 2024-04-25T11:33:31.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:33:31 smithi146 ceph-mon[32821]: Unable to set osd_memory_target on smithi152 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-25T11:33:31.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:33:31 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:33:31.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:33:31 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:33:31.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:33:31 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:33:31.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:33:31 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:33:31.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:33:31 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:33:31.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:33:31 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:33:31.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:33:31 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:33:31.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:33:31 smithi069 ceph-mon[29188]: Adjusting osd_memory_target on smithi152 to 176.2M 2024-04-25T11:33:31.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:33:31 smithi069 ceph-mon[29188]: Unable to set osd_memory_target on smithi152 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-25T11:33:31.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:33:31 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:33:31.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:33:31 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:33:31.439 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:33:31 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:33:31.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:33:31 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:33:31.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:33:31 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:33:31.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:33:31 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:33:31.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:33:31 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:33:31.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:33:31 smithi152 ceph-mon[32701]: Adjusting osd_memory_target on smithi152 to 176.2M 2024-04-25T11:33:31.477 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:33:31 smithi152 ceph-mon[32701]: Unable to set osd_memory_target on smithi152 to 184787456: error parsing value: Value '184787456' is below minimum 939524096 2024-04-25T11:33:31.477 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:33:31 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:33:31.477 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:33:31 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:33:31.477 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:33:31 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:33:32.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:33:32 smithi146 ceph-mon[32821]: pgmap v584: 129 pgs: 129 active+clean; 4.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:33:32.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:33:32 smithi069 ceph-mon[29188]: pgmap v584: 129 pgs: 129 active+clean; 4.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:33:32.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:33:32 smithi152 ceph-mon[32701]: pgmap v584: 129 pgs: 129 active+clean; 4.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:33:34.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:33:34 smithi146 ceph-mon[32821]: pgmap v585: 129 pgs: 129 active+clean; 4.5 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:33:34.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:33:34 smithi069 ceph-mon[29188]: pgmap v585: 129 pgs: 129 active+clean; 4.5 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:33:34.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:33:34 smithi152 ceph-mon[32701]: pgmap v585: 129 pgs: 129 active+clean; 4.5 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-25T11:33:36.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:33:36 smithi146 ceph-mon[32821]: pgmap v586: 129 pgs: 129 active+clean; 4.5 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:33:36.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:33:36 smithi069 ceph-mon[29188]: pgmap v586: 129 pgs: 129 active+clean; 4.5 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:33:36.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:33:36 smithi152 ceph-mon[32701]: pgmap v586: 129 pgs: 129 active+clean; 4.5 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:33:38.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:33:38 smithi146 ceph-mon[32821]: pgmap v587: 129 pgs: 129 active+clean; 4.5 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:33:38.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:33:38 smithi069 ceph-mon[29188]: pgmap v587: 129 pgs: 129 active+clean; 4.5 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:33:38.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:33:38 smithi152 ceph-mon[32701]: pgmap v587: 129 pgs: 129 active+clean; 4.5 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:33:40.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:33:40 smithi146 ceph-mon[32821]: pgmap v588: 129 pgs: 129 active+clean; 4.5 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:33:40.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:33:40 smithi069 ceph-mon[29188]: pgmap v588: 129 pgs: 129 active+clean; 4.5 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:33:40.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:33:40 smithi152 ceph-mon[32701]: pgmap v588: 129 pgs: 129 active+clean; 4.5 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:33:42.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:33:42 smithi146 ceph-mon[32821]: pgmap v589: 129 pgs: 129 active+clean; 4.5 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:33:42.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:33:42 smithi069 ceph-mon[29188]: pgmap v589: 129 pgs: 129 active+clean; 4.5 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:33:42.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:33:42 smithi152 ceph-mon[32701]: pgmap v589: 129 pgs: 129 active+clean; 4.5 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:33:44.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:33:44 smithi146 ceph-mon[32821]: pgmap v590: 129 pgs: 129 active+clean; 4.6 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:33:44.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:33:44 smithi069 ceph-mon[29188]: pgmap v590: 129 pgs: 129 active+clean; 4.6 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:33:44.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:33:44 smithi152 ceph-mon[32701]: pgmap v590: 129 pgs: 129 active+clean; 4.6 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 19 op/s 2024-04-25T11:33:46.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:33:46 smithi146 ceph-mon[32821]: pgmap v591: 129 pgs: 129 active+clean; 4.6 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:33:46.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:33:46 smithi069 ceph-mon[29188]: pgmap v591: 129 pgs: 129 active+clean; 4.6 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:33:46.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:33:46 smithi152 ceph-mon[32701]: pgmap v591: 129 pgs: 129 active+clean; 4.6 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:33:48.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:33:48 smithi146 ceph-mon[32821]: pgmap v592: 129 pgs: 129 active+clean; 4.6 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-04-25T11:33:48.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:33:48 smithi069 ceph-mon[29188]: pgmap v592: 129 pgs: 129 active+clean; 4.6 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-04-25T11:33:48.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:33:48 smithi152 ceph-mon[32701]: pgmap v592: 129 pgs: 129 active+clean; 4.6 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-04-25T11:33:50.267 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.267 INFO:tasks.workunit.client.0.smithi069.stdout:313/600: open fd = 3 2024-04-25T11:33:50.267 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.267 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.267 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.267 INFO:tasks.workunit.client.0.smithi069.stdout:314/600: open fd = 3 2024-04-25T11:33:50.267 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.267 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.267 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.268 INFO:tasks.workunit.client.0.smithi069.stdout:315/600: open fd = 3 2024-04-25T11:33:50.268 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.268 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.268 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.268 INFO:tasks.workunit.client.0.smithi069.stdout:316/600: open fd = 3 2024-04-25T11:33:50.268 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.268 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.268 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.268 INFO:tasks.workunit.client.0.smithi069.stdout:317/600: open fd = 3 2024-04-25T11:33:50.268 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.268 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.268 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.268 INFO:tasks.workunit.client.0.smithi069.stdout:318/600: open fd = 3 2024-04-25T11:33:50.268 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.269 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.269 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.269 INFO:tasks.workunit.client.0.smithi069.stdout:319/600: open fd = 3 2024-04-25T11:33:50.269 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.269 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.269 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.269 INFO:tasks.workunit.client.0.smithi069.stdout:320/600: open fd = 3 2024-04-25T11:33:50.269 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.269 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.269 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.269 INFO:tasks.workunit.client.0.smithi069.stdout:321/600: open fd = 3 2024-04-25T11:33:50.269 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.269 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.270 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.270 INFO:tasks.workunit.client.0.smithi069.stdout:322/600: open fd = 3 2024-04-25T11:33:50.270 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.270 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.270 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.270 INFO:tasks.workunit.client.0.smithi069.stdout:323/600: open fd = 3 2024-04-25T11:33:50.270 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.270 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.270 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.270 INFO:tasks.workunit.client.0.smithi069.stdout:324/600: open fd = 3 2024-04-25T11:33:50.270 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.270 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.271 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.271 INFO:tasks.workunit.client.0.smithi069.stdout:325/600: open fd = 3 2024-04-25T11:33:50.271 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.271 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.271 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.271 INFO:tasks.workunit.client.0.smithi069.stdout:326/600: open fd = 3 2024-04-25T11:33:50.271 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.271 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.271 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.271 INFO:tasks.workunit.client.0.smithi069.stdout:327/600: open fd = 3 2024-04-25T11:33:50.271 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.271 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.271 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.271 INFO:tasks.workunit.client.0.smithi069.stdout:328/600: open fd = 3 2024-04-25T11:33:50.272 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.272 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.272 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.272 INFO:tasks.workunit.client.0.smithi069.stdout:329/600: open fd = 3 2024-04-25T11:33:50.272 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.272 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.272 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.272 INFO:tasks.workunit.client.0.smithi069.stdout:330/600: open fd = 3 2024-04-25T11:33:50.272 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.272 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.272 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.272 INFO:tasks.workunit.client.0.smithi069.stdout:331/600: open fd = 3 2024-04-25T11:33:50.272 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.273 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.273 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.273 INFO:tasks.workunit.client.0.smithi069.stdout:332/600: open fd = 3 2024-04-25T11:33:50.273 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.273 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.273 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.273 INFO:tasks.workunit.client.0.smithi069.stdout:333/600: open fd = 3 2024-04-25T11:33:50.273 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.273 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.273 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.274 INFO:tasks.workunit.client.0.smithi069.stdout:334/600: open fd = 3 2024-04-25T11:33:50.274 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.274 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.274 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.274 INFO:tasks.workunit.client.0.smithi069.stdout:335/600: open fd = 3 2024-04-25T11:33:50.274 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.274 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.274 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.274 INFO:tasks.workunit.client.0.smithi069.stdout:336/600: open fd = 3 2024-04-25T11:33:50.274 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.274 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.274 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.274 INFO:tasks.workunit.client.0.smithi069.stdout:337/600: open fd = 3 2024-04-25T11:33:50.275 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.275 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.275 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.275 INFO:tasks.workunit.client.0.smithi069.stdout:338/600: open fd = 3 2024-04-25T11:33:50.275 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.275 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.275 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.275 INFO:tasks.workunit.client.0.smithi069.stdout:339/600: open fd = 3 2024-04-25T11:33:50.275 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.275 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.275 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.275 INFO:tasks.workunit.client.0.smithi069.stdout:340/600: open fd = 3 2024-04-25T11:33:50.275 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.276 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.276 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.276 INFO:tasks.workunit.client.0.smithi069.stdout:341/600: open fd = 3 2024-04-25T11:33:50.276 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.277 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.277 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.277 INFO:tasks.workunit.client.0.smithi069.stdout:342/600: open fd = 3 2024-04-25T11:33:50.277 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.277 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.277 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.277 INFO:tasks.workunit.client.0.smithi069.stdout:343/600: open fd = 3 2024-04-25T11:33:50.277 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.277 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.277 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.277 INFO:tasks.workunit.client.0.smithi069.stdout:344/600: open fd = 3 2024-04-25T11:33:50.277 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.278 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.278 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.278 INFO:tasks.workunit.client.0.smithi069.stdout:345/600: open fd = 3 2024-04-25T11:33:50.278 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.278 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.278 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.278 INFO:tasks.workunit.client.0.smithi069.stdout:346/600: open fd = 3 2024-04-25T11:33:50.278 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.278 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.278 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.278 INFO:tasks.workunit.client.0.smithi069.stdout:347/600: open fd = 3 2024-04-25T11:33:50.279 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.279 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.279 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.279 INFO:tasks.workunit.client.0.smithi069.stdout:348/600: open fd = 3 2024-04-25T11:33:50.279 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.279 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.279 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.280 INFO:tasks.workunit.client.0.smithi069.stdout:349/600: open fd = 3 2024-04-25T11:33:50.280 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.280 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.280 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.280 INFO:tasks.workunit.client.0.smithi069.stdout:350/600: open fd = 3 2024-04-25T11:33:50.280 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.280 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.280 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.280 INFO:tasks.workunit.client.0.smithi069.stdout:351/600: open fd = 3 2024-04-25T11:33:50.280 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.280 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.280 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.280 INFO:tasks.workunit.client.0.smithi069.stdout:352/600: open fd = 3 2024-04-25T11:33:50.280 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.281 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.281 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.281 INFO:tasks.workunit.client.0.smithi069.stdout:353/600: open fd = 3 2024-04-25T11:33:50.281 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.281 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.281 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.281 INFO:tasks.workunit.client.0.smithi069.stdout:354/600: open fd = 3 2024-04-25T11:33:50.281 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.281 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.281 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.281 INFO:tasks.workunit.client.0.smithi069.stdout:355/600: open fd = 3 2024-04-25T11:33:50.281 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.281 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.281 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.281 INFO:tasks.workunit.client.0.smithi069.stdout:356/600: open fd = 3 2024-04-25T11:33:50.282 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.282 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.282 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.282 INFO:tasks.workunit.client.0.smithi069.stdout:357/600: open fd = 3 2024-04-25T11:33:50.282 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.282 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.282 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.282 INFO:tasks.workunit.client.0.smithi069.stdout:358/600: open fd = 3 2024-04-25T11:33:50.282 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.282 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.282 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.282 INFO:tasks.workunit.client.0.smithi069.stdout:359/600: open fd = 3 2024-04-25T11:33:50.282 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.283 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.283 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.283 INFO:tasks.workunit.client.0.smithi069.stdout:360/600: open fd = 3 2024-04-25T11:33:50.283 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.283 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.283 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.283 INFO:tasks.workunit.client.0.smithi069.stdout:361/600: open fd = 3 2024-04-25T11:33:50.283 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.283 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.283 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.283 INFO:tasks.workunit.client.0.smithi069.stdout:362/600: open fd = 3 2024-04-25T11:33:50.283 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.284 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.284 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.284 INFO:tasks.workunit.client.0.smithi069.stdout:363/600: open fd = 3 2024-04-25T11:33:50.284 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.284 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.284 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.284 INFO:tasks.workunit.client.0.smithi069.stdout:364/600: open fd = 3 2024-04-25T11:33:50.284 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.284 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.284 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.284 INFO:tasks.workunit.client.0.smithi069.stdout:365/600: open fd = 3 2024-04-25T11:33:50.284 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.284 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.284 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.284 INFO:tasks.workunit.client.0.smithi069.stdout:366/600: open fd = 3 2024-04-25T11:33:50.285 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.285 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.285 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.285 INFO:tasks.workunit.client.0.smithi069.stdout:367/600: open fd = 3 2024-04-25T11:33:50.285 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.285 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.285 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.285 INFO:tasks.workunit.client.0.smithi069.stdout:368/600: open fd = 3 2024-04-25T11:33:50.285 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.285 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.285 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.285 INFO:tasks.workunit.client.0.smithi069.stdout:369/600: open fd = 3 2024-04-25T11:33:50.285 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.285 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.285 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.286 INFO:tasks.workunit.client.0.smithi069.stdout:370/600: open fd = 3 2024-04-25T11:33:50.286 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.286 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.286 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.286 INFO:tasks.workunit.client.0.smithi069.stdout:371/600: open fd = 3 2024-04-25T11:33:50.286 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.286 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.286 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.286 INFO:tasks.workunit.client.0.smithi069.stdout:372/600: open fd = 3 2024-04-25T11:33:50.286 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.286 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.286 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.287 INFO:tasks.workunit.client.0.smithi069.stdout:373/600: open fd = 3 2024-04-25T11:33:50.287 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.287 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.287 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.287 INFO:tasks.workunit.client.0.smithi069.stdout:374/600: open fd = 3 2024-04-25T11:33:50.287 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.288 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:33:50.288 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:33:50.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:33:50 smithi146 ceph-mon[32821]: pgmap v593: 129 pgs: 129 active+clean; 4.6 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-04-25T11:33:50.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:33:50 smithi069 ceph-mon[29188]: pgmap v593: 129 pgs: 129 active+clean; 4.6 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-04-25T11:33:50.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:33:50 smithi152 ceph-mon[32701]: pgmap v593: 129 pgs: 129 active+clean; 4.6 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-04-25T11:33:52.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:33:52 smithi146 ceph-mon[32821]: pgmap v594: 129 pgs: 129 active+clean; 4.7 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-04-25T11:33:52.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:33:52 smithi069 ceph-mon[29188]: pgmap v594: 129 pgs: 129 active+clean; 4.7 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-04-25T11:33:52.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:33:52 smithi152 ceph-mon[32701]: pgmap v594: 129 pgs: 129 active+clean; 4.7 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-04-25T11:33:54.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:33:54 smithi146 ceph-mon[32821]: pgmap v595: 129 pgs: 129 active+clean; 4.7 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-04-25T11:33:54.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:33:54 smithi069 ceph-mon[29188]: pgmap v595: 129 pgs: 129 active+clean; 4.7 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-04-25T11:33:54.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:33:54 smithi152 ceph-mon[32701]: pgmap v595: 129 pgs: 129 active+clean; 4.7 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-04-25T11:33:56.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:33:56 smithi146 ceph-mon[32821]: pgmap v596: 129 pgs: 129 active+clean; 4.7 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 KiB/s wr, 14 op/s 2024-04-25T11:33:56.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:33:56 smithi069 ceph-mon[29188]: pgmap v596: 129 pgs: 129 active+clean; 4.7 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 KiB/s wr, 14 op/s 2024-04-25T11:33:56.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:33:56 smithi152 ceph-mon[32701]: pgmap v596: 129 pgs: 129 active+clean; 4.7 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 KiB/s wr, 14 op/s 2024-04-25T11:33:58.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:33:58 smithi146 ceph-mon[32821]: pgmap v597: 129 pgs: 129 active+clean; 4.7 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-25T11:33:58.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:33:58 smithi069 ceph-mon[29188]: pgmap v597: 129 pgs: 129 active+clean; 4.7 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-25T11:33:58.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:33:58 smithi152 ceph-mon[32701]: pgmap v597: 129 pgs: 129 active+clean; 4.7 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-25T11:34:00.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:34:00 smithi146 ceph-mon[32821]: pgmap v598: 129 pgs: 129 active+clean; 4.7 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 18 op/s 2024-04-25T11:34:00.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:34:00 smithi069 ceph-mon[29188]: pgmap v598: 129 pgs: 129 active+clean; 4.7 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 18 op/s 2024-04-25T11:34:00.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:34:00 smithi152 ceph-mon[32701]: pgmap v598: 129 pgs: 129 active+clean; 4.7 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 18 op/s 2024-04-25T11:34:02.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:34:02 smithi146 ceph-mon[32821]: pgmap v599: 129 pgs: 129 active+clean; 4.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-04-25T11:34:02.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:34:02 smithi069 ceph-mon[29188]: pgmap v599: 129 pgs: 129 active+clean; 4.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-04-25T11:34:02.476 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:34:02 smithi152 ceph-mon[32701]: pgmap v599: 129 pgs: 129 active+clean; 4.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-04-25T11:34:04.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:34:04 smithi146 ceph-mon[32821]: pgmap v600: 129 pgs: 129 active+clean; 4.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:34:04.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:34:04 smithi069 ceph-mon[29188]: pgmap v600: 129 pgs: 129 active+clean; 4.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:34:04.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:34:04 smithi152 ceph-mon[32701]: pgmap v600: 129 pgs: 129 active+clean; 4.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:34:06.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:34:06 smithi146 ceph-mon[32821]: pgmap v601: 129 pgs: 129 active+clean; 4.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:34:06.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:34:06 smithi069 ceph-mon[29188]: pgmap v601: 129 pgs: 129 active+clean; 4.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:34:06.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:34:06 smithi152 ceph-mon[32701]: pgmap v601: 129 pgs: 129 active+clean; 4.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:34:08.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:34:08 smithi146 ceph-mon[32821]: pgmap v602: 129 pgs: 129 active+clean; 4.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:34:08.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:34:08 smithi069 ceph-mon[29188]: pgmap v602: 129 pgs: 129 active+clean; 4.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:34:08.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:34:08 smithi152 ceph-mon[32701]: pgmap v602: 129 pgs: 129 active+clean; 4.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:34:10.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:34:10 smithi146 ceph-mon[32821]: pgmap v603: 129 pgs: 129 active+clean; 4.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 17 op/s 2024-04-25T11:34:10.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:34:10 smithi069 ceph-mon[29188]: pgmap v603: 129 pgs: 129 active+clean; 4.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 17 op/s 2024-04-25T11:34:10.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:34:10 smithi152 ceph-mon[32701]: pgmap v603: 129 pgs: 129 active+clean; 4.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 17 op/s 2024-04-25T11:34:12.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:34:12 smithi146 ceph-mon[32821]: pgmap v604: 129 pgs: 129 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:34:12.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:34:12 smithi069 ceph-mon[29188]: pgmap v604: 129 pgs: 129 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:34:12.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:34:12 smithi152 ceph-mon[32701]: pgmap v604: 129 pgs: 129 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:34:14.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:34:14 smithi146 ceph-mon[32821]: pgmap v605: 129 pgs: 129 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:34:14.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:34:14 smithi069 ceph-mon[29188]: pgmap v605: 129 pgs: 129 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:34:14.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:34:14 smithi152 ceph-mon[32701]: pgmap v605: 129 pgs: 129 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:34:16.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:34:16 smithi146 ceph-mon[32821]: pgmap v606: 129 pgs: 129 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:34:16.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:34:16 smithi069 ceph-mon[29188]: pgmap v606: 129 pgs: 129 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:34:16.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:34:16 smithi152 ceph-mon[32701]: pgmap v606: 129 pgs: 129 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:34:18.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:34:18 smithi146 ceph-mon[32821]: pgmap v607: 129 pgs: 129 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:34:18.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:34:18 smithi069 ceph-mon[29188]: pgmap v607: 129 pgs: 129 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:34:18.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:34:18 smithi152 ceph-mon[32701]: pgmap v607: 129 pgs: 129 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:34:20.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:34:20 smithi146 ceph-mon[32821]: pgmap v608: 129 pgs: 129 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-25T11:34:20.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:34:20 smithi069 ceph-mon[29188]: pgmap v608: 129 pgs: 129 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-25T11:34:20.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:34:20 smithi152 ceph-mon[32701]: pgmap v608: 129 pgs: 129 active+clean; 4.9 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-25T11:34:22.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:34:22 smithi146 ceph-mon[32821]: pgmap v609: 129 pgs: 129 active+clean; 5.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:34:22.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:34:22 smithi069 ceph-mon[29188]: pgmap v609: 129 pgs: 129 active+clean; 5.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:34:22.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:34:22 smithi152 ceph-mon[32701]: pgmap v609: 129 pgs: 129 active+clean; 5.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:34:24.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:34:24 smithi146 ceph-mon[32821]: pgmap v610: 129 pgs: 129 active+clean; 5.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:34:24.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:34:24 smithi069 ceph-mon[29188]: pgmap v610: 129 pgs: 129 active+clean; 5.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:34:24.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:34:24 smithi152 ceph-mon[32701]: pgmap v610: 129 pgs: 129 active+clean; 5.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:34:26.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:34:26 smithi146 ceph-mon[32821]: pgmap v611: 129 pgs: 129 active+clean; 5.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:34:26.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:34:26 smithi069 ceph-mon[29188]: pgmap v611: 129 pgs: 129 active+clean; 5.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:34:26.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:34:26 smithi152 ceph-mon[32701]: pgmap v611: 129 pgs: 129 active+clean; 5.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:34:28.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:34:28 smithi146 ceph-mon[32821]: pgmap v612: 129 pgs: 129 active+clean; 5.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:34:28.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:34:28 smithi069 ceph-mon[29188]: pgmap v612: 129 pgs: 129 active+clean; 5.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:34:28.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:34:28 smithi152 ceph-mon[32701]: pgmap v612: 129 pgs: 129 active+clean; 5.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:34:30.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:34:30 smithi146 ceph-mon[32821]: pgmap v613: 129 pgs: 129 active+clean; 5.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-25T11:34:30.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:34:30 smithi069 ceph-mon[29188]: pgmap v613: 129 pgs: 129 active+clean; 5.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-25T11:34:30.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:34:30 smithi152 ceph-mon[32701]: pgmap v613: 129 pgs: 129 active+clean; 5.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-25T11:34:31.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:34:31 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:34:31.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:34:31 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:34:31.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:34:31 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:34:32.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:34:32 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:34:32.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:34:32 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:34:32.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:34:32 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:34:32.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:34:32 smithi146 ceph-mon[32821]: pgmap v614: 129 pgs: 129 active+clean; 5.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-04-25T11:34:32.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:34:32 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:34:32.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:34:32 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:34:32.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:34:32 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:34:32.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:34:32 smithi069 ceph-mon[29188]: pgmap v614: 129 pgs: 129 active+clean; 5.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-04-25T11:34:32.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:34:32 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:34:32.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:34:32 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:34:32.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:34:32 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:34:32.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:34:32 smithi152 ceph-mon[32701]: pgmap v614: 129 pgs: 129 active+clean; 5.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-04-25T11:34:34.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:34:34 smithi146 ceph-mon[32821]: pgmap v615: 129 pgs: 129 active+clean; 5.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:34:34.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:34:34 smithi069 ceph-mon[29188]: pgmap v615: 129 pgs: 129 active+clean; 5.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:34:34.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:34:34 smithi152 ceph-mon[32701]: pgmap v615: 129 pgs: 129 active+clean; 5.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:34:36.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:34:36 smithi146 ceph-mon[32821]: pgmap v616: 129 pgs: 129 active+clean; 5.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:34:36.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:34:36 smithi069 ceph-mon[29188]: pgmap v616: 129 pgs: 129 active+clean; 5.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:34:36.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:34:36 smithi152 ceph-mon[32701]: pgmap v616: 129 pgs: 129 active+clean; 5.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:34:38.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:34:38 smithi146 ceph-mon[32821]: pgmap v617: 129 pgs: 129 active+clean; 5.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 17 op/s 2024-04-25T11:34:38.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:34:38 smithi069 ceph-mon[29188]: pgmap v617: 129 pgs: 129 active+clean; 5.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 17 op/s 2024-04-25T11:34:38.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:34:38 smithi152 ceph-mon[32701]: pgmap v617: 129 pgs: 129 active+clean; 5.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 17 op/s 2024-04-25T11:34:40.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:34:40 smithi146 ceph-mon[32821]: pgmap v618: 129 pgs: 129 active+clean; 5.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:34:40.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:34:40 smithi069 ceph-mon[29188]: pgmap v618: 129 pgs: 129 active+clean; 5.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:34:40.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:34:40 smithi152 ceph-mon[32701]: pgmap v618: 129 pgs: 129 active+clean; 5.1 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:34:42.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:34:42 smithi146 ceph-mon[32821]: pgmap v619: 129 pgs: 129 active+clean; 5.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-04-25T11:34:42.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:34:42 smithi069 ceph-mon[29188]: pgmap v619: 129 pgs: 129 active+clean; 5.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-04-25T11:34:42.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:34:42 smithi152 ceph-mon[32701]: pgmap v619: 129 pgs: 129 active+clean; 5.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-04-25T11:34:44.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:34:44 smithi146 ceph-mon[32821]: pgmap v620: 129 pgs: 129 active+clean; 5.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:34:44.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:34:44 smithi069 ceph-mon[29188]: pgmap v620: 129 pgs: 129 active+clean; 5.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:34:44.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:34:44 smithi152 ceph-mon[32701]: pgmap v620: 129 pgs: 129 active+clean; 5.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:34:46.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:34:46 smithi146 ceph-mon[32821]: pgmap v621: 129 pgs: 129 active+clean; 5.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:34:46.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:34:46 smithi069 ceph-mon[29188]: pgmap v621: 129 pgs: 129 active+clean; 5.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:34:46.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:34:46 smithi152 ceph-mon[32701]: pgmap v621: 129 pgs: 129 active+clean; 5.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:34:48.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:34:48 smithi146 ceph-mon[32821]: pgmap v622: 129 pgs: 129 active+clean; 5.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:34:48.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:34:48 smithi069 ceph-mon[29188]: pgmap v622: 129 pgs: 129 active+clean; 5.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:34:48.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:34:48 smithi152 ceph-mon[32701]: pgmap v622: 129 pgs: 129 active+clean; 5.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:34:51.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:34:51 smithi146 ceph-mon[32821]: pgmap v623: 129 pgs: 129 active+clean; 5.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.7 KiB/s wr, 15 op/s 2024-04-25T11:34:51.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:34:51 smithi069 ceph-mon[29188]: pgmap v623: 129 pgs: 129 active+clean; 5.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.7 KiB/s wr, 15 op/s 2024-04-25T11:34:51.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:34:51 smithi152 ceph-mon[32701]: pgmap v623: 129 pgs: 129 active+clean; 5.2 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.7 KiB/s wr, 15 op/s 2024-04-25T11:34:52.840 INFO:tasks.workunit.client.0.smithi069.stdout:375/600: open fd = 3 2024-04-25T11:34:52.840 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.840 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.840 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.840 INFO:tasks.workunit.client.0.smithi069.stdout:376/600: open fd = 3 2024-04-25T11:34:52.840 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.840 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.840 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.841 INFO:tasks.workunit.client.0.smithi069.stdout:377/600: open fd = 3 2024-04-25T11:34:52.841 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.841 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.841 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.841 INFO:tasks.workunit.client.0.smithi069.stdout:378/600: open fd = 3 2024-04-25T11:34:52.841 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.841 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.841 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.841 INFO:tasks.workunit.client.0.smithi069.stdout:379/600: open fd = 3 2024-04-25T11:34:52.841 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.841 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.841 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.842 INFO:tasks.workunit.client.0.smithi069.stdout:380/600: open fd = 3 2024-04-25T11:34:52.842 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.842 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.842 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.842 INFO:tasks.workunit.client.0.smithi069.stdout:381/600: open fd = 3 2024-04-25T11:34:52.842 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.842 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.842 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.842 INFO:tasks.workunit.client.0.smithi069.stdout:382/600: open fd = 3 2024-04-25T11:34:52.842 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.842 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.843 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.843 INFO:tasks.workunit.client.0.smithi069.stdout:383/600: open fd = 3 2024-04-25T11:34:52.843 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.843 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.843 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.843 INFO:tasks.workunit.client.0.smithi069.stdout:384/600: open fd = 3 2024-04-25T11:34:52.843 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.843 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.843 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.843 INFO:tasks.workunit.client.0.smithi069.stdout:385/600: open fd = 3 2024-04-25T11:34:52.843 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.843 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.843 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.843 INFO:tasks.workunit.client.0.smithi069.stdout:386/600: open fd = 3 2024-04-25T11:34:52.844 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.844 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.844 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.844 INFO:tasks.workunit.client.0.smithi069.stdout:387/600: open fd = 3 2024-04-25T11:34:52.844 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.844 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.844 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.844 INFO:tasks.workunit.client.0.smithi069.stdout:388/600: open fd = 3 2024-04-25T11:34:52.844 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.844 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.844 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.844 INFO:tasks.workunit.client.0.smithi069.stdout:389/600: open fd = 3 2024-04-25T11:34:52.844 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.845 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.845 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.845 INFO:tasks.workunit.client.0.smithi069.stdout:390/600: open fd = 3 2024-04-25T11:34:52.845 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.845 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.845 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.845 INFO:tasks.workunit.client.0.smithi069.stdout:391/600: open fd = 3 2024-04-25T11:34:52.845 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.845 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.845 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.845 INFO:tasks.workunit.client.0.smithi069.stdout:392/600: open fd = 3 2024-04-25T11:34:52.845 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.845 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.845 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.845 INFO:tasks.workunit.client.0.smithi069.stdout:393/600: open fd = 3 2024-04-25T11:34:52.846 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.846 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.846 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.846 INFO:tasks.workunit.client.0.smithi069.stdout:394/600: open fd = 3 2024-04-25T11:34:52.846 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.846 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.846 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.846 INFO:tasks.workunit.client.0.smithi069.stdout:395/600: open fd = 3 2024-04-25T11:34:52.846 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.846 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.846 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.846 INFO:tasks.workunit.client.0.smithi069.stdout:396/600: open fd = 3 2024-04-25T11:34:52.846 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.846 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.847 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.847 INFO:tasks.workunit.client.0.smithi069.stdout:397/600: open fd = 3 2024-04-25T11:34:52.847 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.847 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.847 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.847 INFO:tasks.workunit.client.0.smithi069.stdout:398/600: open fd = 3 2024-04-25T11:34:52.847 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.847 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.847 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.847 INFO:tasks.workunit.client.0.smithi069.stdout:399/600: open fd = 3 2024-04-25T11:34:52.848 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.848 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.848 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.848 INFO:tasks.workunit.client.0.smithi069.stdout:400/600: open fd = 3 2024-04-25T11:34:52.848 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.848 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.848 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.848 INFO:tasks.workunit.client.0.smithi069.stdout:401/600: open fd = 3 2024-04-25T11:34:52.848 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.848 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.848 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.848 INFO:tasks.workunit.client.0.smithi069.stdout:402/600: open fd = 3 2024-04-25T11:34:52.848 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.848 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.849 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.849 INFO:tasks.workunit.client.0.smithi069.stdout:403/600: open fd = 3 2024-04-25T11:34:52.849 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.849 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.849 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.849 INFO:tasks.workunit.client.0.smithi069.stdout:404/600: open fd = 3 2024-04-25T11:34:52.849 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.849 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.849 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.849 INFO:tasks.workunit.client.0.smithi069.stdout:405/600: open fd = 3 2024-04-25T11:34:52.849 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.849 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.849 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.849 INFO:tasks.workunit.client.0.smithi069.stdout:406/600: open fd = 3 2024-04-25T11:34:52.849 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.849 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.849 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.850 INFO:tasks.workunit.client.0.smithi069.stdout:407/600: open fd = 3 2024-04-25T11:34:52.850 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.850 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.850 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.850 INFO:tasks.workunit.client.0.smithi069.stdout:408/600: open fd = 3 2024-04-25T11:34:52.850 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.850 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.850 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.850 INFO:tasks.workunit.client.0.smithi069.stdout:409/600: open fd = 3 2024-04-25T11:34:52.850 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.850 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.850 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.850 INFO:tasks.workunit.client.0.smithi069.stdout:410/600: open fd = 3 2024-04-25T11:34:52.850 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.851 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.851 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.851 INFO:tasks.workunit.client.0.smithi069.stdout:411/600: open fd = 3 2024-04-25T11:34:52.851 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.851 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.851 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.851 INFO:tasks.workunit.client.0.smithi069.stdout:412/600: open fd = 3 2024-04-25T11:34:52.851 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.851 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.851 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.851 INFO:tasks.workunit.client.0.smithi069.stdout:413/600: open fd = 3 2024-04-25T11:34:52.851 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.851 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.852 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.852 INFO:tasks.workunit.client.0.smithi069.stdout:414/600: open fd = 3 2024-04-25T11:34:52.852 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.852 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.852 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.852 INFO:tasks.workunit.client.0.smithi069.stdout:415/600: open fd = 3 2024-04-25T11:34:52.852 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.852 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.852 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.852 INFO:tasks.workunit.client.0.smithi069.stdout:416/600: open fd = 3 2024-04-25T11:34:52.852 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.852 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.852 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.852 INFO:tasks.workunit.client.0.smithi069.stdout:417/600: open fd = 3 2024-04-25T11:34:52.853 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.853 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.853 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.853 INFO:tasks.workunit.client.0.smithi069.stdout:418/600: open fd = 3 2024-04-25T11:34:52.853 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.853 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.853 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.853 INFO:tasks.workunit.client.0.smithi069.stdout:419/600: open fd = 3 2024-04-25T11:34:52.853 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.853 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.853 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.853 INFO:tasks.workunit.client.0.smithi069.stdout:420/600: open fd = 3 2024-04-25T11:34:52.853 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.854 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.854 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.854 INFO:tasks.workunit.client.0.smithi069.stdout:421/600: open fd = 3 2024-04-25T11:34:52.854 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.854 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.854 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.854 INFO:tasks.workunit.client.0.smithi069.stdout:422/600: open fd = 3 2024-04-25T11:34:52.854 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.854 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.854 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.854 INFO:tasks.workunit.client.0.smithi069.stdout:423/600: open fd = 3 2024-04-25T11:34:52.855 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.855 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.855 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.855 INFO:tasks.workunit.client.0.smithi069.stdout:424/600: open fd = 3 2024-04-25T11:34:52.855 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.855 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.855 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.855 INFO:tasks.workunit.client.0.smithi069.stdout:425/600: open fd = 3 2024-04-25T11:34:52.855 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.855 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.856 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.856 INFO:tasks.workunit.client.0.smithi069.stdout:426/600: open fd = 3 2024-04-25T11:34:52.856 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.856 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.856 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.856 INFO:tasks.workunit.client.0.smithi069.stdout:427/600: open fd = 3 2024-04-25T11:34:52.856 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.856 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.856 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.856 INFO:tasks.workunit.client.0.smithi069.stdout:428/600: open fd = 3 2024-04-25T11:34:52.856 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.856 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.856 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.857 INFO:tasks.workunit.client.0.smithi069.stdout:429/600: open fd = 3 2024-04-25T11:34:52.857 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.857 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.857 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.857 INFO:tasks.workunit.client.0.smithi069.stdout:430/600: open fd = 3 2024-04-25T11:34:52.857 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.857 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.857 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.857 INFO:tasks.workunit.client.0.smithi069.stdout:431/600: open fd = 3 2024-04-25T11:34:52.857 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.857 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.857 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.857 INFO:tasks.workunit.client.0.smithi069.stdout:432/600: open fd = 3 2024-04-25T11:34:52.857 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.857 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.858 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.858 INFO:tasks.workunit.client.0.smithi069.stdout:433/600: open fd = 3 2024-04-25T11:34:52.858 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.858 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.858 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.858 INFO:tasks.workunit.client.0.smithi069.stdout:434/600: open fd = 3 2024-04-25T11:34:52.858 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.858 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.858 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.859 INFO:tasks.workunit.client.0.smithi069.stdout:435/600: open fd = 3 2024-04-25T11:34:52.859 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.859 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.859 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:52.859 INFO:tasks.workunit.client.0.smithi069.stdout:436/600: open fd = 3 2024-04-25T11:34:52.859 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.859 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:34:52.859 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:34:53.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:34:53 smithi146 ceph-mon[32821]: pgmap v624: 129 pgs: 129 active+clean; 5.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:34:53.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:34:53 smithi069 ceph-mon[29188]: pgmap v624: 129 pgs: 129 active+clean; 5.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:34:53.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:34:53 smithi152 ceph-mon[32701]: pgmap v624: 129 pgs: 129 active+clean; 5.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:34:55.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:34:55 smithi146 ceph-mon[32821]: pgmap v625: 129 pgs: 129 active+clean; 5.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-25T11:34:55.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:34:55 smithi069 ceph-mon[29188]: pgmap v625: 129 pgs: 129 active+clean; 5.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-25T11:34:55.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:34:55 smithi152 ceph-mon[32701]: pgmap v625: 129 pgs: 129 active+clean; 5.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-25T11:34:57.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:34:57 smithi146 ceph-mon[32821]: pgmap v626: 129 pgs: 129 active+clean; 5.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 14 op/s 2024-04-25T11:34:57.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:34:57 smithi069 ceph-mon[29188]: pgmap v626: 129 pgs: 129 active+clean; 5.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 14 op/s 2024-04-25T11:34:57.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:34:57 smithi152 ceph-mon[32701]: pgmap v626: 129 pgs: 129 active+clean; 5.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 14 op/s 2024-04-25T11:34:59.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:34:59 smithi146 ceph-mon[32821]: pgmap v627: 129 pgs: 129 active+clean; 5.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:34:59.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:34:59 smithi069 ceph-mon[29188]: pgmap v627: 129 pgs: 129 active+clean; 5.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:34:59.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:34:59 smithi152 ceph-mon[32701]: pgmap v627: 129 pgs: 129 active+clean; 5.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:35:01.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:35:01 smithi146 ceph-mon[32821]: pgmap v628: 129 pgs: 129 active+clean; 5.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:35:01.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:35:01 smithi069 ceph-mon[29188]: pgmap v628: 129 pgs: 129 active+clean; 5.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:35:01.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:35:01 smithi152 ceph-mon[32701]: pgmap v628: 129 pgs: 129 active+clean; 5.3 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:35:03.343 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:35:03 smithi146 ceph-mon[32821]: pgmap v629: 129 pgs: 129 active+clean; 5.4 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-04-25T11:35:03.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:35:03 smithi069 ceph-mon[29188]: pgmap v629: 129 pgs: 129 active+clean; 5.4 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-04-25T11:35:03.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:35:03 smithi152 ceph-mon[32701]: pgmap v629: 129 pgs: 129 active+clean; 5.4 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-04-25T11:35:05.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:35:05 smithi146 ceph-mon[32821]: pgmap v630: 129 pgs: 129 active+clean; 5.4 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:35:05.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:35:05 smithi069 ceph-mon[29188]: pgmap v630: 129 pgs: 129 active+clean; 5.4 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:35:05.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:35:05 smithi152 ceph-mon[32701]: pgmap v630: 129 pgs: 129 active+clean; 5.4 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:35:07.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:35:07 smithi146 ceph-mon[32821]: pgmap v631: 129 pgs: 129 active+clean; 5.4 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:35:07.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:35:07 smithi069 ceph-mon[29188]: pgmap v631: 129 pgs: 129 active+clean; 5.4 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:35:07.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:35:07 smithi152 ceph-mon[32701]: pgmap v631: 129 pgs: 129 active+clean; 5.4 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:35:09.346 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:35:09 smithi146 ceph-mon[32821]: pgmap v632: 129 pgs: 129 active+clean; 5.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:35:09.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:35:09 smithi069 ceph-mon[29188]: pgmap v632: 129 pgs: 129 active+clean; 5.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:35:09.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:35:09 smithi152 ceph-mon[32701]: pgmap v632: 129 pgs: 129 active+clean; 5.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:35:11.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:35:11 smithi146 ceph-mon[32821]: pgmap v633: 129 pgs: 129 active+clean; 5.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:35:11.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:35:11 smithi069 ceph-mon[29188]: pgmap v633: 129 pgs: 129 active+clean; 5.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:35:11.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:35:11 smithi152 ceph-mon[32701]: pgmap v633: 129 pgs: 129 active+clean; 5.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:35:13.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:35:13 smithi146 ceph-mon[32821]: pgmap v634: 129 pgs: 129 active+clean; 5.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:35:13.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:35:13 smithi069 ceph-mon[29188]: pgmap v634: 129 pgs: 129 active+clean; 5.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:35:13.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:35:13 smithi152 ceph-mon[32701]: pgmap v634: 129 pgs: 129 active+clean; 5.5 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:35:15.345 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:35:15 smithi146 ceph-mon[32821]: pgmap v635: 129 pgs: 129 active+clean; 5.5 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:35:15.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:35:15 smithi069 ceph-mon[29188]: pgmap v635: 129 pgs: 129 active+clean; 5.5 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:35:15.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:35:15 smithi152 ceph-mon[32701]: pgmap v635: 129 pgs: 129 active+clean; 5.5 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:35:17.343 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:35:17 smithi146 ceph-mon[32821]: pgmap v636: 129 pgs: 129 active+clean; 5.5 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:35:17.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:35:17 smithi069 ceph-mon[29188]: pgmap v636: 129 pgs: 129 active+clean; 5.5 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:35:17.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:35:17 smithi152 ceph-mon[32701]: pgmap v636: 129 pgs: 129 active+clean; 5.5 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:35:19.343 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:35:19 smithi146 ceph-mon[32821]: pgmap v637: 129 pgs: 129 active+clean; 5.6 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:35:19.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:35:19 smithi069 ceph-mon[29188]: pgmap v637: 129 pgs: 129 active+clean; 5.6 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:35:19.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:35:19 smithi152 ceph-mon[32701]: pgmap v637: 129 pgs: 129 active+clean; 5.6 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:35:21.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:35:21 smithi069 ceph-mon[29188]: pgmap v638: 129 pgs: 129 active+clean; 5.6 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-25T11:35:21.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:35:21 smithi152 ceph-mon[32701]: pgmap v638: 129 pgs: 129 active+clean; 5.6 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-25T11:35:21.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:35:21 smithi146 ceph-mon[32821]: pgmap v638: 129 pgs: 129 active+clean; 5.6 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-25T11:35:23.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:35:23 smithi069 ceph-mon[29188]: pgmap v639: 129 pgs: 129 active+clean; 5.6 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:35:23.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:35:23 smithi152 ceph-mon[32701]: pgmap v639: 129 pgs: 129 active+clean; 5.6 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:35:23.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:35:23 smithi146 ceph-mon[32821]: pgmap v639: 129 pgs: 129 active+clean; 5.6 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:35:25.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:35:25 smithi069 ceph-mon[29188]: pgmap v640: 129 pgs: 129 active+clean; 5.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:35:25.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:35:25 smithi152 ceph-mon[32701]: pgmap v640: 129 pgs: 129 active+clean; 5.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:35:25.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:35:25 smithi146 ceph-mon[32821]: pgmap v640: 129 pgs: 129 active+clean; 5.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:35:27.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:35:27 smithi069 ceph-mon[29188]: pgmap v641: 129 pgs: 129 active+clean; 5.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:35:27.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:35:27 smithi152 ceph-mon[32701]: pgmap v641: 129 pgs: 129 active+clean; 5.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:35:27.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:35:27 smithi146 ceph-mon[32821]: pgmap v641: 129 pgs: 129 active+clean; 5.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:35:29.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:35:29 smithi069 ceph-mon[29188]: pgmap v642: 129 pgs: 129 active+clean; 5.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:35:29.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:35:29 smithi152 ceph-mon[32701]: pgmap v642: 129 pgs: 129 active+clean; 5.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:35:29.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:35:29 smithi146 ceph-mon[32821]: pgmap v642: 129 pgs: 129 active+clean; 5.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:35:31.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:35:31 smithi069 ceph-mon[29188]: pgmap v643: 129 pgs: 129 active+clean; 5.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-25T11:35:31.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:35:31 smithi152 ceph-mon[32701]: pgmap v643: 129 pgs: 129 active+clean; 5.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-25T11:35:31.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:35:31 smithi146 ceph-mon[32821]: pgmap v643: 129 pgs: 129 active+clean; 5.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-25T11:35:32.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:35:32 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:35:32.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:35:32 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:35:32.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:35:32 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:35:33.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:35:33 smithi069 ceph-mon[29188]: pgmap v644: 129 pgs: 129 active+clean; 5.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:35:33.445 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:35:33 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:35:33.445 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:35:33 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:35:33.445 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:35:33 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:35:33.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:35:33 smithi152 ceph-mon[32701]: pgmap v644: 129 pgs: 129 active+clean; 5.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:35:33.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:35:33 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:35:33.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:35:33 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:35:33.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:35:33 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:35:33.594 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:35:33 smithi146 ceph-mon[32821]: pgmap v644: 129 pgs: 129 active+clean; 5.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:35:33.594 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:35:33 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:35:33.594 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:35:33 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:35:33.594 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:35:33 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:35:35.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:35:35 smithi069 ceph-mon[29188]: pgmap v645: 129 pgs: 129 active+clean; 5.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:35:35.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:35:35 smithi152 ceph-mon[32701]: pgmap v645: 129 pgs: 129 active+clean; 5.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:35:35.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:35:35 smithi146 ceph-mon[32821]: pgmap v645: 129 pgs: 129 active+clean; 5.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:35:37.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:35:37 smithi069 ceph-mon[29188]: pgmap v646: 129 pgs: 129 active+clean; 5.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:35:37.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:35:37 smithi152 ceph-mon[32701]: pgmap v646: 129 pgs: 129 active+clean; 5.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:35:37.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:35:37 smithi146 ceph-mon[32821]: pgmap v646: 129 pgs: 129 active+clean; 5.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:35:39.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:35:39 smithi152 ceph-mon[32701]: pgmap v647: 129 pgs: 129 active+clean; 5.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:35:39.594 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:35:39 smithi146 ceph-mon[32821]: pgmap v647: 129 pgs: 129 active+clean; 5.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:35:39.688 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:35:39 smithi069 ceph-mon[29188]: pgmap v647: 129 pgs: 129 active+clean; 5.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:35:41.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:35:41 smithi152 ceph-mon[32701]: pgmap v648: 129 pgs: 129 active+clean; 5.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-25T11:35:41.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:35:41 smithi146 ceph-mon[32821]: pgmap v648: 129 pgs: 129 active+clean; 5.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-25T11:35:41.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:35:41 smithi069 ceph-mon[29188]: pgmap v648: 129 pgs: 129 active+clean; 5.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-25T11:35:43.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:35:43 smithi152 ceph-mon[32701]: pgmap v649: 129 pgs: 129 active+clean; 5.8 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:35:43.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:35:43 smithi146 ceph-mon[32821]: pgmap v649: 129 pgs: 129 active+clean; 5.8 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:35:43.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:35:43 smithi069 ceph-mon[29188]: pgmap v649: 129 pgs: 129 active+clean; 5.8 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:35:45.477 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:35:45 smithi152 ceph-mon[32701]: pgmap v650: 129 pgs: 129 active+clean; 5.8 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:35:45.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:35:45 smithi146 ceph-mon[32821]: pgmap v650: 129 pgs: 129 active+clean; 5.8 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:35:45.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:35:45 smithi069 ceph-mon[29188]: pgmap v650: 129 pgs: 129 active+clean; 5.8 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:35:47.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:35:47 smithi152 ceph-mon[32701]: pgmap v651: 129 pgs: 129 active+clean; 5.8 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:35:47.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:35:47 smithi146 ceph-mon[32821]: pgmap v651: 129 pgs: 129 active+clean; 5.8 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:35:47.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:35:47 smithi069 ceph-mon[29188]: pgmap v651: 129 pgs: 129 active+clean; 5.8 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:35:49.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:35:49 smithi152 ceph-mon[32701]: pgmap v652: 129 pgs: 129 active+clean; 5.9 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:35:49.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:35:49 smithi146 ceph-mon[32821]: pgmap v652: 129 pgs: 129 active+clean; 5.9 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:35:49.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:35:49 smithi069 ceph-mon[29188]: pgmap v652: 129 pgs: 129 active+clean; 5.9 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:35:51.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:35:51 smithi152 ceph-mon[32701]: pgmap v653: 129 pgs: 129 active+clean; 5.9 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-25T11:35:51.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:35:51 smithi146 ceph-mon[32821]: pgmap v653: 129 pgs: 129 active+clean; 5.9 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-25T11:35:51.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:35:51 smithi069 ceph-mon[29188]: pgmap v653: 129 pgs: 129 active+clean; 5.9 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-25T11:35:53.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:35:53 smithi146 ceph-mon[32821]: pgmap v654: 129 pgs: 129 active+clean; 5.9 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:35:53.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:35:53 smithi069 ceph-mon[29188]: pgmap v654: 129 pgs: 129 active+clean; 5.9 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:35:53.725 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:35:53 smithi152 ceph-mon[32701]: pgmap v654: 129 pgs: 129 active+clean; 5.9 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:35:55.438 INFO:tasks.workunit.client.0.smithi069.stdout:437/600: open fd = 3 2024-04-25T11:35:55.438 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.438 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.438 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.438 INFO:tasks.workunit.client.0.smithi069.stdout:438/600: open fd = 3 2024-04-25T11:35:55.438 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.438 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.439 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.439 INFO:tasks.workunit.client.0.smithi069.stdout:439/600: open fd = 3 2024-04-25T11:35:55.439 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.439 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.439 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.439 INFO:tasks.workunit.client.0.smithi069.stdout:440/600: open fd = 3 2024-04-25T11:35:55.439 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.439 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.439 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.439 INFO:tasks.workunit.client.0.smithi069.stdout:441/600: open fd = 3 2024-04-25T11:35:55.439 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.439 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.439 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.439 INFO:tasks.workunit.client.0.smithi069.stdout:442/600: open fd = 3 2024-04-25T11:35:55.439 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.440 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.440 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.440 INFO:tasks.workunit.client.0.smithi069.stdout:443/600: open fd = 3 2024-04-25T11:35:55.440 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.440 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.440 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.440 INFO:tasks.workunit.client.0.smithi069.stdout:444/600: open fd = 3 2024-04-25T11:35:55.440 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.440 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.440 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.440 INFO:tasks.workunit.client.0.smithi069.stdout:445/600: open fd = 3 2024-04-25T11:35:55.440 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.440 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.440 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.441 INFO:tasks.workunit.client.0.smithi069.stdout:446/600: open fd = 3 2024-04-25T11:35:55.441 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.441 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.441 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.441 INFO:tasks.workunit.client.0.smithi069.stdout:447/600: open fd = 3 2024-04-25T11:35:55.441 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.441 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.441 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.441 INFO:tasks.workunit.client.0.smithi069.stdout:448/600: open fd = 3 2024-04-25T11:35:55.441 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.441 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.441 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.441 INFO:tasks.workunit.client.0.smithi069.stdout:449/600: open fd = 3 2024-04-25T11:35:55.441 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.441 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.442 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.442 INFO:tasks.workunit.client.0.smithi069.stdout:450/600: open fd = 3 2024-04-25T11:35:55.442 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.442 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.442 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.442 INFO:tasks.workunit.client.0.smithi069.stdout:451/600: open fd = 3 2024-04-25T11:35:55.442 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.442 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.442 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.442 INFO:tasks.workunit.client.0.smithi069.stdout:452/600: open fd = 3 2024-04-25T11:35:55.442 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.442 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.442 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.442 INFO:tasks.workunit.client.0.smithi069.stdout:453/600: open fd = 3 2024-04-25T11:35:55.443 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.443 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.443 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.443 INFO:tasks.workunit.client.0.smithi069.stdout:454/600: open fd = 3 2024-04-25T11:35:55.443 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.443 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.443 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.443 INFO:tasks.workunit.client.0.smithi069.stdout:455/600: open fd = 3 2024-04-25T11:35:55.443 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.443 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.443 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.443 INFO:tasks.workunit.client.0.smithi069.stdout:456/600: open fd = 3 2024-04-25T11:35:55.443 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.443 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.443 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.444 INFO:tasks.workunit.client.0.smithi069.stdout:457/600: open fd = 3 2024-04-25T11:35:55.444 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.444 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.444 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.444 INFO:tasks.workunit.client.0.smithi069.stdout:458/600: open fd = 3 2024-04-25T11:35:55.444 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.444 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.444 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.444 INFO:tasks.workunit.client.0.smithi069.stdout:459/600: open fd = 3 2024-04-25T11:35:55.444 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.444 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.444 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.444 INFO:tasks.workunit.client.0.smithi069.stdout:460/600: open fd = 3 2024-04-25T11:35:55.444 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.444 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.445 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.445 INFO:tasks.workunit.client.0.smithi069.stdout:461/600: open fd = 3 2024-04-25T11:35:55.445 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.445 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.445 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.445 INFO:tasks.workunit.client.0.smithi069.stdout:462/600: open fd = 3 2024-04-25T11:35:55.445 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.445 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.445 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.445 INFO:tasks.workunit.client.0.smithi069.stdout:463/600: open fd = 3 2024-04-25T11:35:55.445 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.445 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.445 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.445 INFO:tasks.workunit.client.0.smithi069.stdout:464/600: open fd = 3 2024-04-25T11:35:55.446 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.446 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.446 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.446 INFO:tasks.workunit.client.0.smithi069.stdout:465/600: open fd = 3 2024-04-25T11:35:55.446 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.446 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.446 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.446 INFO:tasks.workunit.client.0.smithi069.stdout:466/600: open fd = 3 2024-04-25T11:35:55.446 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.446 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.446 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.446 INFO:tasks.workunit.client.0.smithi069.stdout:467/600: open fd = 3 2024-04-25T11:35:55.446 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.446 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.447 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.447 INFO:tasks.workunit.client.0.smithi069.stdout:468/600: open fd = 3 2024-04-25T11:35:55.447 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.447 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.447 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.447 INFO:tasks.workunit.client.0.smithi069.stdout:469/600: open fd = 3 2024-04-25T11:35:55.447 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.447 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.447 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.447 INFO:tasks.workunit.client.0.smithi069.stdout:470/600: open fd = 3 2024-04-25T11:35:55.447 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.447 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.447 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.447 INFO:tasks.workunit.client.0.smithi069.stdout:471/600: open fd = 3 2024-04-25T11:35:55.447 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.448 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.448 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.448 INFO:tasks.workunit.client.0.smithi069.stdout:472/600: open fd = 3 2024-04-25T11:35:55.448 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.448 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.448 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.448 INFO:tasks.workunit.client.0.smithi069.stdout:473/600: open fd = 3 2024-04-25T11:35:55.448 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.448 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.448 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.448 INFO:tasks.workunit.client.0.smithi069.stdout:474/600: open fd = 3 2024-04-25T11:35:55.448 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.448 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.448 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.448 INFO:tasks.workunit.client.0.smithi069.stdout:475/600: open fd = 3 2024-04-25T11:35:55.449 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.449 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.449 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.449 INFO:tasks.workunit.client.0.smithi069.stdout:476/600: open fd = 3 2024-04-25T11:35:55.449 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.449 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.449 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.449 INFO:tasks.workunit.client.0.smithi069.stdout:477/600: open fd = 3 2024-04-25T11:35:55.449 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.449 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.449 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.449 INFO:tasks.workunit.client.0.smithi069.stdout:478/600: open fd = 3 2024-04-25T11:35:55.449 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.449 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.449 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.450 INFO:tasks.workunit.client.0.smithi069.stdout:479/600: open fd = 3 2024-04-25T11:35:55.450 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.450 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.450 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.450 INFO:tasks.workunit.client.0.smithi069.stdout:480/600: open fd = 3 2024-04-25T11:35:55.450 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.450 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.450 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.450 INFO:tasks.workunit.client.0.smithi069.stdout:481/600: open fd = 3 2024-04-25T11:35:55.450 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.450 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.450 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.450 INFO:tasks.workunit.client.0.smithi069.stdout:482/600: open fd = 3 2024-04-25T11:35:55.450 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.451 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.451 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.451 INFO:tasks.workunit.client.0.smithi069.stdout:483/600: open fd = 3 2024-04-25T11:35:55.451 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.451 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.451 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.451 INFO:tasks.workunit.client.0.smithi069.stdout:484/600: open fd = 3 2024-04-25T11:35:55.451 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.451 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.451 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.451 INFO:tasks.workunit.client.0.smithi069.stdout:485/600: open fd = 3 2024-04-25T11:35:55.451 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.451 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.451 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.452 INFO:tasks.workunit.client.0.smithi069.stdout:486/600: open fd = 3 2024-04-25T11:35:55.452 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.452 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.452 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.452 INFO:tasks.workunit.client.0.smithi069.stdout:487/600: open fd = 3 2024-04-25T11:35:55.452 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.452 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.452 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.452 INFO:tasks.workunit.client.0.smithi069.stdout:488/600: open fd = 3 2024-04-25T11:35:55.452 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.452 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.452 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.452 INFO:tasks.workunit.client.0.smithi069.stdout:489/600: open fd = 3 2024-04-25T11:35:55.453 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.453 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.453 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.453 INFO:tasks.workunit.client.0.smithi069.stdout:490/600: open fd = 3 2024-04-25T11:35:55.453 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.453 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.453 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.453 INFO:tasks.workunit.client.0.smithi069.stdout:491/600: open fd = 3 2024-04-25T11:35:55.453 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.453 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.453 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.453 INFO:tasks.workunit.client.0.smithi069.stdout:492/600: open fd = 3 2024-04-25T11:35:55.453 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.453 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.453 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.453 INFO:tasks.workunit.client.0.smithi069.stdout:493/600: open fd = 3 2024-04-25T11:35:55.454 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.454 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.454 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.454 INFO:tasks.workunit.client.0.smithi069.stdout:494/600: open fd = 3 2024-04-25T11:35:55.454 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.454 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.454 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.454 INFO:tasks.workunit.client.0.smithi069.stdout:495/600: open fd = 3 2024-04-25T11:35:55.454 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.454 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.454 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.454 INFO:tasks.workunit.client.0.smithi069.stdout:496/600: open fd = 3 2024-04-25T11:35:55.454 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.454 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.454 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.455 INFO:tasks.workunit.client.0.smithi069.stdout:497/600: open fd = 3 2024-04-25T11:35:55.455 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.455 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.455 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.455 INFO:tasks.workunit.client.0.smithi069.stdout:498/600: open fd = 3 2024-04-25T11:35:55.455 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.455 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:35:55.455 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:35:55.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:35:55 smithi146 ceph-mon[32821]: pgmap v655: 129 pgs: 129 active+clean; 5.9 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:35:55.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:35:55 smithi069 ceph-mon[29188]: pgmap v655: 129 pgs: 129 active+clean; 5.9 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:35:55.725 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:35:55 smithi152 ceph-mon[32701]: pgmap v655: 129 pgs: 129 active+clean; 5.9 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:35:57.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:35:57 smithi146 ceph-mon[32821]: pgmap v656: 129 pgs: 129 active+clean; 5.9 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:35:57.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:35:57 smithi069 ceph-mon[29188]: pgmap v656: 129 pgs: 129 active+clean; 5.9 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:35:57.725 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:35:57 smithi152 ceph-mon[32701]: pgmap v656: 129 pgs: 129 active+clean; 5.9 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:35:59.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:35:59 smithi146 ceph-mon[32821]: pgmap v657: 129 pgs: 129 active+clean; 6.0 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:35:59.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:35:59 smithi069 ceph-mon[29188]: pgmap v657: 129 pgs: 129 active+clean; 6.0 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:35:59.725 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:35:59 smithi152 ceph-mon[32701]: pgmap v657: 129 pgs: 129 active+clean; 6.0 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:36:01.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:36:01 smithi146 ceph-mon[32821]: pgmap v658: 129 pgs: 129 active+clean; 6.0 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-25T11:36:01.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:36:01 smithi069 ceph-mon[29188]: pgmap v658: 129 pgs: 129 active+clean; 6.0 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-25T11:36:01.725 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:36:01 smithi152 ceph-mon[32701]: pgmap v658: 129 pgs: 129 active+clean; 6.0 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-25T11:36:03.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:36:03 smithi146 ceph-mon[32821]: pgmap v659: 129 pgs: 129 active+clean; 6.0 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:36:03.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:36:03 smithi069 ceph-mon[29188]: pgmap v659: 129 pgs: 129 active+clean; 6.0 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:36:03.725 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:36:03 smithi152 ceph-mon[32701]: pgmap v659: 129 pgs: 129 active+clean; 6.0 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:36:05.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:36:05 smithi146 ceph-mon[32821]: pgmap v660: 129 pgs: 129 active+clean; 6.0 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:36:05.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:36:05 smithi069 ceph-mon[29188]: pgmap v660: 129 pgs: 129 active+clean; 6.0 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:36:05.725 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:36:05 smithi152 ceph-mon[32701]: pgmap v660: 129 pgs: 129 active+clean; 6.0 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:36:07.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:36:07 smithi146 ceph-mon[32821]: pgmap v661: 129 pgs: 129 active+clean; 6.0 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:36:07.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:36:07 smithi069 ceph-mon[29188]: pgmap v661: 129 pgs: 129 active+clean; 6.0 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:36:07.725 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:36:07 smithi152 ceph-mon[32701]: pgmap v661: 129 pgs: 129 active+clean; 6.0 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:36:09.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:36:09 smithi146 ceph-mon[32821]: pgmap v662: 129 pgs: 129 active+clean; 6.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:36:09.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:36:09 smithi069 ceph-mon[29188]: pgmap v662: 129 pgs: 129 active+clean; 6.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:36:09.725 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:36:09 smithi152 ceph-mon[32701]: pgmap v662: 129 pgs: 129 active+clean; 6.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:36:11.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:36:11 smithi146 ceph-mon[32821]: pgmap v663: 129 pgs: 129 active+clean; 6.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-25T11:36:11.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:36:11 smithi069 ceph-mon[29188]: pgmap v663: 129 pgs: 129 active+clean; 6.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-25T11:36:11.725 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:36:11 smithi152 ceph-mon[32701]: pgmap v663: 129 pgs: 129 active+clean; 6.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-25T11:36:13.527 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:36:13 smithi152 ceph-mon[32701]: pgmap v664: 129 pgs: 129 active+clean; 6.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:36:13.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:36:13 smithi146 ceph-mon[32821]: pgmap v664: 129 pgs: 129 active+clean; 6.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:36:13.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:36:13 smithi069 ceph-mon[29188]: pgmap v664: 129 pgs: 129 active+clean; 6.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:36:15.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:36:15 smithi146 ceph-mon[32821]: pgmap v665: 129 pgs: 129 active+clean; 6.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:36:15.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:36:15 smithi069 ceph-mon[29188]: pgmap v665: 129 pgs: 129 active+clean; 6.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:36:15.725 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:36:15 smithi152 ceph-mon[32701]: pgmap v665: 129 pgs: 129 active+clean; 6.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:36:17.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:36:17 smithi146 ceph-mon[32821]: pgmap v666: 129 pgs: 129 active+clean; 6.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:36:17.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:36:17 smithi069 ceph-mon[29188]: pgmap v666: 129 pgs: 129 active+clean; 6.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:36:17.725 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:36:17 smithi152 ceph-mon[32701]: pgmap v666: 129 pgs: 129 active+clean; 6.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:36:19.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:36:19 smithi146 ceph-mon[32821]: pgmap v667: 129 pgs: 129 active+clean; 6.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:36:19.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:36:19 smithi069 ceph-mon[29188]: pgmap v667: 129 pgs: 129 active+clean; 6.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:36:19.725 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:36:19 smithi152 ceph-mon[32701]: pgmap v667: 129 pgs: 129 active+clean; 6.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:36:21.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:36:21 smithi146 ceph-mon[32821]: pgmap v668: 129 pgs: 129 active+clean; 6.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-25T11:36:21.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:36:21 smithi069 ceph-mon[29188]: pgmap v668: 129 pgs: 129 active+clean; 6.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-25T11:36:21.724 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:36:21 smithi152 ceph-mon[32701]: pgmap v668: 129 pgs: 129 active+clean; 6.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-25T11:36:23.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:36:23 smithi146 ceph-mon[32821]: pgmap v669: 129 pgs: 129 active+clean; 6.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:36:23.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:36:23 smithi069 ceph-mon[29188]: pgmap v669: 129 pgs: 129 active+clean; 6.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:36:23.724 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:36:23 smithi152 ceph-mon[32701]: pgmap v669: 129 pgs: 129 active+clean; 6.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:36:25.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:36:25 smithi146 ceph-mon[32821]: pgmap v670: 129 pgs: 129 active+clean; 6.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:36:25.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:36:25 smithi069 ceph-mon[29188]: pgmap v670: 129 pgs: 129 active+clean; 6.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:36:25.724 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:36:25 smithi152 ceph-mon[32701]: pgmap v670: 129 pgs: 129 active+clean; 6.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:36:27.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:36:27 smithi146 ceph-mon[32821]: pgmap v671: 129 pgs: 129 active+clean; 6.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:36:27.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:36:27 smithi069 ceph-mon[29188]: pgmap v671: 129 pgs: 129 active+clean; 6.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:36:27.724 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:36:27 smithi152 ceph-mon[32701]: pgmap v671: 129 pgs: 129 active+clean; 6.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:36:29.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:36:29 smithi146 ceph-mon[32821]: pgmap v672: 129 pgs: 129 active+clean; 6.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:36:29.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:36:29 smithi069 ceph-mon[29188]: pgmap v672: 129 pgs: 129 active+clean; 6.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:36:29.725 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:36:29 smithi152 ceph-mon[32701]: pgmap v672: 129 pgs: 129 active+clean; 6.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:36:31.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:36:31 smithi146 ceph-mon[32821]: pgmap v673: 129 pgs: 129 active+clean; 6.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-25T11:36:31.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:36:31 smithi069 ceph-mon[29188]: pgmap v673: 129 pgs: 129 active+clean; 6.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-25T11:36:31.725 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:36:31 smithi152 ceph-mon[32701]: pgmap v673: 129 pgs: 129 active+clean; 6.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-25T11:36:33.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:36:33 smithi152 ceph-mon[32701]: pgmap v674: 129 pgs: 129 active+clean; 6.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:36:33.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:36:33 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:36:33.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:36:33 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:36:33.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:36:33 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:36:33.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:36:33 smithi146 ceph-mon[32821]: pgmap v674: 129 pgs: 129 active+clean; 6.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:36:33.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:36:33 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:36:33.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:36:33 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:36:33.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:36:33 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:36:33.620 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:36:33 smithi069 ceph-mon[29188]: pgmap v674: 129 pgs: 129 active+clean; 6.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:36:33.620 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:36:33 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:36:33.620 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:36:33 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:36:33.620 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:36:33 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:36:35.343 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:36:34 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:36:35.343 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:36:34 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:36:35.343 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:36:34 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:36:35.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:36:34 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:36:35.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:36:34 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:36:35.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:36:34 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:36:35.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:36:34 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:36:35.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:36:34 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:36:35.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:36:34 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:36:35.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:36:34 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:36:35.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:36:34 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:36:35.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:36:34 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:36:35.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:36:34 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:36:35.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:36:34 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:36:35.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:36:34 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:36:35.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:36:34 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:36:35.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:36:34 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:36:35.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:36:34 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:36:36.343 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:36:36 smithi146 ceph-mon[32821]: pgmap v675: 129 pgs: 129 active+clean; 6.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-25T11:36:36.343 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:36:36 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:36:36.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:36:36 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:36:36.344 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:36:36 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:36:36.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:36:36 smithi069 ceph-mon[29188]: pgmap v675: 129 pgs: 129 active+clean; 6.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-25T11:36:36.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:36:36 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:36:36.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:36:36 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:36:36.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:36:36 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:36:36.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:36:36 smithi152 ceph-mon[32701]: pgmap v675: 129 pgs: 129 active+clean; 6.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-25T11:36:36.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:36:36 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:36:36.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:36:36 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:36:36.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:36:36 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:36:38.343 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:36:38 smithi146 ceph-mon[32821]: pgmap v676: 129 pgs: 129 active+clean; 6.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 14 op/s 2024-04-25T11:36:38.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:36:38 smithi069 ceph-mon[29188]: pgmap v676: 129 pgs: 129 active+clean; 6.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 14 op/s 2024-04-25T11:36:38.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:36:38 smithi152 ceph-mon[32701]: pgmap v676: 129 pgs: 129 active+clean; 6.3 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 14 op/s 2024-04-25T11:36:40.343 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:36:40 smithi146 ceph-mon[32821]: pgmap v677: 129 pgs: 129 active+clean; 6.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:36:40.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:36:40 smithi069 ceph-mon[29188]: pgmap v677: 129 pgs: 129 active+clean; 6.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:36:40.474 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:36:40 smithi152 ceph-mon[32701]: pgmap v677: 129 pgs: 129 active+clean; 6.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:36:42.343 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:36:42 smithi146 ceph-mon[32821]: pgmap v678: 129 pgs: 129 active+clean; 6.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.7 KiB/s wr, 15 op/s 2024-04-25T11:36:42.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:36:42 smithi069 ceph-mon[29188]: pgmap v678: 129 pgs: 129 active+clean; 6.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.7 KiB/s wr, 15 op/s 2024-04-25T11:36:42.474 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:36:42 smithi152 ceph-mon[32701]: pgmap v678: 129 pgs: 129 active+clean; 6.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.7 KiB/s wr, 15 op/s 2024-04-25T11:36:44.276 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:36:44 smithi152 ceph-mon[32701]: pgmap v679: 129 pgs: 129 active+clean; 6.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:36:44.343 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:36:44 smithi146 ceph-mon[32821]: pgmap v679: 129 pgs: 129 active+clean; 6.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:36:44.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:36:44 smithi069 ceph-mon[29188]: pgmap v679: 129 pgs: 129 active+clean; 6.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:36:46.343 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:36:46 smithi146 ceph-mon[32821]: pgmap v680: 129 pgs: 129 active+clean; 6.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:36:46.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:36:46 smithi069 ceph-mon[29188]: pgmap v680: 129 pgs: 129 active+clean; 6.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:36:46.474 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:36:46 smithi152 ceph-mon[32701]: pgmap v680: 129 pgs: 129 active+clean; 6.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:36:48.343 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:36:48 smithi146 ceph-mon[32821]: pgmap v681: 129 pgs: 129 active+clean; 6.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:36:48.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:36:48 smithi069 ceph-mon[29188]: pgmap v681: 129 pgs: 129 active+clean; 6.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:36:48.474 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:36:48 smithi152 ceph-mon[32701]: pgmap v681: 129 pgs: 129 active+clean; 6.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:36:49.343 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:36:49 smithi146 ceph-mon[32821]: pgmap v682: 129 pgs: 129 active+clean; 6.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:36:49.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:36:49 smithi069 ceph-mon[29188]: pgmap v682: 129 pgs: 129 active+clean; 6.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:36:49.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:36:49 smithi152 ceph-mon[32701]: pgmap v682: 129 pgs: 129 active+clean; 6.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:36:51.343 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:36:51 smithi146 ceph-mon[32821]: pgmap v683: 129 pgs: 129 active+clean; 6.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-25T11:36:51.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:36:51 smithi069 ceph-mon[29188]: pgmap v683: 129 pgs: 129 active+clean; 6.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-25T11:36:51.474 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:36:51 smithi152 ceph-mon[32701]: pgmap v683: 129 pgs: 129 active+clean; 6.5 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-25T11:36:53.343 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:36:53 smithi146 ceph-mon[32821]: pgmap v684: 129 pgs: 129 active+clean; 6.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:36:53.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:36:53 smithi069 ceph-mon[29188]: pgmap v684: 129 pgs: 129 active+clean; 6.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:36:53.474 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:36:53 smithi152 ceph-mon[32701]: pgmap v684: 129 pgs: 129 active+clean; 6.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:36:55.343 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:36:55 smithi146 ceph-mon[32821]: pgmap v685: 129 pgs: 129 active+clean; 6.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:36:55.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:36:55 smithi069 ceph-mon[29188]: pgmap v685: 129 pgs: 129 active+clean; 6.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:36:55.474 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:36:55 smithi152 ceph-mon[32701]: pgmap v685: 129 pgs: 129 active+clean; 6.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:36:57.343 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:36:57 smithi146 ceph-mon[32821]: pgmap v686: 129 pgs: 129 active+clean; 6.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:36:57.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:36:57 smithi069 ceph-mon[29188]: pgmap v686: 129 pgs: 129 active+clean; 6.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:36:57.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:36:57 smithi152 ceph-mon[32701]: pgmap v686: 129 pgs: 129 active+clean; 6.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:36:58.032 INFO:tasks.workunit.client.0.smithi069.stdout:499/600: open fd = 3 2024-04-25T11:36:58.032 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.032 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.032 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.032 INFO:tasks.workunit.client.0.smithi069.stdout:500/600: open fd = 3 2024-04-25T11:36:58.032 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.033 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.033 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.033 INFO:tasks.workunit.client.0.smithi069.stdout:501/600: open fd = 3 2024-04-25T11:36:58.033 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.033 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.033 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.033 INFO:tasks.workunit.client.0.smithi069.stdout:502/600: open fd = 3 2024-04-25T11:36:58.033 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.033 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.033 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.033 INFO:tasks.workunit.client.0.smithi069.stdout:503/600: open fd = 3 2024-04-25T11:36:58.033 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.033 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.033 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.033 INFO:tasks.workunit.client.0.smithi069.stdout:504/600: open fd = 3 2024-04-25T11:36:58.034 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.034 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.034 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.034 INFO:tasks.workunit.client.0.smithi069.stdout:505/600: open fd = 3 2024-04-25T11:36:58.034 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.034 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.034 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.034 INFO:tasks.workunit.client.0.smithi069.stdout:506/600: open fd = 3 2024-04-25T11:36:58.034 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.034 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.034 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.034 INFO:tasks.workunit.client.0.smithi069.stdout:507/600: open fd = 3 2024-04-25T11:36:58.034 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.035 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.035 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.035 INFO:tasks.workunit.client.0.smithi069.stdout:508/600: open fd = 3 2024-04-25T11:36:58.035 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.035 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.035 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.035 INFO:tasks.workunit.client.0.smithi069.stdout:509/600: open fd = 3 2024-04-25T11:36:58.035 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.035 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.035 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.035 INFO:tasks.workunit.client.0.smithi069.stdout:510/600: open fd = 3 2024-04-25T11:36:58.035 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.035 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.035 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.035 INFO:tasks.workunit.client.0.smithi069.stdout:511/600: open fd = 3 2024-04-25T11:36:58.036 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.036 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.036 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.036 INFO:tasks.workunit.client.0.smithi069.stdout:512/600: open fd = 3 2024-04-25T11:36:58.036 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.036 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.036 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.036 INFO:tasks.workunit.client.0.smithi069.stdout:513/600: open fd = 3 2024-04-25T11:36:58.036 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.036 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.036 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.036 INFO:tasks.workunit.client.0.smithi069.stdout:514/600: open fd = 3 2024-04-25T11:36:58.036 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.036 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.036 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.037 INFO:tasks.workunit.client.0.smithi069.stdout:515/600: open fd = 3 2024-04-25T11:36:58.037 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.037 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.037 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.037 INFO:tasks.workunit.client.0.smithi069.stdout:516/600: open fd = 3 2024-04-25T11:36:58.037 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.037 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.037 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.037 INFO:tasks.workunit.client.0.smithi069.stdout:517/600: open fd = 3 2024-04-25T11:36:58.037 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.037 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.037 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.037 INFO:tasks.workunit.client.0.smithi069.stdout:518/600: open fd = 3 2024-04-25T11:36:58.037 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.037 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.038 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.038 INFO:tasks.workunit.client.0.smithi069.stdout:519/600: open fd = 3 2024-04-25T11:36:58.038 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.038 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.038 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.038 INFO:tasks.workunit.client.0.smithi069.stdout:520/600: open fd = 3 2024-04-25T11:36:58.038 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.038 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.038 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.038 INFO:tasks.workunit.client.0.smithi069.stdout:521/600: open fd = 3 2024-04-25T11:36:58.038 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.038 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.038 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.038 INFO:tasks.workunit.client.0.smithi069.stdout:522/600: open fd = 3 2024-04-25T11:36:58.038 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.039 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.039 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.039 INFO:tasks.workunit.client.0.smithi069.stdout:523/600: open fd = 3 2024-04-25T11:36:58.039 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.039 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.039 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.039 INFO:tasks.workunit.client.0.smithi069.stdout:524/600: open fd = 3 2024-04-25T11:36:58.039 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.039 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.039 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.039 INFO:tasks.workunit.client.0.smithi069.stdout:525/600: open fd = 3 2024-04-25T11:36:58.039 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.039 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.039 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.039 INFO:tasks.workunit.client.0.smithi069.stdout:526/600: open fd = 3 2024-04-25T11:36:58.040 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.040 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.040 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.040 INFO:tasks.workunit.client.0.smithi069.stdout:527/600: open fd = 3 2024-04-25T11:36:58.040 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.040 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.040 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.040 INFO:tasks.workunit.client.0.smithi069.stdout:528/600: open fd = 3 2024-04-25T11:36:58.040 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.040 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.040 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.040 INFO:tasks.workunit.client.0.smithi069.stdout:529/600: open fd = 3 2024-04-25T11:36:58.040 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.040 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.041 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.041 INFO:tasks.workunit.client.0.smithi069.stdout:530/600: open fd = 3 2024-04-25T11:36:58.041 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.041 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.041 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.041 INFO:tasks.workunit.client.0.smithi069.stdout:531/600: open fd = 3 2024-04-25T11:36:58.041 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.041 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.041 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.041 INFO:tasks.workunit.client.0.smithi069.stdout:532/600: open fd = 3 2024-04-25T11:36:58.041 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.041 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.041 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.041 INFO:tasks.workunit.client.0.smithi069.stdout:533/600: open fd = 3 2024-04-25T11:36:58.041 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.041 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.042 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.042 INFO:tasks.workunit.client.0.smithi069.stdout:534/600: open fd = 3 2024-04-25T11:36:58.042 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.042 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.042 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.042 INFO:tasks.workunit.client.0.smithi069.stdout:535/600: open fd = 3 2024-04-25T11:36:58.042 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.042 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.042 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.042 INFO:tasks.workunit.client.0.smithi069.stdout:536/600: open fd = 3 2024-04-25T11:36:58.042 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.042 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.042 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.042 INFO:tasks.workunit.client.0.smithi069.stdout:537/600: open fd = 3 2024-04-25T11:36:58.042 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.043 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.043 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.043 INFO:tasks.workunit.client.0.smithi069.stdout:538/600: open fd = 3 2024-04-25T11:36:58.043 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.043 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.043 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.043 INFO:tasks.workunit.client.0.smithi069.stdout:539/600: open fd = 3 2024-04-25T11:36:58.043 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.043 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.043 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.043 INFO:tasks.workunit.client.0.smithi069.stdout:540/600: open fd = 3 2024-04-25T11:36:58.043 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.043 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.043 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.043 INFO:tasks.workunit.client.0.smithi069.stdout:541/600: open fd = 3 2024-04-25T11:36:58.044 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.044 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.044 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.044 INFO:tasks.workunit.client.0.smithi069.stdout:542/600: open fd = 3 2024-04-25T11:36:58.044 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.044 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.044 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.044 INFO:tasks.workunit.client.0.smithi069.stdout:543/600: open fd = 3 2024-04-25T11:36:58.044 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.044 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.044 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.044 INFO:tasks.workunit.client.0.smithi069.stdout:544/600: open fd = 3 2024-04-25T11:36:58.044 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.044 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.044 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.045 INFO:tasks.workunit.client.0.smithi069.stdout:545/600: open fd = 3 2024-04-25T11:36:58.045 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.045 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.045 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.045 INFO:tasks.workunit.client.0.smithi069.stdout:546/600: open fd = 3 2024-04-25T11:36:58.045 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.045 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.045 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.045 INFO:tasks.workunit.client.0.smithi069.stdout:547/600: open fd = 3 2024-04-25T11:36:58.045 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.045 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.045 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.045 INFO:tasks.workunit.client.0.smithi069.stdout:548/600: open fd = 3 2024-04-25T11:36:58.046 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.046 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.046 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.046 INFO:tasks.workunit.client.0.smithi069.stdout:549/600: open fd = 3 2024-04-25T11:36:58.046 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.046 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.046 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.046 INFO:tasks.workunit.client.0.smithi069.stdout:550/600: open fd = 3 2024-04-25T11:36:58.046 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.046 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.046 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.046 INFO:tasks.workunit.client.0.smithi069.stdout:551/600: open fd = 3 2024-04-25T11:36:58.046 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.046 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.047 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.047 INFO:tasks.workunit.client.0.smithi069.stdout:552/600: open fd = 3 2024-04-25T11:36:58.047 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.047 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.047 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.047 INFO:tasks.workunit.client.0.smithi069.stdout:553/600: open fd = 3 2024-04-25T11:36:58.047 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.047 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.047 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.047 INFO:tasks.workunit.client.0.smithi069.stdout:554/600: open fd = 3 2024-04-25T11:36:58.047 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.047 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.047 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.047 INFO:tasks.workunit.client.0.smithi069.stdout:555/600: open fd = 3 2024-04-25T11:36:58.047 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.047 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.048 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.048 INFO:tasks.workunit.client.0.smithi069.stdout:556/600: open fd = 3 2024-04-25T11:36:58.048 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.048 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.048 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.048 INFO:tasks.workunit.client.0.smithi069.stdout:557/600: open fd = 3 2024-04-25T11:36:58.048 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.048 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.048 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.048 INFO:tasks.workunit.client.0.smithi069.stdout:558/600: open fd = 3 2024-04-25T11:36:58.048 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.048 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.048 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.048 INFO:tasks.workunit.client.0.smithi069.stdout:559/600: open fd = 3 2024-04-25T11:36:58.048 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.049 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.049 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:58.049 INFO:tasks.workunit.client.0.smithi069.stdout:560/600: open fd = 3 2024-04-25T11:36:58.049 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.049 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:36:58.049 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:36:59.343 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:36:59 smithi146 ceph-mon[32821]: pgmap v687: 129 pgs: 129 active+clean; 6.6 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:36:59.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:36:59 smithi069 ceph-mon[29188]: pgmap v687: 129 pgs: 129 active+clean; 6.6 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:36:59.474 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:36:59 smithi152 ceph-mon[32701]: pgmap v687: 129 pgs: 129 active+clean; 6.6 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:37:01.343 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:37:01 smithi146 ceph-mon[32821]: pgmap v688: 129 pgs: 129 active+clean; 6.6 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-04-25T11:37:01.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:37:01 smithi069 ceph-mon[29188]: pgmap v688: 129 pgs: 129 active+clean; 6.6 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-04-25T11:37:01.474 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:37:01 smithi152 ceph-mon[32701]: pgmap v688: 129 pgs: 129 active+clean; 6.6 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-04-25T11:37:03.343 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:37:03 smithi146 ceph-mon[32821]: pgmap v689: 129 pgs: 129 active+clean; 6.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:37:03.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:37:03 smithi069 ceph-mon[29188]: pgmap v689: 129 pgs: 129 active+clean; 6.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:37:03.474 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:37:03 smithi152 ceph-mon[32701]: pgmap v689: 129 pgs: 129 active+clean; 6.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:37:05.343 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:37:05 smithi146 ceph-mon[32821]: pgmap v690: 129 pgs: 129 active+clean; 6.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:37:05.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:37:05 smithi069 ceph-mon[29188]: pgmap v690: 129 pgs: 129 active+clean; 6.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:37:05.474 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:37:05 smithi152 ceph-mon[32701]: pgmap v690: 129 pgs: 129 active+clean; 6.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:37:07.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:37:07 smithi069 ceph-mon[29188]: pgmap v691: 129 pgs: 129 active+clean; 6.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:37:07.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:37:07 smithi152 ceph-mon[32701]: pgmap v691: 129 pgs: 129 active+clean; 6.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:37:07.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:37:07 smithi146 ceph-mon[32821]: pgmap v691: 129 pgs: 129 active+clean; 6.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:37:09.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:37:09 smithi069 ceph-mon[29188]: pgmap v692: 129 pgs: 129 active+clean; 6.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:37:09.474 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:37:09 smithi152 ceph-mon[32701]: pgmap v692: 129 pgs: 129 active+clean; 6.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:37:09.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:37:09 smithi146 ceph-mon[32821]: pgmap v692: 129 pgs: 129 active+clean; 6.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:37:11.438 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:37:11 smithi069 ceph-mon[29188]: pgmap v693: 129 pgs: 129 active+clean; 6.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-25T11:37:11.474 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:37:11 smithi152 ceph-mon[32701]: pgmap v693: 129 pgs: 129 active+clean; 6.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-25T11:37:11.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:37:11 smithi146 ceph-mon[32821]: pgmap v693: 129 pgs: 129 active+clean; 6.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-25T11:37:13.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:37:13 smithi069 ceph-mon[29188]: pgmap v694: 129 pgs: 129 active+clean; 6.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-04-25T11:37:13.474 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:37:13 smithi152 ceph-mon[32701]: pgmap v694: 129 pgs: 129 active+clean; 6.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-04-25T11:37:13.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:37:13 smithi146 ceph-mon[32821]: pgmap v694: 129 pgs: 129 active+clean; 6.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-04-25T11:37:15.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:37:15 smithi069 ceph-mon[29188]: pgmap v695: 129 pgs: 129 active+clean; 6.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:37:15.474 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:37:15 smithi152 ceph-mon[32701]: pgmap v695: 129 pgs: 129 active+clean; 6.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:37:15.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:37:15 smithi146 ceph-mon[32821]: pgmap v695: 129 pgs: 129 active+clean; 6.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:37:17.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:37:17 smithi069 ceph-mon[29188]: pgmap v696: 129 pgs: 129 active+clean; 6.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:37:17.474 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:37:17 smithi152 ceph-mon[32701]: pgmap v696: 129 pgs: 129 active+clean; 6.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:37:17.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:37:17 smithi146 ceph-mon[32821]: pgmap v696: 129 pgs: 129 active+clean; 6.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T11:37:19.395 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:37:19 smithi146 ceph-mon[32821]: pgmap v697: 129 pgs: 129 active+clean; 6.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 17 op/s 2024-04-25T11:37:19.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:37:19 smithi069 ceph-mon[29188]: pgmap v697: 129 pgs: 129 active+clean; 6.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 17 op/s 2024-04-25T11:37:19.474 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:37:19 smithi152 ceph-mon[32701]: pgmap v697: 129 pgs: 129 active+clean; 6.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 17 op/s 2024-04-25T11:37:21.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:37:21 smithi069 ceph-mon[29188]: pgmap v698: 129 pgs: 129 active+clean; 6.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:37:21.474 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:37:21 smithi152 ceph-mon[32701]: pgmap v698: 129 pgs: 129 active+clean; 6.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:37:21.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:37:21 smithi146 ceph-mon[32821]: pgmap v698: 129 pgs: 129 active+clean; 6.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:37:23.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:37:23 smithi069 ceph-mon[29188]: pgmap v699: 129 pgs: 129 active+clean; 6.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:37:23.474 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:37:23 smithi152 ceph-mon[32701]: pgmap v699: 129 pgs: 129 active+clean; 6.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:37:23.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:37:23 smithi146 ceph-mon[32821]: pgmap v699: 129 pgs: 129 active+clean; 6.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:37:25.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:37:25 smithi069 ceph-mon[29188]: pgmap v700: 129 pgs: 129 active+clean; 6.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:37:25.474 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:37:25 smithi152 ceph-mon[32701]: pgmap v700: 129 pgs: 129 active+clean; 6.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:37:25.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:37:25 smithi146 ceph-mon[32821]: pgmap v700: 129 pgs: 129 active+clean; 6.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:37:27.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:37:27 smithi069 ceph-mon[29188]: pgmap v701: 129 pgs: 129 active+clean; 6.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:37:27.474 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:37:27 smithi152 ceph-mon[32701]: pgmap v701: 129 pgs: 129 active+clean; 6.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:37:27.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:37:27 smithi146 ceph-mon[32821]: pgmap v701: 129 pgs: 129 active+clean; 6.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:37:29.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:37:29 smithi069 ceph-mon[29188]: pgmap v702: 129 pgs: 129 active+clean; 6.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:37:29.474 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:37:29 smithi152 ceph-mon[32701]: pgmap v702: 129 pgs: 129 active+clean; 6.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:37:29.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:37:29 smithi146 ceph-mon[32821]: pgmap v702: 129 pgs: 129 active+clean; 6.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:37:31.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:37:31 smithi069 ceph-mon[29188]: pgmap v703: 129 pgs: 129 active+clean; 6.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-04-25T11:37:31.474 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:37:31 smithi152 ceph-mon[32701]: pgmap v703: 129 pgs: 129 active+clean; 6.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-04-25T11:37:31.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:37:31 smithi146 ceph-mon[32821]: pgmap v703: 129 pgs: 129 active+clean; 6.9 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-04-25T11:37:33.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:37:33 smithi069 ceph-mon[29188]: pgmap v704: 129 pgs: 129 active+clean; 7.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:37:33.474 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:37:33 smithi152 ceph-mon[32701]: pgmap v704: 129 pgs: 129 active+clean; 7.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:37:33.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:37:33 smithi146 ceph-mon[32821]: pgmap v704: 129 pgs: 129 active+clean; 7.0 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-25T11:37:35.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:37:35 smithi069 ceph-mon[29188]: pgmap v705: 129 pgs: 129 active+clean; 7.0 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:37:35.474 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:37:35 smithi152 ceph-mon[32701]: pgmap v705: 129 pgs: 129 active+clean; 7.0 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:37:35.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:37:35 smithi146 ceph-mon[32821]: pgmap v705: 129 pgs: 129 active+clean; 7.0 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-25T11:37:36.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:37:36 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:37:36.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:37:36 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:37:36.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:37:36 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:37:36.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:37:36 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:37:36.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:37:36 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:37:36.474 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:37:36 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:37:36.474 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:37:36 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:37:36.474 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:37:36 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:37:36.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:37:36 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:37:36.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:37:36 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:37:36.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:37:36 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:37:36.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:37:36 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:37:36.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:37:36 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:37:36.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:37:36 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:37:36.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:37:36 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:37:37.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:37:37 smithi069 ceph-mon[29188]: pgmap v706: 129 pgs: 129 active+clean; 7.0 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:37:37.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:37:37 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:37:37.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:37:37 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:37:37.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:37:37 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:37:37.474 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:37:37 smithi152 ceph-mon[32701]: pgmap v706: 129 pgs: 129 active+clean; 7.0 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:37:37.474 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:37:37 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:37:37.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:37:37 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:37:37.475 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:37:37 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:37:37.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:37:37 smithi146 ceph-mon[32821]: pgmap v706: 129 pgs: 129 active+clean; 7.0 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-25T11:37:37.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:37:37 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:37:37.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:37:37 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:37:37.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:37:37 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' 2024-04-25T11:37:38.418 INFO:tasks.workunit.client.0.smithi069.stdout:561/600: open fd = 3 2024-04-25T11:37:38.418 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.418 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.418 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:37:38.418 INFO:tasks.workunit.client.0.smithi069.stdout:562/600: open fd = 3 2024-04-25T11:37:38.418 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.418 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.418 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:37:38.419 INFO:tasks.workunit.client.0.smithi069.stdout:563/600: open fd = 3 2024-04-25T11:37:38.419 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.419 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.419 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:37:38.419 INFO:tasks.workunit.client.0.smithi069.stdout:564/600: open fd = 3 2024-04-25T11:37:38.419 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.419 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.419 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:37:38.419 INFO:tasks.workunit.client.0.smithi069.stdout:565/600: open fd = 3 2024-04-25T11:37:38.419 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.419 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.419 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:37:38.419 INFO:tasks.workunit.client.0.smithi069.stdout:566/600: open fd = 3 2024-04-25T11:37:38.419 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.420 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.420 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:37:38.420 INFO:tasks.workunit.client.0.smithi069.stdout:567/600: open fd = 3 2024-04-25T11:37:38.420 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.420 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.420 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:37:38.420 INFO:tasks.workunit.client.0.smithi069.stdout:568/600: open fd = 3 2024-04-25T11:37:38.420 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.420 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.420 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:37:38.420 INFO:tasks.workunit.client.0.smithi069.stdout:569/600: open fd = 3 2024-04-25T11:37:38.420 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.420 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.420 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:37:38.420 INFO:tasks.workunit.client.0.smithi069.stdout:570/600: open fd = 3 2024-04-25T11:37:38.421 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.421 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.421 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:37:38.421 INFO:tasks.workunit.client.0.smithi069.stdout:571/600: open fd = 3 2024-04-25T11:37:38.421 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.421 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.421 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:37:38.421 INFO:tasks.workunit.client.0.smithi069.stdout:572/600: open fd = 3 2024-04-25T11:37:38.421 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.421 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.421 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:37:38.421 INFO:tasks.workunit.client.0.smithi069.stdout:573/600: open fd = 3 2024-04-25T11:37:38.421 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.421 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.421 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:37:38.421 INFO:tasks.workunit.client.0.smithi069.stdout:574/600: open fd = 3 2024-04-25T11:37:38.422 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.422 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.422 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:37:38.422 INFO:tasks.workunit.client.0.smithi069.stdout:575/600: open fd = 3 2024-04-25T11:37:38.422 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.422 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.422 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:37:38.422 INFO:tasks.workunit.client.0.smithi069.stdout:576/600: open fd = 3 2024-04-25T11:37:38.422 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.422 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.422 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:37:38.422 INFO:tasks.workunit.client.0.smithi069.stdout:577/600: open fd = 3 2024-04-25T11:37:38.422 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.422 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.423 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:37:38.423 INFO:tasks.workunit.client.0.smithi069.stdout:578/600: open fd = 3 2024-04-25T11:37:38.423 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.423 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.423 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:37:38.423 INFO:tasks.workunit.client.0.smithi069.stdout:579/600: open fd = 3 2024-04-25T11:37:38.423 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.423 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.423 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:37:38.423 INFO:tasks.workunit.client.0.smithi069.stdout:580/600: open fd = 3 2024-04-25T11:37:38.423 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.423 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.423 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:37:38.423 INFO:tasks.workunit.client.0.smithi069.stdout:581/600: open fd = 3 2024-04-25T11:37:38.423 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.423 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.424 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:37:38.424 INFO:tasks.workunit.client.0.smithi069.stdout:582/600: open fd = 3 2024-04-25T11:37:38.424 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.424 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.424 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:37:38.424 INFO:tasks.workunit.client.0.smithi069.stdout:583/600: open fd = 3 2024-04-25T11:37:38.424 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.424 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.424 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:37:38.424 INFO:tasks.workunit.client.0.smithi069.stdout:584/600: open fd = 3 2024-04-25T11:37:38.424 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.424 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.424 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:37:38.424 INFO:tasks.workunit.client.0.smithi069.stdout:585/600: open fd = 3 2024-04-25T11:37:38.424 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.425 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.425 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:37:38.425 INFO:tasks.workunit.client.0.smithi069.stdout:586/600: open fd = 3 2024-04-25T11:37:38.425 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.425 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.425 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:37:38.425 INFO:tasks.workunit.client.0.smithi069.stdout:587/600: open fd = 3 2024-04-25T11:37:38.425 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.425 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.425 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:37:38.425 INFO:tasks.workunit.client.0.smithi069.stdout:588/600: open fd = 3 2024-04-25T11:37:38.425 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.425 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.425 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:37:38.425 INFO:tasks.workunit.client.0.smithi069.stdout:589/600: open fd = 3 2024-04-25T11:37:38.426 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.426 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.426 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:37:38.426 INFO:tasks.workunit.client.0.smithi069.stdout:590/600: open fd = 3 2024-04-25T11:37:38.426 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.426 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.426 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:37:38.426 INFO:tasks.workunit.client.0.smithi069.stdout:591/600: open fd = 3 2024-04-25T11:37:38.426 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.426 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.426 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:37:38.426 INFO:tasks.workunit.client.0.smithi069.stdout:592/600: open fd = 3 2024-04-25T11:37:38.426 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.426 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.426 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:37:38.427 INFO:tasks.workunit.client.0.smithi069.stdout:593/600: open fd = 3 2024-04-25T11:37:38.427 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.427 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.427 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:37:38.427 INFO:tasks.workunit.client.0.smithi069.stdout:594/600: open fd = 3 2024-04-25T11:37:38.427 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.427 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.427 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:37:38.427 INFO:tasks.workunit.client.0.smithi069.stdout:595/600: open fd = 3 2024-04-25T11:37:38.427 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.427 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.427 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:37:38.427 INFO:tasks.workunit.client.0.smithi069.stdout:596/600: open fd = 3 2024-04-25T11:37:38.427 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.427 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.427 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:37:38.428 INFO:tasks.workunit.client.0.smithi069.stdout:597/600: open fd = 3 2024-04-25T11:37:38.428 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.428 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.428 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:37:38.428 INFO:tasks.workunit.client.0.smithi069.stdout:598/600: open fd = 3 2024-04-25T11:37:38.428 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.428 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.428 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:37:38.428 INFO:tasks.workunit.client.0.smithi069.stdout:599/600: open fd = 3 2024-04-25T11:37:38.428 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.428 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.428 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:37:38.428 INFO:tasks.workunit.client.0.smithi069.stdout:600/600: open fd = 3 2024-04-25T11:37:38.428 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.428 INFO:tasks.workunit.client.0.smithi069.stdout:write ret = 32 2024-04-25T11:37:38.429 INFO:tasks.workunit.client.0.smithi069.stdout:pread ret = 64 2024-04-25T11:37:38.429 INFO:tasks.workunit.client.0.smithi069.stderr:+ echo OK 2024-04-25T11:37:38.430 INFO:tasks.workunit.client.0.smithi069.stdout:OK 2024-04-25T11:37:38.430 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-25T11:37:38.431 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-25T11:37:38.517 INFO:tasks.workunit:Stopping ['fs/test_o_trunc.sh'] on client.0... 2024-04-25T11:37:38.518 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-25T11:37:38.788 DEBUG:teuthology.parallel:result is None 2024-04-25T11:37:38.789 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T11:37:38.821 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T11:37:38.821 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-25T11:37:38.833 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:37:38.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:38.982+0000 7fc915d6c640 1 -- 172.21.15.69:0/3437173355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc910103b00 msgr2=0x7fc910104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:38.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:38.982+0000 7fc915d6c640 1 --2- 172.21.15.69:0/3437173355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc910103b00 0x7fc910104800 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc900009900 tx=0x7fc90002a410 comp rx=0 tx=0).stop 2024-04-25T11:37:38.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:38.982+0000 7fc915d6c640 1 -- 172.21.15.69:0/3437173355 shutdown_connections 2024-04-25T11:37:38.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:38.982+0000 7fc915d6c640 1 --2- 172.21.15.69:0/3437173355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc910109a40 0x7fc91010be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:38.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:38.982+0000 7fc915d6c640 1 --2- 172.21.15.69:0/3437173355 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc910104d40 0x7fc910105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:38.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:38.982+0000 7fc915d6c640 1 --2- 172.21.15.69:0/3437173355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc910103b00 0x7fc910104800 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:38.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:38.982+0000 7fc915d6c640 1 -- 172.21.15.69:0/3437173355 >> 172.21.15.69:0/3437173355 conn(0x7fc9100fed30 msgr2=0x7fc9101011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:38.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:38.982+0000 7fc915d6c640 1 -- 172.21.15.69:0/3437173355 shutdown_connections 2024-04-25T11:37:38.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:38.982+0000 7fc915d6c640 1 -- 172.21.15.69:0/3437173355 wait complete. 2024-04-25T11:37:38.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:38.983+0000 7fc915d6c640 1 Processor -- start 2024-04-25T11:37:38.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:38.983+0000 7fc915d6c640 1 -- start start 2024-04-25T11:37:38.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:38.983+0000 7fc915d6c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc910103b00 0x7fc91019a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:38.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:38.983+0000 7fc915d6c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc910104d40 0x7fc91019a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:38.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:38.983+0000 7fc915d6c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc910109a40 0x7fc9101a15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:38.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:38.983+0000 7fc915d6c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc91010f490 con 0x7fc910103b00 2024-04-25T11:37:38.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:38.983+0000 7fc915d6c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc91010f310 con 0x7fc910104d40 2024-04-25T11:37:38.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:38.983+0000 7fc915d6c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc91010f610 con 0x7fc910109a40 2024-04-25T11:37:38.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:38.983+0000 7fc90f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc910103b00 0x7fc91019a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:38.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:38.983+0000 7fc90f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc910103b00 0x7fc91019a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48768/0 (socket says 172.21.15.69:48768) 2024-04-25T11:37:38.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:38.983+0000 7fc90f7fe640 1 -- 172.21.15.69:0/2695678601 learned_addr learned my addr 172.21.15.69:0/2695678601 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:37:38.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:38.983+0000 7fc90ffff640 1 --2- 172.21.15.69:0/2695678601 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc910109a40 0x7fc9101a15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:38.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:38.983+0000 7fc907fff640 1 --2- 172.21.15.69:0/2695678601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc910104d40 0x7fc91019a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:38.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:38.983+0000 7fc90f7fe640 1 -- 172.21.15.69:0/2695678601 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc910109a40 msgr2=0x7fc9101a15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:38.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:38.983+0000 7fc90f7fe640 1 --2- 172.21.15.69:0/2695678601 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc910109a40 0x7fc9101a15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:38.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:38.983+0000 7fc90f7fe640 1 -- 172.21.15.69:0/2695678601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc910104d40 msgr2=0x7fc91019a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:38.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:38.983+0000 7fc90f7fe640 1 --2- 172.21.15.69:0/2695678601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc910104d40 0x7fc91019a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:38.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:38.983+0000 7fc90f7fe640 1 -- 172.21.15.69:0/2695678601 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9101a1c60 con 0x7fc910103b00 2024-04-25T11:37:38.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:38.983+0000 7fc907fff640 1 --2- 172.21.15.69:0/2695678601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc910104d40 0x7fc91019a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:37:38.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:38.983+0000 7fc90f7fe640 1 --2- 172.21.15.69:0/2695678601 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc910103b00 0x7fc91019a120 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fc900005ab0 tx=0x7fc900004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:38.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:38.984+0000 7fc90d7fa640 1 -- 172.21.15.69:0/2695678601 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc90003f070 con 0x7fc910103b00 2024-04-25T11:37:38.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:38.984+0000 7fc90d7fa640 1 -- 172.21.15.69:0/2695678601 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc900003c80 con 0x7fc910103b00 2024-04-25T11:37:38.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:38.984+0000 7fc915d6c640 1 -- 172.21.15.69:0/2695678601 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9101a1ef0 con 0x7fc910103b00 2024-04-25T11:37:38.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:38.984+0000 7fc90d7fa640 1 -- 172.21.15.69:0/2695678601 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc900035070 con 0x7fc910103b00 2024-04-25T11:37:38.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:38.984+0000 7fc915d6c640 1 -- 172.21.15.69:0/2695678601 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9101a22f0 con 0x7fc910103b00 2024-04-25T11:37:38.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:38.985+0000 7fc915d6c640 1 -- 172.21.15.69:0/2695678601 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc910108850 con 0x7fc910103b00 2024-04-25T11:37:38.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:38.986+0000 7fc90d7fa640 1 -- 172.21.15.69:0/2695678601 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fc90003b610 con 0x7fc910103b00 2024-04-25T11:37:38.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:38.986+0000 7fc90d7fa640 1 --2- 172.21.15.69:0/2695678601 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc8fc0afaf0 0x7fc8fc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:38.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:38.987+0000 7fc907fff640 1 --2- 172.21.15.69:0/2695678601 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc8fc0afaf0 0x7fc8fc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:38.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:38.987+0000 7fc90d7fa640 1 -- 172.21.15.69:0/2695678601 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7fc900100000 con 0x7fc910103b00 2024-04-25T11:37:38.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:38.987+0000 7fc907fff640 1 --2- 172.21.15.69:0/2695678601 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc8fc0afaf0 0x7fc8fc0b2000 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc8f80058c0 tx=0x7fc8f8005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:38.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:38.991+0000 7fc90d7fa640 1 -- 172.21.15.69:0/2695678601 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc9000c8810 con 0x7fc910103b00 2024-04-25T11:37:39.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.168+0000 7fc915d6c640 1 -- 172.21.15.69:0/2695678601 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc91002dd20 con 0x7fc910103b00 2024-04-25T11:37:39.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.170+0000 7fc90d7fa640 1 -- 172.21.15.69:0/2695678601 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+24862 (secure 0 0 0) 0x7fc9000c8110 con 0x7fc910103b00 2024-04-25T11:37:39.158 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:37:39.158 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":84,"fsid":"f7ed73dc-02f4-11ef-bc93-c7b262605968","created":"2024-04-25T11:15:37.965281+0000","modified":"2024-04-25T11:25:33.888184+0000","last_up_change":"2024-04-25T11:22:50.691869+0000","last_in_change":"2024-04-25T11:22:30.271959+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:18:57.781263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:23:38.685110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:23:39.943803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T11:23:43.650521+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6a0d179-4998-4754-bf82-2ea3dac54e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6803","nonce":3756025235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6805","nonce":3756025235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6809","nonce":3756025235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6807","nonce":3756025235}]},"public_addr":"172.21.15.69:6803/3756025235","cluster_addr":"172.21.15.69:6805/3756025235","heartbeat_back_addr":"172.21.15.69:6809/3756025235","heartbeat_front_addr":"172.21.15.69:6807/3756025235","state":["exists","up"]},{"osd":1,"uuid":"c0c66930-4233-493f-b2bb-99c473994b59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6801","nonce":3166305213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6803","nonce":3166305213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6807","nonce":3166305213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6805","nonce":3166305213}]},"public_addr":"172.21.15.146:6801/3166305213","cluster_addr":"172.21.15.146:6803/3166305213","heartbeat_back_addr":"172.21.15.146:6807/3166305213","heartbeat_front_addr":"172.21.15.146:6805/3166305213","state":["exists","up"]},{"osd":2,"uuid":"d4ef735b-9786-4cb9-83ef-7c3f0691a729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6801","nonce":1474319175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6803","nonce":1474319175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6807","nonce":1474319175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6805","nonce":1474319175}]},"public_addr":"172.21.15.152:6801/1474319175","cluster_addr":"172.21.15.152:6803/1474319175","heartbeat_back_addr":"172.21.15.152:6807/1474319175","heartbeat_front_addr":"172.21.15.152:6805/1474319175","state":["exists","up"]},{"osd":3,"uuid":"0ab85fb8-5bbb-4ffc-bddf-cae7644a64ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6811","nonce":1809071296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6813","nonce":1809071296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6817","nonce":1809071296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6815","nonce":1809071296}]},"public_addr":"172.21.15.69:6811/1809071296","cluster_addr":"172.21.15.69:6813/1809071296","heartbeat_back_addr":"172.21.15.69:6817/1809071296","heartbeat_front_addr":"172.21.15.69:6815/1809071296","state":["exists","up"]},{"osd":4,"uuid":"273e0592-bbbc-4188-8eeb-46a3879cf632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6809","nonce":1898520972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6811","nonce":1898520972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6815","nonce":1898520972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6813","nonce":1898520972}]},"public_addr":"172.21.15.146:6809/1898520972","cluster_addr":"172.21.15.146:6811/1898520972","heartbeat_back_addr":"172.21.15.146:6815/1898520972","heartbeat_front_addr":"172.21.15.146:6813/1898520972","state":["exists","up"]},{"osd":5,"uuid":"bbeae945-9fa2-48f9-a1d6-8bf18dcaef19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6809","nonce":1996555108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6811","nonce":1996555108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6815","nonce":1996555108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6813","nonce":1996555108}]},"public_addr":"172.21.15.152:6809/1996555108","cluster_addr":"172.21.15.152:6811/1996555108","heartbeat_back_addr":"172.21.15.152:6815/1996555108","heartbeat_front_addr":"172.21.15.152:6813/1996555108","state":["exists","up"]},{"osd":6,"uuid":"dbb61253-30bc-418e-bd75-118c80bbf931","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6819","nonce":491215762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6821","nonce":491215762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6825","nonce":491215762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6823","nonce":491215762}]},"public_addr":"172.21.15.69:6819/491215762","cluster_addr":"172.21.15.69:6821/491215762","heartbeat_back_addr":"172.21.15.69:6825/491215762","heartbeat_front_addr":"172.21.15.69:6823/491215762","state":["exists","up"]},{"osd":7,"uuid":"75f23854-e283-4e3e-94cf-7b4079d83876","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6817","nonce":2330294690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6819","nonce":2330294690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6823","nonce":2330294690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6821","nonce":2330294690}]},"public_addr":"172.21.15.146:6817/2330294690","cluster_addr":"172.21.15.146:6819/2330294690","heartbeat_back_addr":"172.21.15.146:6823/2330294690","heartbeat_front_addr":"172.21.15.146:6821/2330294690","state":["exists","up"]},{"osd":8,"uuid":"1aa1060c-0c14-4abd-be56-5ebfeb78759c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6817","nonce":950669381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6819","nonce":950669381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6823","nonce":950669381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6821","nonce":950669381}]},"public_addr":"172.21.15.152:6817/950669381","cluster_addr":"172.21.15.152:6819/950669381","heartbeat_back_addr":"172.21.15.152:6823/950669381","heartbeat_front_addr":"172.21.15.152:6821/950669381","state":["exists","up"]},{"osd":9,"uuid":"3f4bf800-4090-4138-9bb3-d001f13f450c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6827","nonce":2414186758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6829","nonce":2414186758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6833","nonce":2414186758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6831","nonce":2414186758}]},"public_addr":"172.21.15.69:6827/2414186758","cluster_addr":"172.21.15.69:6829/2414186758","heartbeat_back_addr":"172.21.15.69:6833/2414186758","heartbeat_front_addr":"172.21.15.69:6831/2414186758","state":["exists","up"]},{"osd":10,"uuid":"19e967e5-51ce-4506-80c9-0a95441baf99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6825","nonce":707256400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6827","nonce":707256400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6831","nonce":707256400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6829","nonce":707256400}]},"public_addr":"172.21.15.146:6825/707256400","cluster_addr":"172.21.15.146:6827/707256400","heartbeat_back_addr":"172.21.15.146:6831/707256400","heartbeat_front_addr":"172.21.15.146:6829/707256400","state":["exists","up"]},{"osd":11,"uuid":"7788c2de-6930-4808-be0f-f1fd1ae0cb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6825","nonce":2934336833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6827","nonce":2934336833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6831","nonce":2934336833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6829","nonce":2934336833}]},"public_addr":"172.21.15.152:6825/2934336833","cluster_addr":"172.21.15.152:6827/2934336833","heartbeat_back_addr":"172.21.15.152:6831/2934336833","heartbeat_front_addr":"172.21.15.152:6829/2934336833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:07.497448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:31.227494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:55.027978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:20.713437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:46.214704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:20:11.199983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:20:37.177070+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:02.951240+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:28.963680+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:56.095807+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:22:22.221744+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:22:48.929334+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1b","mappings":[{"from":5,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/1372727546":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6801/1097124842":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6800/1508198215":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3103465990":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3540132960":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3899912190":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6800/1097124842":"2024-04-26T11:16:31.688159+0000","172.21.15.69:0/1199907202":"2024-04-26T11:16:04.860792+0000","172.21.15.69:6801/1508198215":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/2879102529":"2024-04-26T11:16:31.688159+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:37:39.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.172+0000 7fc915d6c640 1 -- 172.21.15.69:0/2695678601 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc8fc0afaf0 msgr2=0x7fc8fc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:39.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.172+0000 7fc915d6c640 1 --2- 172.21.15.69:0/2695678601 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc8fc0afaf0 0x7fc8fc0b2000 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc8f80058c0 tx=0x7fc8f8005d50 comp rx=0 tx=0).stop 2024-04-25T11:37:39.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.172+0000 7fc915d6c640 1 -- 172.21.15.69:0/2695678601 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc910103b00 msgr2=0x7fc91019a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:39.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.172+0000 7fc915d6c640 1 --2- 172.21.15.69:0/2695678601 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc910103b00 0x7fc91019a120 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fc900005ab0 tx=0x7fc900004270 comp rx=0 tx=0).stop 2024-04-25T11:37:39.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.172+0000 7fc915d6c640 1 -- 172.21.15.69:0/2695678601 shutdown_connections 2024-04-25T11:37:39.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.172+0000 7fc915d6c640 1 --2- 172.21.15.69:0/2695678601 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc8fc0afaf0 0x7fc8fc0b2000 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:39.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.172+0000 7fc915d6c640 1 --2- 172.21.15.69:0/2695678601 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc910109a40 0x7fc9101a15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:39.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.172+0000 7fc915d6c640 1 --2- 172.21.15.69:0/2695678601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc910104d40 0x7fc91019a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:39.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.172+0000 7fc915d6c640 1 --2- 172.21.15.69:0/2695678601 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc910103b00 0x7fc91019a120 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:39.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.172+0000 7fc915d6c640 1 -- 172.21.15.69:0/2695678601 >> 172.21.15.69:0/2695678601 conn(0x7fc9100fed30 msgr2=0x7fc910105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:39.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.172+0000 7fc915d6c640 1 -- 172.21.15.69:0/2695678601 shutdown_connections 2024-04-25T11:37:39.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.172+0000 7fc915d6c640 1 -- 172.21.15.69:0/2695678601 wait complete. 2024-04-25T11:37:39.171 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:18:57.781263+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:23:38.685110+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:23:39.943803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T11:23:43.650521+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T11:37:39.172 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:37:39.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.297+0000 7ff54e251640 1 -- 172.21.15.69:0/2017714596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff548101740 msgr2=0x7ff548101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:39.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.297+0000 7ff54e251640 1 --2- 172.21.15.69:0/2017714596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff548101740 0x7ff548101bb0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ff53c009900 tx=0x7ff53c02a410 comp rx=0 tx=0).stop 2024-04-25T11:37:39.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.297+0000 7ff54e251640 1 -- 172.21.15.69:0/2017714596 shutdown_connections 2024-04-25T11:37:39.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.297+0000 7ff54e251640 1 --2- 172.21.15.69:0/2017714596 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff548102240 0x7ff54810bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:39.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.297+0000 7ff54e251640 1 --2- 172.21.15.69:0/2017714596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff548101740 0x7ff548101bb0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:39.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.297+0000 7ff54e251640 1 --2- 172.21.15.69:0/2017714596 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff548105090 0x7ff5481054a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:39.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.297+0000 7ff54e251640 1 -- 172.21.15.69:0/2017714596 >> 172.21.15.69:0/2017714596 conn(0x7ff5480fd340 msgr2=0x7ff5480ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:39.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.297+0000 7ff54e251640 1 -- 172.21.15.69:0/2017714596 shutdown_connections 2024-04-25T11:37:39.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.297+0000 7ff54e251640 1 -- 172.21.15.69:0/2017714596 wait complete. 2024-04-25T11:37:39.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.297+0000 7ff54e251640 1 Processor -- start 2024-04-25T11:37:39.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.298+0000 7ff54e251640 1 -- start start 2024-04-25T11:37:39.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.298+0000 7ff54e251640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff548101740 0x7ff54819a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:39.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.298+0000 7ff54e251640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff548102240 0x7ff54819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:39.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.298+0000 7ff54e251640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff548105090 0x7ff5481a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:39.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.298+0000 7ff54e251640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff54810f4b0 con 0x7ff548101740 2024-04-25T11:37:39.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.298+0000 7ff54e251640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff54810f330 con 0x7ff548102240 2024-04-25T11:37:39.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.298+0000 7ff54e251640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff54810f630 con 0x7ff548105090 2024-04-25T11:37:39.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.298+0000 7ff5477fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff548101740 0x7ff54819a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:39.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.298+0000 7ff5477fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff548101740 0x7ff54819a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48778/0 (socket says 172.21.15.69:48778) 2024-04-25T11:37:39.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.298+0000 7ff5477fe640 1 -- 172.21.15.69:0/159174650 learned_addr learned my addr 172.21.15.69:0/159174650 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:37:39.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.298+0000 7ff546ffd640 1 --2- 172.21.15.69:0/159174650 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff548102240 0x7ff54819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:39.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.298+0000 7ff5477fe640 1 -- 172.21.15.69:0/159174650 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff548105090 msgr2=0x7ff5481a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:39.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.298+0000 7ff5477fe640 1 --2- 172.21.15.69:0/159174650 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff548105090 0x7ff5481a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:39.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.298+0000 7ff5477fe640 1 -- 172.21.15.69:0/159174650 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff548102240 msgr2=0x7ff54819a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:39.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.298+0000 7ff5477fe640 1 --2- 172.21.15.69:0/159174650 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff548102240 0x7ff54819a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:39.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.298+0000 7ff5477fe640 1 -- 172.21.15.69:0/159174650 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5481a1c90 con 0x7ff548101740 2024-04-25T11:37:39.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.298+0000 7ff5477fe640 1 --2- 172.21.15.69:0/159174650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff548101740 0x7ff54819a150 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ff538005fd0 tx=0x7ff53800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:39.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.298+0000 7ff544ff9640 1 -- 172.21.15.69:0/159174650 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff538017070 con 0x7ff548101740 2024-04-25T11:37:39.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.299+0000 7ff544ff9640 1 -- 172.21.15.69:0/159174650 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff538009c30 con 0x7ff548101740 2024-04-25T11:37:39.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.299+0000 7ff54e251640 1 -- 172.21.15.69:0/159174650 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5481a1f80 con 0x7ff548101740 2024-04-25T11:37:39.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.299+0000 7ff544ff9640 1 -- 172.21.15.69:0/159174650 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff538003c90 con 0x7ff548101740 2024-04-25T11:37:39.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.299+0000 7ff54e251640 1 -- 172.21.15.69:0/159174650 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5481a2440 con 0x7ff548101740 2024-04-25T11:37:39.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.300+0000 7ff54e251640 1 -- 172.21.15.69:0/159174650 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff548108a20 con 0x7ff548101740 2024-04-25T11:37:39.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.301+0000 7ff544ff9640 1 -- 172.21.15.69:0/159174650 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7ff538003e30 con 0x7ff548101740 2024-04-25T11:37:39.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.302+0000 7ff544ff9640 1 --2- 172.21.15.69:0/159174650 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ff5300afb70 0x7ff5300b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:39.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.302+0000 7ff546ffd640 1 --2- 172.21.15.69:0/159174650 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ff5300afb70 0x7ff5300b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:39.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.302+0000 7ff544ff9640 1 -- 172.21.15.69:0/159174650 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7ff5380df390 con 0x7ff548101740 2024-04-25T11:37:39.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.302+0000 7ff546ffd640 1 --2- 172.21.15.69:0/159174650 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ff5300afb70 0x7ff5300b2080 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff53c006010 tx=0x7ff53c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:39.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.306+0000 7ff544ff9640 1 -- 172.21.15.69:0/159174650 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff538020030 con 0x7ff548101740 2024-04-25T11:37:39.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:37:39 smithi069 ceph-mon[29188]: pgmap v707: 129 pgs: 129 active+clean; 7.0 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:37:39.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:37:39 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2695678601' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:37:39.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.487+0000 7ff54e251640 1 -- 172.21.15.69:0/159174650 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff54802dce0 con 0x7ff548101740 2024-04-25T11:37:39.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.487+0000 7ff544ff9640 1 -- 172.21.15.69:0/159174650 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7ff5380a7cc0 con 0x7ff548101740 2024-04-25T11:37:39.475 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-04-25T11:37:39.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.489+0000 7ff54e251640 1 -- 172.21.15.69:0/159174650 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ff5300afb70 msgr2=0x7ff5300b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:39.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.489+0000 7ff54e251640 1 --2- 172.21.15.69:0/159174650 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ff5300afb70 0x7ff5300b2080 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff53c006010 tx=0x7ff53c000bc0 comp rx=0 tx=0).stop 2024-04-25T11:37:39.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.489+0000 7ff54e251640 1 -- 172.21.15.69:0/159174650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff548101740 msgr2=0x7ff54819a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:39.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.489+0000 7ff54e251640 1 --2- 172.21.15.69:0/159174650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff548101740 0x7ff54819a150 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ff538005fd0 tx=0x7ff53800dde0 comp rx=0 tx=0).stop 2024-04-25T11:37:39.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.489+0000 7ff54e251640 1 -- 172.21.15.69:0/159174650 shutdown_connections 2024-04-25T11:37:39.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.490+0000 7ff54e251640 1 --2- 172.21.15.69:0/159174650 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ff5300afb70 0x7ff5300b2080 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:39.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.490+0000 7ff54e251640 1 --2- 172.21.15.69:0/159174650 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff548105090 0x7ff5481a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:39.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.490+0000 7ff54e251640 1 --2- 172.21.15.69:0/159174650 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff548102240 0x7ff54819a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:39.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.490+0000 7ff54e251640 1 --2- 172.21.15.69:0/159174650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff548101740 0x7ff54819a150 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:39.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.490+0000 7ff54e251640 1 -- 172.21.15.69:0/159174650 >> 172.21.15.69:0/159174650 conn(0x7ff5480fd340 msgr2=0x7ff548105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:39.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.490+0000 7ff54e251640 1 -- 172.21.15.69:0/159174650 shutdown_connections 2024-04-25T11:37:39.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.490+0000 7ff54e251640 1 -- 172.21.15.69:0/159174650 wait complete. 2024-04-25T11:37:39.489 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:37:39.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:37:39 smithi146 ceph-mon[32821]: pgmap v707: 129 pgs: 129 active+clean; 7.0 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:37:39.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:37:39 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/2695678601' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:37:39.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.612+0000 7fc53a1d5640 1 -- 172.21.15.69:0/533586806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc534101e40 msgr2=0x7fc534102250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:39.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.612+0000 7fc53a1d5640 1 --2- 172.21.15.69:0/533586806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc534101e40 0x7fc534102250 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc528009900 tx=0x7fc52802a410 comp rx=0 tx=0).stop 2024-04-25T11:37:39.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.612+0000 7fc53a1d5640 1 -- 172.21.15.69:0/533586806 shutdown_connections 2024-04-25T11:37:39.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.612+0000 7fc53a1d5640 1 --2- 172.21.15.69:0/533586806 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc534109790 0x7fc53410bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:39.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.612+0000 7fc53a1d5640 1 --2- 172.21.15.69:0/533586806 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc534106c20 0x7fc534109100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:39.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.612+0000 7fc53a1d5640 1 --2- 172.21.15.69:0/533586806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc534101e40 0x7fc534102250 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:39.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.612+0000 7fc53a1d5640 1 -- 172.21.15.69:0/533586806 >> 172.21.15.69:0/533586806 conn(0x7fc5340fd350 msgr2=0x7fc5340ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:39.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.612+0000 7fc53a1d5640 1 -- 172.21.15.69:0/533586806 shutdown_connections 2024-04-25T11:37:39.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.612+0000 7fc53a1d5640 1 -- 172.21.15.69:0/533586806 wait complete. 2024-04-25T11:37:39.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.613+0000 7fc53a1d5640 1 Processor -- start 2024-04-25T11:37:39.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.613+0000 7fc53a1d5640 1 -- start start 2024-04-25T11:37:39.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.613+0000 7fc53a1d5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc534101e40 0x7fc534104310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:39.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.613+0000 7fc53a1d5640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc534106c20 0x7fc5341029b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:39.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.613+0000 7fc53a1d5640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc534109790 0x7fc534102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:39.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.613+0000 7fc53a1d5640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc53410f4a0 con 0x7fc534101e40 2024-04-25T11:37:39.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.613+0000 7fc53a1d5640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc53410f320 con 0x7fc534109790 2024-04-25T11:37:39.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.613+0000 7fc53a1d5640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc53410f620 con 0x7fc534106c20 2024-04-25T11:37:39.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.613+0000 7fc5337fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc534101e40 0x7fc534104310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:39.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.614+0000 7fc5337fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc534101e40 0x7fc534104310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48802/0 (socket says 172.21.15.69:48802) 2024-04-25T11:37:39.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.614+0000 7fc5337fe640 1 -- 172.21.15.69:0/1907339538 learned_addr learned my addr 172.21.15.69:0/1907339538 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:37:39.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.614+0000 7fc532ffd640 1 --2- 172.21.15.69:0/1907339538 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc534106c20 0x7fc5341029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:39.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.614+0000 7fc533fff640 1 --2- 172.21.15.69:0/1907339538 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc534109790 0x7fc534102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:39.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.614+0000 7fc5337fe640 1 -- 172.21.15.69:0/1907339538 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc534106c20 msgr2=0x7fc5341029b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:39.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.614+0000 7fc5337fe640 1 --2- 172.21.15.69:0/1907339538 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc534106c20 0x7fc5341029b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:39.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.614+0000 7fc5337fe640 1 -- 172.21.15.69:0/1907339538 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc534109790 msgr2=0x7fc534102ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:39.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.614+0000 7fc5337fe640 1 --2- 172.21.15.69:0/1907339538 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc534109790 0x7fc534102ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:39.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.614+0000 7fc5337fe640 1 -- 172.21.15.69:0/1907339538 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc534103620 con 0x7fc534101e40 2024-04-25T11:37:39.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.614+0000 7fc5337fe640 1 --2- 172.21.15.69:0/1907339538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc534101e40 0x7fc534104310 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc528031460 tx=0x7fc528004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:39.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.614+0000 7fc530ff9640 1 -- 172.21.15.69:0/1907339538 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52803f030 con 0x7fc534101e40 2024-04-25T11:37:39.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.614+0000 7fc53a1d5640 1 -- 172.21.15.69:0/1907339538 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5341a41e0 con 0x7fc534101e40 2024-04-25T11:37:39.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.614+0000 7fc530ff9640 1 -- 172.21.15.69:0/1907339538 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc528003a80 con 0x7fc534101e40 2024-04-25T11:37:39.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.614+0000 7fc530ff9640 1 -- 172.21.15.69:0/1907339538 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc528035070 con 0x7fc534101e40 2024-04-25T11:37:39.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.614+0000 7fc53a1d5640 1 -- 172.21.15.69:0/1907339538 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5341a4640 con 0x7fc534101e40 2024-04-25T11:37:39.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.615+0000 7fc53a1d5640 1 -- 172.21.15.69:0/1907339538 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5340fee70 con 0x7fc534101e40 2024-04-25T11:37:39.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.617+0000 7fc530ff9640 1 -- 172.21.15.69:0/1907339538 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fc528042030 con 0x7fc534101e40 2024-04-25T11:37:39.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.617+0000 7fc530ff9640 1 --2- 172.21.15.69:0/1907339538 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc51c0afaf0 0x7fc51c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:39.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.618+0000 7fc530ff9640 1 -- 172.21.15.69:0/1907339538 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7fc5280ffbb0 con 0x7fc534101e40 2024-04-25T11:37:39.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.618+0000 7fc532ffd640 1 --2- 172.21.15.69:0/1907339538 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc51c0afaf0 0x7fc51c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:39.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.618+0000 7fc532ffd640 1 --2- 172.21.15.69:0/1907339538 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc51c0afaf0 0x7fc51c0b2000 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc534103cb0 tx=0x7fc52400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:39.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.621+0000 7fc530ff9640 1 -- 172.21.15.69:0/1907339538 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc5280c7d70 con 0x7fc534101e40 2024-04-25T11:37:39.724 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:37:39 smithi152 ceph-mon[32701]: pgmap v707: 129 pgs: 129 active+clean; 7.0 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T11:37:39.724 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:37:39 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/2695678601' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:37:39.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.802+0000 7fc53a1d5640 1 -- 172.21.15.69:0/1907339538 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc5340637a0 con 0x7fc534101e40 2024-04-25T11:37:39.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.803+0000 7fc530ff9640 1 -- 172.21.15.69:0/1907339538 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) ==== 104+0+11 (secure 0 0 0) 0x7fc5280cc070 con 0x7fc534101e40 2024-04-25T11:37:39.791 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-25T11:37:39.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.805+0000 7fc53a1d5640 1 -- 172.21.15.69:0/1907339538 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc51c0afaf0 msgr2=0x7fc51c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:39.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.805+0000 7fc53a1d5640 1 --2- 172.21.15.69:0/1907339538 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc51c0afaf0 0x7fc51c0b2000 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc534103cb0 tx=0x7fc52400b390 comp rx=0 tx=0).stop 2024-04-25T11:37:39.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.805+0000 7fc53a1d5640 1 -- 172.21.15.69:0/1907339538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc534101e40 msgr2=0x7fc534104310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:39.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.805+0000 7fc53a1d5640 1 --2- 172.21.15.69:0/1907339538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc534101e40 0x7fc534104310 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc528031460 tx=0x7fc528004270 comp rx=0 tx=0).stop 2024-04-25T11:37:39.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.805+0000 7fc53a1d5640 1 -- 172.21.15.69:0/1907339538 shutdown_connections 2024-04-25T11:37:39.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.805+0000 7fc53a1d5640 1 --2- 172.21.15.69:0/1907339538 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fc51c0afaf0 0x7fc51c0b2000 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:39.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.805+0000 7fc53a1d5640 1 --2- 172.21.15.69:0/1907339538 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc534109790 0x7fc534102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:39.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.805+0000 7fc53a1d5640 1 --2- 172.21.15.69:0/1907339538 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc534106c20 0x7fc5341029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:39.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.805+0000 7fc53a1d5640 1 --2- 172.21.15.69:0/1907339538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc534101e40 0x7fc534104310 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:39.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.805+0000 7fc53a1d5640 1 -- 172.21.15.69:0/1907339538 >> 172.21.15.69:0/1907339538 conn(0x7fc5340fd350 msgr2=0x7fc534107d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:39.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.805+0000 7fc53a1d5640 1 -- 172.21.15.69:0/1907339538 shutdown_connections 2024-04-25T11:37:39.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.805+0000 7fc53a1d5640 1 -- 172.21.15.69:0/1907339538 wait complete. 2024-04-25T11:37:39.804 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:37:39.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.927+0000 7feb0c284640 1 -- 172.21.15.69:0/1403503469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb04104470 msgr2=0x7feb0410abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:39.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.927+0000 7feb0c284640 1 --2- 172.21.15.69:0/1403503469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb04104470 0x7feb0410abc0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7feafc009ca0 tx=0x7feafc02a8c0 comp rx=0 tx=0).stop 2024-04-25T11:37:39.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.927+0000 7feb0c284640 1 -- 172.21.15.69:0/1403503469 shutdown_connections 2024-04-25T11:37:39.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.927+0000 7feb0c284640 1 --2- 172.21.15.69:0/1403503469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb04104470 0x7feb0410abc0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:39.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.927+0000 7feb0c284640 1 --2- 172.21.15.69:0/1403503469 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb04103950 0x7feb04103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:39.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.927+0000 7feb0c284640 1 --2- 172.21.15.69:0/1403503469 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb04102f70 0x7feb04103380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:39.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.927+0000 7feb0c284640 1 -- 172.21.15.69:0/1403503469 >> 172.21.15.69:0/1403503469 conn(0x7feb040fd350 msgr2=0x7feb040ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:39.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.927+0000 7feb0c284640 1 -- 172.21.15.69:0/1403503469 shutdown_connections 2024-04-25T11:37:39.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.927+0000 7feb0c284640 1 -- 172.21.15.69:0/1403503469 wait complete. 2024-04-25T11:37:39.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.928+0000 7feb0c284640 1 Processor -- start 2024-04-25T11:37:39.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.928+0000 7feb0c284640 1 -- start start 2024-04-25T11:37:39.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.928+0000 7feb0c284640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb04102f70 0x7feb0419a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:39.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.928+0000 7feb0c284640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb04103950 0x7feb0419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:39.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.928+0000 7feb0c284640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb04104470 0x7feb041a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:39.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.928+0000 7feb0c284640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7feb0410f550 con 0x7feb04104470 2024-04-25T11:37:39.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.928+0000 7feb0c284640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7feb0410f3d0 con 0x7feb04102f70 2024-04-25T11:37:39.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.928+0000 7feb0c284640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7feb0410f6d0 con 0x7feb04103950 2024-04-25T11:37:39.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.928+0000 7feb0a821640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb04104470 0x7feb041a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:39.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.928+0000 7feb0a821640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb04104470 0x7feb041a1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48810/0 (socket says 172.21.15.69:48810) 2024-04-25T11:37:39.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.928+0000 7feb0a821640 1 -- 172.21.15.69:0/1205837835 learned_addr learned my addr 172.21.15.69:0/1205837835 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:37:39.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.928+0000 7feb0981f640 1 --2- 172.21.15.69:0/1205837835 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb04103950 0x7feb0419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:39.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.928+0000 7feb0a020640 1 --2- 172.21.15.69:0/1205837835 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb04102f70 0x7feb0419a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:39.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.928+0000 7feb0a821640 1 -- 172.21.15.69:0/1205837835 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb04103950 msgr2=0x7feb0419a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:39.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.928+0000 7feb0a821640 1 --2- 172.21.15.69:0/1205837835 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb04103950 0x7feb0419a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:39.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.928+0000 7feb0a821640 1 -- 172.21.15.69:0/1205837835 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb04102f70 msgr2=0x7feb0419a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:39.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.928+0000 7feb0a821640 1 --2- 172.21.15.69:0/1205837835 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb04102f70 0x7feb0419a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:39.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.928+0000 7feb0a821640 1 -- 172.21.15.69:0/1205837835 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb041a1bc0 con 0x7feb04104470 2024-04-25T11:37:39.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.928+0000 7feb0a020640 1 --2- 172.21.15.69:0/1205837835 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb04102f70 0x7feb0419a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:37:39.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.929+0000 7feb0a821640 1 --2- 172.21.15.69:0/1205837835 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb04104470 0x7feb041a1550 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7feafc031040 tx=0x7feafc003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:39.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.929+0000 7feafb7fe640 1 -- 172.21.15.69:0/1205837835 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feafc004040 con 0x7feb04104470 2024-04-25T11:37:39.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.929+0000 7feafb7fe640 1 -- 172.21.15.69:0/1205837835 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feafc004670 con 0x7feb04104470 2024-04-25T11:37:39.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.929+0000 7feb0c284640 1 -- 172.21.15.69:0/1205837835 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb041a1e50 con 0x7feb04104470 2024-04-25T11:37:39.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.929+0000 7feafb7fe640 1 -- 172.21.15.69:0/1205837835 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feafc030d50 con 0x7feb04104470 2024-04-25T11:37:39.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.929+0000 7feb0c284640 1 -- 172.21.15.69:0/1205837835 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb041a22b0 con 0x7feb04104470 2024-04-25T11:37:39.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.930+0000 7feb0c284640 1 -- 172.21.15.69:0/1205837835 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb0406a190 con 0x7feb04104470 2024-04-25T11:37:39.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.931+0000 7feafb7fe640 1 -- 172.21.15.69:0/1205837835 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7feafc0305c0 con 0x7feb04104470 2024-04-25T11:37:39.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.932+0000 7feafb7fe640 1 --2- 172.21.15.69:0/1205837835 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7feaf00afaf0 0x7feaf00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:39.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.932+0000 7feb0a020640 1 --2- 172.21.15.69:0/1205837835 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7feaf00afaf0 0x7feaf00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:39.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.932+0000 7feafb7fe640 1 -- 172.21.15.69:0/1205837835 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7feafc1006f0 con 0x7feb04104470 2024-04-25T11:37:39.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.932+0000 7feb0a020640 1 --2- 172.21.15.69:0/1205837835 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7feaf00afaf0 0x7feaf00b2000 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7feb00005fd0 tx=0x7feb00005ea0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:39.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:39.936+0000 7feafb7fe640 1 -- 172.21.15.69:0/1205837835 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feb0406a190 con 0x7feb04104470 2024-04-25T11:37:40.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.114+0000 7feb0c284640 1 -- 172.21.15.69:0/1205837835 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7feb040637a0 con 0x7feb04104470 2024-04-25T11:37:40.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.114+0000 7feafb7fe640 1 -- 172.21.15.69:0/1205837835 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) ==== 100+0+11 (secure 0 0 0) 0x7feafc0c8f00 con 0x7feb04104470 2024-04-25T11:37:40.102 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-04-25T11:37:40.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.117+0000 7feb0c284640 1 -- 172.21.15.69:0/1205837835 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7feaf00afaf0 msgr2=0x7feaf00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:40.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.117+0000 7feb0c284640 1 --2- 172.21.15.69:0/1205837835 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7feaf00afaf0 0x7feaf00b2000 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7feb00005fd0 tx=0x7feb00005ea0 comp rx=0 tx=0).stop 2024-04-25T11:37:40.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.117+0000 7feb0c284640 1 -- 172.21.15.69:0/1205837835 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb04104470 msgr2=0x7feb041a1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:40.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.117+0000 7feb0c284640 1 --2- 172.21.15.69:0/1205837835 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb04104470 0x7feb041a1550 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7feafc031040 tx=0x7feafc003ba0 comp rx=0 tx=0).stop 2024-04-25T11:37:40.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.117+0000 7feb0c284640 1 -- 172.21.15.69:0/1205837835 shutdown_connections 2024-04-25T11:37:40.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.117+0000 7feb0c284640 1 --2- 172.21.15.69:0/1205837835 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7feaf00afaf0 0x7feaf00b2000 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:40.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.117+0000 7feb0c284640 1 --2- 172.21.15.69:0/1205837835 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb04104470 0x7feb041a1550 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:40.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.117+0000 7feb0c284640 1 --2- 172.21.15.69:0/1205837835 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb04103950 0x7feb0419a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:40.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.117+0000 7feb0c284640 1 --2- 172.21.15.69:0/1205837835 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb04102f70 0x7feb0419a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:40.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.117+0000 7feb0c284640 1 -- 172.21.15.69:0/1205837835 >> 172.21.15.69:0/1205837835 conn(0x7feb040fd350 msgr2=0x7feb04106af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:40.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.117+0000 7feb0c284640 1 -- 172.21.15.69:0/1205837835 shutdown_connections 2024-04-25T11:37:40.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.117+0000 7feb0c284640 1 -- 172.21.15.69:0/1205837835 wait complete. 2024-04-25T11:37:40.116 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T11:37:40.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.239+0000 7faea5a9f640 1 -- 172.21.15.69:0/117083731 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faea0109a40 msgr2=0x7faea010be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:40.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.239+0000 7faea5a9f640 1 --2- 172.21.15.69:0/117083731 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faea0109a40 0x7faea010be60 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fae98009ca0 tx=0x7fae9802a8c0 comp rx=0 tx=0).stop 2024-04-25T11:37:40.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.239+0000 7faea5a9f640 1 -- 172.21.15.69:0/117083731 shutdown_connections 2024-04-25T11:37:40.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.239+0000 7faea5a9f640 1 --2- 172.21.15.69:0/117083731 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faea0109a40 0x7faea010be60 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:40.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.239+0000 7faea5a9f640 1 --2- 172.21.15.69:0/117083731 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faea0104d40 0x7faea0105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:40.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.239+0000 7faea5a9f640 1 --2- 172.21.15.69:0/117083731 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faea0103b00 0x7faea0104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:40.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.239+0000 7faea5a9f640 1 -- 172.21.15.69:0/117083731 >> 172.21.15.69:0/117083731 conn(0x7faea00fecf0 msgr2=0x7faea01011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:40.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.239+0000 7faea5a9f640 1 -- 172.21.15.69:0/117083731 shutdown_connections 2024-04-25T11:37:40.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.239+0000 7faea5a9f640 1 -- 172.21.15.69:0/117083731 wait complete. 2024-04-25T11:37:40.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.240+0000 7faea5a9f640 1 Processor -- start 2024-04-25T11:37:40.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.240+0000 7faea5a9f640 1 -- start start 2024-04-25T11:37:40.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.240+0000 7faea5a9f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faea0103b00 0x7faea019a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:40.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.240+0000 7faea5a9f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faea0104d40 0x7faea019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:40.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.240+0000 7faea5a9f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faea0109a40 0x7faea01a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:40.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.240+0000 7faea5a9f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7faea010f530 con 0x7faea0104d40 2024-04-25T11:37:40.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.240+0000 7faea5a9f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7faea010f3b0 con 0x7faea0109a40 2024-04-25T11:37:40.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.240+0000 7faea5a9f640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7faea010f6b0 con 0x7faea0103b00 2024-04-25T11:37:40.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.240+0000 7fae9effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faea0104d40 0x7faea019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:40.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.240+0000 7fae9effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faea0104d40 0x7faea019a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48814/0 (socket says 172.21.15.69:48814) 2024-04-25T11:37:40.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.240+0000 7fae9effd640 1 -- 172.21.15.69:0/3763292593 learned_addr learned my addr 172.21.15.69:0/3763292593 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:37:40.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.241+0000 7fae9ffff640 1 --2- 172.21.15.69:0/3763292593 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faea0109a40 0x7faea01a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:40.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.241+0000 7fae9f7fe640 1 --2- 172.21.15.69:0/3763292593 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faea0103b00 0x7faea019a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:40.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.241+0000 7fae9effd640 1 -- 172.21.15.69:0/3763292593 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faea0103b00 msgr2=0x7faea019a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:40.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.241+0000 7fae9effd640 1 --2- 172.21.15.69:0/3763292593 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faea0103b00 0x7faea019a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:40.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.241+0000 7fae9effd640 1 -- 172.21.15.69:0/3763292593 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faea0109a40 msgr2=0x7faea01a15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:40.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.241+0000 7fae9effd640 1 --2- 172.21.15.69:0/3763292593 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faea0109a40 0x7faea01a15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:40.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.241+0000 7fae9effd640 1 -- 172.21.15.69:0/3763292593 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faea01a1c40 con 0x7faea0104d40 2024-04-25T11:37:40.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.241+0000 7fae9ffff640 1 --2- 172.21.15.69:0/3763292593 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faea0109a40 0x7faea01a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:37:40.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.241+0000 7fae9effd640 1 --2- 172.21.15.69:0/3763292593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faea0104d40 0x7faea019a640 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fae9400d950 tx=0x7fae9400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:40.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.241+0000 7fae9cff9640 1 -- 172.21.15.69:0/3763292593 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae94017030 con 0x7faea0104d40 2024-04-25T11:37:40.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.241+0000 7fae9cff9640 1 -- 172.21.15.69:0/3763292593 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae94004020 con 0x7faea0104d40 2024-04-25T11:37:40.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.241+0000 7faea5a9f640 1 -- 172.21.15.69:0/3763292593 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faea01a1f30 con 0x7faea0104d40 2024-04-25T11:37:40.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.241+0000 7fae9cff9640 1 -- 172.21.15.69:0/3763292593 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae940055d0 con 0x7faea0104d40 2024-04-25T11:37:40.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.241+0000 7faea5a9f640 1 -- 172.21.15.69:0/3763292593 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faea01a23a0 con 0x7faea0104d40 2024-04-25T11:37:40.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.242+0000 7faea5a9f640 1 -- 172.21.15.69:0/3763292593 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faea0108850 con 0x7faea0104d40 2024-04-25T11:37:40.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.244+0000 7fae9cff9640 1 -- 172.21.15.69:0/3763292593 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fae94009a30 con 0x7faea0104d40 2024-04-25T11:37:40.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.245+0000 7fae9cff9640 1 --2- 172.21.15.69:0/3763292593 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fae880afaf0 0x7fae880b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:40.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.245+0000 7fae9f7fe640 1 --2- 172.21.15.69:0/3763292593 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fae880afaf0 0x7fae880b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:40.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.245+0000 7fae9cff9640 1 -- 172.21.15.69:0/3763292593 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7fae940df0f0 con 0x7faea0104d40 2024-04-25T11:37:40.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.245+0000 7fae9f7fe640 1 --2- 172.21.15.69:0/3763292593 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fae880afaf0 0x7fae880b2000 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fae900058c0 tx=0x7fae9000a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:40.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.251+0000 7fae9cff9640 1 -- 172.21.15.69:0/3763292593 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fae940a7900 con 0x7faea0104d40 2024-04-25T11:37:40.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.428+0000 7faea5a9f640 1 -- 172.21.15.69:0/3763292593 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7faea002db50 con 0x7faea0104d40 2024-04-25T11:37:40.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.428+0000 7fae9cff9640 1 -- 172.21.15.69:0/3763292593 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v84) ==== 103+0+11 (secure 0 0 0) 0x7fae940a7200 con 0x7faea0104d40 2024-04-25T11:37:40.416 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-25T11:37:40.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.430+0000 7faea5a9f640 1 -- 172.21.15.69:0/3763292593 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fae880afaf0 msgr2=0x7fae880b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:40.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.430+0000 7faea5a9f640 1 --2- 172.21.15.69:0/3763292593 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fae880afaf0 0x7fae880b2000 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fae900058c0 tx=0x7fae9000a4c0 comp rx=0 tx=0).stop 2024-04-25T11:37:40.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.430+0000 7faea5a9f640 1 -- 172.21.15.69:0/3763292593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faea0104d40 msgr2=0x7faea019a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:40.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.430+0000 7faea5a9f640 1 --2- 172.21.15.69:0/3763292593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faea0104d40 0x7faea019a640 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fae9400d950 tx=0x7fae9400de10 comp rx=0 tx=0).stop 2024-04-25T11:37:40.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.430+0000 7faea5a9f640 1 -- 172.21.15.69:0/3763292593 shutdown_connections 2024-04-25T11:37:40.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.430+0000 7faea5a9f640 1 --2- 172.21.15.69:0/3763292593 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fae880afaf0 0x7fae880b2000 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:40.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.430+0000 7faea5a9f640 1 --2- 172.21.15.69:0/3763292593 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faea0109a40 0x7faea01a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:40.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.430+0000 7faea5a9f640 1 --2- 172.21.15.69:0/3763292593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faea0104d40 0x7faea019a640 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:40.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.430+0000 7faea5a9f640 1 --2- 172.21.15.69:0/3763292593 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faea0103b00 0x7faea019a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:40.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.430+0000 7faea5a9f640 1 -- 172.21.15.69:0/3763292593 >> 172.21.15.69:0/3763292593 conn(0x7faea00fecf0 msgr2=0x7faea0105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:40.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.430+0000 7faea5a9f640 1 -- 172.21.15.69:0/3763292593 shutdown_connections 2024-04-25T11:37:40.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.430+0000 7faea5a9f640 1 -- 172.21.15.69:0/3763292593 wait complete. 2024-04-25T11:37:40.428 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:37:40 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/159174650' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:37:40.428 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:37:40 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/1907339538' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:37:40.428 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:37:40 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/1205837835' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:37:40.429 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-25T11:37:40.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.550+0000 7f7052434640 1 -- 172.21.15.69:0/2147076123 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f704c108910 msgr2=0x7f704c10ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:40.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.550+0000 7f7052434640 1 --2- 172.21.15.69:0/2147076123 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f704c108910 0x7f704c10ad30 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7044009ca0 tx=0x7f704402a8c0 comp rx=0 tx=0).stop 2024-04-25T11:37:40.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.550+0000 7f7052434640 1 -- 172.21.15.69:0/2147076123 shutdown_connections 2024-04-25T11:37:40.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.550+0000 7f7052434640 1 --2- 172.21.15.69:0/2147076123 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f704c108910 0x7f704c10ad30 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:40.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.550+0000 7f7052434640 1 --2- 172.21.15.69:0/2147076123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f704c103c10 0x7f704c104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:40.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.550+0000 7f7052434640 1 --2- 172.21.15.69:0/2147076123 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f704c1029d0 0x7f704c1036d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:40.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.550+0000 7f7052434640 1 -- 172.21.15.69:0/2147076123 >> 172.21.15.69:0/2147076123 conn(0x7f704c0fdba0 msgr2=0x7f704c100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:40.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.550+0000 7f7052434640 1 -- 172.21.15.69:0/2147076123 shutdown_connections 2024-04-25T11:37:40.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.550+0000 7f7052434640 1 -- 172.21.15.69:0/2147076123 wait complete. 2024-04-25T11:37:40.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.550+0000 7f7052434640 1 Processor -- start 2024-04-25T11:37:40.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.550+0000 7f7052434640 1 -- start start 2024-04-25T11:37:40.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.551+0000 7f7052434640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f704c1029d0 0x7f704c19a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:40.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.551+0000 7f7052434640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f704c103c10 0x7f704c19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:40.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.551+0000 7f7052434640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f704c108910 0x7f704c1a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:40.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.551+0000 7f7052434640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f704c10e400 con 0x7f704c108910 2024-04-25T11:37:40.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.551+0000 7f7052434640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f704c10e280 con 0x7f704c103c10 2024-04-25T11:37:40.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.551+0000 7f7052434640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f704c10e580 con 0x7f704c1029d0 2024-04-25T11:37:40.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.551+0000 7f70509d1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f704c108910 0x7f704c1a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:40.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.551+0000 7f70509d1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f704c108910 0x7f704c1a15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48830/0 (socket says 172.21.15.69:48830) 2024-04-25T11:37:40.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.551+0000 7f70509d1640 1 -- 172.21.15.69:0/3659100046 learned_addr learned my addr 172.21.15.69:0/3659100046 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:37:40.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.551+0000 7f704bfff640 1 --2- 172.21.15.69:0/3659100046 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f704c1029d0 0x7f704c19a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:40.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.551+0000 7f704b7fe640 1 --2- 172.21.15.69:0/3659100046 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f704c103c10 0x7f704c19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:40.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.551+0000 7f70509d1640 1 -- 172.21.15.69:0/3659100046 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f704c1029d0 msgr2=0x7f704c19a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:40.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.551+0000 7f70509d1640 1 --2- 172.21.15.69:0/3659100046 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f704c1029d0 0x7f704c19a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:40.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.551+0000 7f70509d1640 1 -- 172.21.15.69:0/3659100046 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f704c103c10 msgr2=0x7f704c19a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:40.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.551+0000 7f70509d1640 1 --2- 172.21.15.69:0/3659100046 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f704c103c10 0x7f704c19a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:40.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.551+0000 7f70509d1640 1 -- 172.21.15.69:0/3659100046 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f704c1a1c30 con 0x7f704c108910 2024-04-25T11:37:40.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.551+0000 7f70509d1640 1 --2- 172.21.15.69:0/3659100046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f704c108910 0x7f704c1a15c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7044031040 tx=0x7f7044003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:40.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.552+0000 7f70497fa640 1 -- 172.21.15.69:0/3659100046 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7044004040 con 0x7f704c108910 2024-04-25T11:37:40.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.552+0000 7f7052434640 1 -- 172.21.15.69:0/3659100046 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f704c1a1ec0 con 0x7f704c108910 2024-04-25T11:37:40.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.552+0000 7f70497fa640 1 -- 172.21.15.69:0/3659100046 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7044004670 con 0x7f704c108910 2024-04-25T11:37:40.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.552+0000 7f70497fa640 1 -- 172.21.15.69:0/3659100046 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7044030d50 con 0x7f704c108910 2024-04-25T11:37:40.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.552+0000 7f7052434640 1 -- 172.21.15.69:0/3659100046 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f704c1a2320 con 0x7f704c108910 2024-04-25T11:37:40.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.553+0000 7f7052434640 1 -- 172.21.15.69:0/3659100046 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f704c1a2580 con 0x7f704c108910 2024-04-25T11:37:40.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.554+0000 7f70497fa640 1 -- 172.21.15.69:0/3659100046 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f70440305c0 con 0x7f704c108910 2024-04-25T11:37:40.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.555+0000 7f70497fa640 1 --2- 172.21.15.69:0/3659100046 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f70340afaf0 0x7f70340b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:40.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.555+0000 7f704bfff640 1 --2- 172.21.15.69:0/3659100046 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f70340afaf0 0x7f70340b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:40.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.555+0000 7f70497fa640 1 -- 172.21.15.69:0/3659100046 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f7044100730 con 0x7f704c108910 2024-04-25T11:37:40.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.555+0000 7f704bfff640 1 --2- 172.21.15.69:0/3659100046 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f70340afaf0 0x7f70340b2000 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f703c005fd0 tx=0x7f703c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:40.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.558+0000 7f70497fa640 1 -- 172.21.15.69:0/3659100046 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f70440c8f40 con 0x7f704c108910 2024-04-25T11:37:40.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:37:40 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/159174650' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:37:40.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:37:40 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/1907339538' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:37:40.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:37:40 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/1205837835' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:37:40.724 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:37:40 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/159174650' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:37:40.724 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:37:40 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/1907339538' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:37:40.725 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:37:40 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/1205837835' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:37:40.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.785+0000 7f7052434640 1 -- 172.21.15.69:0/3659100046 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f704c1a2870 con 0x7f704c108910 2024-04-25T11:37:40.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.794+0000 7f70497fa640 1 -- 172.21.15.69:0/3659100046 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+803126 (secure 0 0 0) 0x7f70440c8840 con 0x7f704c108910 2024-04-25T11:37:40.792 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:37:40.792 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-25T11:37:40.792 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 20, 2024-04-25T11:37:40.792 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.792 INFO:teuthology.orchestra.run.smithi069.stdout: "active_gid": 14152, 2024-04-25T11:37:40.792 INFO:teuthology.orchestra.run.smithi069.stdout: "active_name": "x", 2024-04-25T11:37:40.792 INFO:teuthology.orchestra.run.smithi069.stdout: "active_addrs": { 2024-04-25T11:37:40.792 INFO:teuthology.orchestra.run.smithi069.stdout: "addrvec": [ 2024-04-25T11:37:40.792 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:40.793 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v2", 2024-04-25T11:37:40.793 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "172.21.15.69:6800", 2024-04-25T11:37:40.793 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 4220065409 2024-04-25T11:37:40.793 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.793 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:40.793 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v1", 2024-04-25T11:37:40.793 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "172.21.15.69:6801", 2024-04-25T11:37:40.793 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 4220065409 2024-04-25T11:37:40.793 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:40.793 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-04-25T11:37:40.793 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.793 INFO:teuthology.orchestra.run.smithi069.stdout: "active_addr": "172.21.15.69:6801/4220065409", 2024-04-25T11:37:40.793 INFO:teuthology.orchestra.run.smithi069.stdout: "active_change": "2024-04-25T11:16:31.688322+0000", 2024-04-25T11:37:40.793 INFO:teuthology.orchestra.run.smithi069.stdout: "active_mgr_features": 4540701547738038271, 2024-04-25T11:37:40.793 INFO:teuthology.orchestra.run.smithi069.stdout: "available": true, 2024-04-25T11:37:40.793 INFO:teuthology.orchestra.run.smithi069.stdout: "standbys": [ 2024-04-25T11:37:40.794 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:40.794 INFO:teuthology.orchestra.run.smithi069.stdout: "gid": 24109, 2024-04-25T11:37:40.794 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "z", 2024-04-25T11:37:40.794 INFO:teuthology.orchestra.run.smithi069.stdout: "mgr_features": 4540701547738038271, 2024-04-25T11:37:40.794 INFO:teuthology.orchestra.run.smithi069.stdout: "available_modules": [ 2024-04-25T11:37:40.794 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:40.794 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "alerts", 2024-04-25T11:37:40.794 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:40.794 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:40.794 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:40.794 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-04-25T11:37:40.794 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-04-25T11:37:40.794 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:40.795 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.795 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.795 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-04-25T11:37:40.795 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.795 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.795 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.795 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T11:37:40.795 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.795 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.795 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.795 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.795 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:40.795 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:40.795 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.795 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.796 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.796 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:40.796 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.796 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.796 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:40.796 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:40.796 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:40.796 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:40.796 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:40.796 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:40.796 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:40.796 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:40.796 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.797 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.797 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.797 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.797 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.797 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:40.797 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:40.797 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.797 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.797 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.797 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:40.797 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.797 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.797 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.797 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.798 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.798 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.798 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.798 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.798 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:40.798 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:40.798 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.798 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.798 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.798 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:40.798 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.798 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.798 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:40.799 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:40.799 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:40.799 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:40.799 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:40.799 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:40.799 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:40.799 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:40.799 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.799 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.799 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.799 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.799 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.800 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:40.800 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:40.800 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.800 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.800 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.800 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:40.800 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.800 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.800 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.800 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.800 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.800 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.800 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.800 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.800 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_destination": { 2024-04-25T11:37:40.801 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_destination", 2024-04-25T11:37:40.801 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.801 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.801 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.801 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:40.801 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.801 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.801 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.801 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Email address to send alerts to", 2024-04-25T11:37:40.801 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.801 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.801 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.802 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.802 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_from_name": { 2024-04-25T11:37:40.802 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_from_name", 2024-04-25T11:37:40.802 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.802 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.802 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.802 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "Ceph", 2024-04-25T11:37:40.802 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.802 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.802 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.802 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Email From: name", 2024-04-25T11:37:40.802 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.802 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.802 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.803 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.803 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_host": { 2024-04-25T11:37:40.803 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_host", 2024-04-25T11:37:40.803 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.803 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.803 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.803 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:40.803 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.803 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.803 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.803 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP server", 2024-04-25T11:37:40.803 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.803 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.804 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.804 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.804 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_password": { 2024-04-25T11:37:40.804 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_password", 2024-04-25T11:37:40.804 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.804 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.804 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.804 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:40.804 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.804 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.804 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.804 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Password to authenticate with", 2024-04-25T11:37:40.804 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.804 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.805 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.805 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.805 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_port": { 2024-04-25T11:37:40.805 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_port", 2024-04-25T11:37:40.805 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:40.805 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.805 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.805 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "465", 2024-04-25T11:37:40.805 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.805 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.805 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.805 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP port", 2024-04-25T11:37:40.805 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.805 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.805 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.806 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.806 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_sender": { 2024-04-25T11:37:40.806 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_sender", 2024-04-25T11:37:40.806 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.806 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.806 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.806 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:40.806 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.806 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.806 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.806 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP envelope sender", 2024-04-25T11:37:40.806 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.806 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.807 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.807 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.807 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_ssl": { 2024-04-25T11:37:40.807 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_ssl", 2024-04-25T11:37:40.807 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.807 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.807 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.807 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:40.807 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.807 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.807 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.807 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T11:37:40.807 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.808 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.808 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.808 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.808 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_user": { 2024-04-25T11:37:40.808 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_user", 2024-04-25T11:37:40.808 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.808 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.808 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.808 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:40.808 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.808 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.808 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.808 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "User to authenticate as", 2024-04-25T11:37:40.809 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.809 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.809 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.809 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.809 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:40.809 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:40.809 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:40.809 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:40.809 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.809 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:40.809 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.809 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.810 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.810 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.810 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.810 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.810 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.810 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:40.810 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:40.810 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.810 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:40.810 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "balancer", 2024-04-25T11:37:40.810 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:40.810 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:40.810 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:40.810 INFO:teuthology.orchestra.run.smithi069.stdout: "active": { 2024-04-25T11:37:40.810 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "active", 2024-04-25T11:37:40.811 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.811 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.811 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.811 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:40.811 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.811 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.811 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.811 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T11:37:40.811 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.811 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.811 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.811 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.811 INFO:teuthology.orchestra.run.smithi069.stdout: "begin_time": { 2024-04-25T11:37:40.811 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "begin_time", 2024-04-25T11:37:40.812 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.812 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.812 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.812 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0000", 2024-04-25T11:37:40.812 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.812 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.812 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.812 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T11:37:40.812 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T11:37:40.812 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.812 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.812 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.812 INFO:teuthology.orchestra.run.smithi069.stdout: "begin_weekday": { 2024-04-25T11:37:40.812 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "begin_weekday", 2024-04-25T11:37:40.813 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-04-25T11:37:40.813 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.813 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.813 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:40.813 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0", 2024-04-25T11:37:40.813 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "6", 2024-04-25T11:37:40.813 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.813 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T11:37:40.813 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T11:37:40.813 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.813 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.813 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.813 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_max_iterations": { 2024-04-25T11:37:40.813 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_max_iterations", 2024-04-25T11:37:40.813 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-04-25T11:37:40.814 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.814 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.814 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "25", 2024-04-25T11:37:40.814 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-04-25T11:37:40.814 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "250", 2024-04-25T11:37:40.814 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.814 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T11:37:40.814 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.814 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.814 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.814 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.815 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_metrics": { 2024-04-25T11:37:40.815 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_metrics", 2024-04-25T11:37:40.815 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.815 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.815 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.815 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T11:37:40.815 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.815 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.815 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.815 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T11:37:40.815 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T11:37:40.815 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.815 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.815 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.816 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_step": { 2024-04-25T11:37:40.816 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_step", 2024-04-25T11:37:40.816 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-04-25T11:37:40.816 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.816 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.816 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0.5", 2024-04-25T11:37:40.816 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0.001", 2024-04-25T11:37:40.816 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "0.999", 2024-04-25T11:37:40.816 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.816 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "aggressiveness of optimization", 2024-04-25T11:37:40.816 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T11:37:40.816 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.816 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.817 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.817 INFO:teuthology.orchestra.run.smithi069.stdout: "end_time": { 2024-04-25T11:37:40.817 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "end_time", 2024-04-25T11:37:40.817 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.817 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.817 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.817 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "2359", 2024-04-25T11:37:40.817 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.817 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.817 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.817 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T11:37:40.817 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T11:37:40.818 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.818 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.818 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.818 INFO:teuthology.orchestra.run.smithi069.stdout: "end_weekday": { 2024-04-25T11:37:40.818 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "end_weekday", 2024-04-25T11:37:40.818 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-04-25T11:37:40.818 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.818 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.818 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:40.818 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0", 2024-04-25T11:37:40.818 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "6", 2024-04-25T11:37:40.818 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.818 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T11:37:40.818 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T11:37:40.819 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.819 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.819 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.819 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:40.819 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:40.819 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.819 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.819 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.819 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:40.819 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.819 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.819 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:40.819 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:40.820 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:40.820 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:40.820 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:40.820 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:40.820 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:40.820 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:40.820 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.820 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.820 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.820 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.820 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.820 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:40.820 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:40.821 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.821 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.821 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.821 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:40.821 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.821 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.821 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.821 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.821 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.821 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.821 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.821 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.821 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:40.821 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:40.821 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.822 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.822 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.822 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:40.822 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.822 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.822 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:40.822 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:40.822 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:40.822 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:40.822 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:40.822 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:40.822 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:40.822 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:40.822 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.823 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.823 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.823 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.823 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.823 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:40.823 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:40.823 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.823 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.823 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.823 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:40.823 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.823 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.823 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.823 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.824 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.824 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.824 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.824 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.824 INFO:teuthology.orchestra.run.smithi069.stdout: "min_score": { 2024-04-25T11:37:40.824 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "min_score", 2024-04-25T11:37:40.824 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-04-25T11:37:40.824 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.824 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.824 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:40.824 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.824 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.824 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.825 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T11:37:40.825 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.825 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.825 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.825 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.825 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": { 2024-04-25T11:37:40.825 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mode", 2024-04-25T11:37:40.825 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.825 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.825 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.825 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "upmap", 2024-04-25T11:37:40.825 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.825 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.826 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:40.826 INFO:teuthology.orchestra.run.smithi069.stdout: "crush-compat", 2024-04-25T11:37:40.826 INFO:teuthology.orchestra.run.smithi069.stdout: "none", 2024-04-25T11:37:40.826 INFO:teuthology.orchestra.run.smithi069.stdout: "read", 2024-04-25T11:37:40.826 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap", 2024-04-25T11:37:40.826 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap-read" 2024-04-25T11:37:40.826 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:40.826 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Balancer mode", 2024-04-25T11:37:40.826 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.826 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.826 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.826 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.826 INFO:teuthology.orchestra.run.smithi069.stdout: "pool_ids": { 2024-04-25T11:37:40.826 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pool_ids", 2024-04-25T11:37:40.827 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.827 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.827 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.827 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:40.827 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.827 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.827 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.827 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T11:37:40.827 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.827 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.827 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.827 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.827 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-04-25T11:37:40.827 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-04-25T11:37:40.827 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:40.827 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.828 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.828 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-04-25T11:37:40.828 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.828 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.828 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.828 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T11:37:40.828 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.828 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.828 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.828 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.828 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:40.828 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:40.828 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:40.828 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:40.828 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.829 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:40.829 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.829 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.829 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.829 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.829 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.829 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.829 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.829 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.829 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap_max_deviation": { 2024-04-25T11:37:40.829 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "upmap_max_deviation", 2024-04-25T11:37:40.829 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:40.829 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.829 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.830 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-04-25T11:37:40.830 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-04-25T11:37:40.830 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.830 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.830 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T11:37:40.830 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T11:37:40.830 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.830 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.830 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.830 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap_max_optimizations": { 2024-04-25T11:37:40.830 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "upmap_max_optimizations", 2024-04-25T11:37:40.830 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-04-25T11:37:40.831 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.831 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.831 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-25T11:37:40.831 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.831 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.831 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.831 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T11:37:40.831 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.831 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.831 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.831 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:40.831 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:40.831 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.831 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:40.831 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cephadm", 2024-04-25T11:37:40.832 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:40.832 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:40.832 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:40.832 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_down_multiplier": { 2024-04-25T11:37:40.832 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_down_multiplier", 2024-04-25T11:37:40.832 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-04-25T11:37:40.832 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.832 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.832 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3.0", 2024-04-25T11:37:40.832 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.832 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.832 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.832 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T11:37:40.832 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.833 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.833 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.833 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.833 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_refresh_rate": { 2024-04-25T11:37:40.833 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_refresh_rate", 2024-04-25T11:37:40.833 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:40.833 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.833 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.833 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "20", 2024-04-25T11:37:40.833 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.833 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.833 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.833 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T11:37:40.833 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.834 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.834 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.834 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.834 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_starting_port": { 2024-04-25T11:37:40.834 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_starting_port", 2024-04-25T11:37:40.834 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:40.834 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.834 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.834 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "4721", 2024-04-25T11:37:40.834 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.834 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.834 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.834 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T11:37:40.835 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.835 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.835 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.835 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.835 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_ptrace": { 2024-04-25T11:37:40.835 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_ptrace", 2024-04-25T11:37:40.835 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.835 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.835 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.835 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:40.835 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.835 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.835 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.836 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T11:37:40.836 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T11:37:40.836 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.836 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.836 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.836 INFO:teuthology.orchestra.run.smithi069.stdout: "autotune_interval": { 2024-04-25T11:37:40.836 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "autotune_interval", 2024-04-25T11:37:40.836 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:40.836 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.836 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.836 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-04-25T11:37:40.836 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.836 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.836 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.836 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T11:37:40.837 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.837 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.837 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.837 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.837 INFO:teuthology.orchestra.run.smithi069.stdout: "autotune_memory_target_ratio": { 2024-04-25T11:37:40.837 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T11:37:40.837 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-04-25T11:37:40.837 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.837 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.837 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0.7", 2024-04-25T11:37:40.837 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.837 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.837 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.837 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T11:37:40.838 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.838 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.838 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.838 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.838 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm_log_destination": { 2024-04-25T11:37:40.838 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cephadm_log_destination", 2024-04-25T11:37:40.838 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.838 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.838 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.838 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:40.838 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.838 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.838 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:40.838 INFO:teuthology.orchestra.run.smithi069.stdout: "file", 2024-04-25T11:37:40.839 INFO:teuthology.orchestra.run.smithi069.stdout: "file,syslog", 2024-04-25T11:37:40.839 INFO:teuthology.orchestra.run.smithi069.stdout: "syslog" 2024-04-25T11:37:40.839 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:40.839 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T11:37:40.839 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.839 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.839 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.839 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.839 INFO:teuthology.orchestra.run.smithi069.stdout: "cgroups_split": { 2024-04-25T11:37:40.839 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cgroups_split", 2024-04-25T11:37:40.839 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.839 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.839 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.840 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:40.840 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.840 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.840 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.840 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T11:37:40.840 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.840 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.840 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.840 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.840 INFO:teuthology.orchestra.run.smithi069.stdout: "config_checks_enabled": { 2024-04-25T11:37:40.840 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_checks_enabled", 2024-04-25T11:37:40.840 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.840 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.841 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.841 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:40.841 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.841 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.841 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.841 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T11:37:40.841 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.841 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.841 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.841 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.841 INFO:teuthology.orchestra.run.smithi069.stdout: "config_dashboard": { 2024-04-25T11:37:40.842 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_dashboard", 2024-04-25T11:37:40.842 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.842 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.842 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.842 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:40.842 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.842 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.842 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.842 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T11:37:40.842 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.842 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.842 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.842 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.842 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_alertmanager": { 2024-04-25T11:37:40.843 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_alertmanager", 2024-04-25T11:37:40.843 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.843 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.843 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.843 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T11:37:40.843 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.843 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.843 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.843 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-04-25T11:37:40.843 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.843 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.843 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.843 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.843 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_base": { 2024-04-25T11:37:40.844 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_base", 2024-04-25T11:37:40.844 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.844 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.844 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.844 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T11:37:40.844 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.844 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.844 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.844 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Container image name, without the tag", 2024-04-25T11:37:40.844 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.844 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.844 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.844 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.844 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_elasticsearch": { 2024-04-25T11:37:40.844 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_elasticsearch", 2024-04-25T11:37:40.845 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.845 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.845 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.845 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T11:37:40.845 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.845 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.845 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.845 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "elasticsearch container image", 2024-04-25T11:37:40.845 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.845 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.845 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.845 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.845 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_grafana": { 2024-04-25T11:37:40.845 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_grafana", 2024-04-25T11:37:40.846 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.846 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.846 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.846 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T11:37:40.846 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.846 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.846 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.846 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-04-25T11:37:40.846 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.846 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.846 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.846 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.846 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_haproxy": { 2024-04-25T11:37:40.847 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_haproxy", 2024-04-25T11:37:40.847 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.847 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.847 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.847 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T11:37:40.847 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.847 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.847 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.847 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "HAproxy container image", 2024-04-25T11:37:40.847 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.847 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.847 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.847 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.847 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_agent": { 2024-04-25T11:37:40.848 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_agent", 2024-04-25T11:37:40.848 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.848 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.848 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.848 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T11:37:40.848 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.848 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.848 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.848 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger agent container image", 2024-04-25T11:37:40.848 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.848 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.848 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.848 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.848 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_collector": { 2024-04-25T11:37:40.848 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_collector", 2024-04-25T11:37:40.849 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.849 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.849 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.849 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T11:37:40.849 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.849 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.849 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.849 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger collector container image", 2024-04-25T11:37:40.849 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.849 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.849 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.849 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.849 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_query": { 2024-04-25T11:37:40.850 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_query", 2024-04-25T11:37:40.850 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.850 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.850 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.850 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T11:37:40.850 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.850 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.850 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.850 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger query container image", 2024-04-25T11:37:40.850 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.850 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.850 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.850 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.851 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_keepalived": { 2024-04-25T11:37:40.851 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_keepalived", 2024-04-25T11:37:40.851 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.851 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.851 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.851 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T11:37:40.851 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.851 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.851 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.851 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Keepalived container image", 2024-04-25T11:37:40.851 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.851 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.851 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.851 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.852 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_loki": { 2024-04-25T11:37:40.852 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_loki", 2024-04-25T11:37:40.852 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.852 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.852 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.852 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T11:37:40.852 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.852 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.852 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.852 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Loki container image", 2024-04-25T11:37:40.852 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.852 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.852 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.853 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.853 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_node_exporter": { 2024-04-25T11:37:40.853 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_node_exporter", 2024-04-25T11:37:40.853 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.853 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.853 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.853 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T11:37:40.853 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.853 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.853 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.853 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-04-25T11:37:40.853 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.853 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.853 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.853 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.853 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_nvmeof": { 2024-04-25T11:37:40.854 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_nvmeof", 2024-04-25T11:37:40.854 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.854 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.854 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.854 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T11:37:40.854 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.854 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.854 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.854 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Nvme-of container image", 2024-04-25T11:37:40.854 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.854 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.854 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.854 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.854 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_prometheus": { 2024-04-25T11:37:40.854 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_prometheus", 2024-04-25T11:37:40.855 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.855 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.855 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.855 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T11:37:40.855 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.855 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.855 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.855 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-04-25T11:37:40.855 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.855 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.855 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.855 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.855 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_promtail": { 2024-04-25T11:37:40.855 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_promtail", 2024-04-25T11:37:40.856 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.856 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.856 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.856 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T11:37:40.856 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.856 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.856 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.856 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Promtail container image", 2024-04-25T11:37:40.856 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.856 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.856 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.856 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.856 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_samba": { 2024-04-25T11:37:40.856 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_samba", 2024-04-25T11:37:40.856 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.857 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.857 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.857 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T11:37:40.857 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.857 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.857 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.857 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Samba/SMB container image", 2024-04-25T11:37:40.857 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.857 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.857 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.857 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.857 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_snmp_gateway": { 2024-04-25T11:37:40.857 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_snmp_gateway", 2024-04-25T11:37:40.857 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.858 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.858 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.858 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T11:37:40.858 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.858 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.858 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.858 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SNMP Gateway container image", 2024-04-25T11:37:40.858 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.858 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.858 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.858 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.858 INFO:teuthology.orchestra.run.smithi069.stdout: "container_init": { 2024-04-25T11:37:40.858 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_init", 2024-04-25T11:37:40.859 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.859 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.859 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.859 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:40.859 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.859 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.859 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.859 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T11:37:40.859 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.859 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.859 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.859 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.859 INFO:teuthology.orchestra.run.smithi069.stdout: "daemon_cache_timeout": { 2024-04-25T11:37:40.860 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "daemon_cache_timeout", 2024-04-25T11:37:40.860 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:40.860 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.860 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.860 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-04-25T11:37:40.860 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.860 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.860 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.860 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T11:37:40.860 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.860 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.860 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.860 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.860 INFO:teuthology.orchestra.run.smithi069.stdout: "default_cephadm_command_timeout": { 2024-04-25T11:37:40.861 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T11:37:40.861 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:40.861 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.861 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.861 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "900", 2024-04-25T11:37:40.861 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.861 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.861 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.861 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T11:37:40.861 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.861 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.861 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.861 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.861 INFO:teuthology.orchestra.run.smithi069.stdout: "default_registry": { 2024-04-25T11:37:40.862 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "default_registry", 2024-04-25T11:37:40.862 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.862 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.862 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.862 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io", 2024-04-25T11:37:40.862 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.862 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.862 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.862 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T11:37:40.862 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.862 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.862 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.862 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.863 INFO:teuthology.orchestra.run.smithi069.stdout: "device_cache_timeout": { 2024-04-25T11:37:40.863 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_cache_timeout", 2024-04-25T11:37:40.863 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:40.863 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.863 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.863 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1800", 2024-04-25T11:37:40.863 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.863 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.863 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.863 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache device inventory", 2024-04-25T11:37:40.863 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.863 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.863 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.864 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.864 INFO:teuthology.orchestra.run.smithi069.stdout: "device_enhanced_scan": { 2024-04-25T11:37:40.864 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_enhanced_scan", 2024-04-25T11:37:40.864 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.864 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.864 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.864 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:40.864 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.864 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.864 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.864 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T11:37:40.864 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.864 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.864 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.865 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.865 INFO:teuthology.orchestra.run.smithi069.stdout: "facts_cache_timeout": { 2024-04-25T11:37:40.865 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "facts_cache_timeout", 2024-04-25T11:37:40.865 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:40.865 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.865 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.865 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-04-25T11:37:40.865 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.865 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.865 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.865 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache host facts data", 2024-04-25T11:37:40.865 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.865 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.865 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.865 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.866 INFO:teuthology.orchestra.run.smithi069.stdout: "grafana_dashboards_path": { 2024-04-25T11:37:40.866 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "grafana_dashboards_path", 2024-04-25T11:37:40.866 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.866 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.866 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.866 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T11:37:40.866 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.866 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.866 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.866 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T11:37:40.866 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.866 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.866 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.866 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.866 INFO:teuthology.orchestra.run.smithi069.stdout: "host_check_interval": { 2024-04-25T11:37:40.867 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "host_check_interval", 2024-04-25T11:37:40.867 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:40.867 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.867 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.867 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-04-25T11:37:40.867 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.867 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.867 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.867 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to perform a host check", 2024-04-25T11:37:40.867 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.867 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.867 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.867 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.867 INFO:teuthology.orchestra.run.smithi069.stdout: "hw_monitoring": { 2024-04-25T11:37:40.868 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "hw_monitoring", 2024-04-25T11:37:40.868 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.868 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.868 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.868 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:40.868 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.868 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.868 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.868 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T11:37:40.868 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.868 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.868 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.868 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.868 INFO:teuthology.orchestra.run.smithi069.stdout: "inventory_list_all": { 2024-04-25T11:37:40.868 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "inventory_list_all", 2024-04-25T11:37:40.869 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.869 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.869 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.869 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:40.869 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.869 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.869 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.869 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T11:37:40.869 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.869 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.869 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.869 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.870 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:40.870 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:40.870 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.870 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.870 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.870 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:40.870 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.870 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.870 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:40.870 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:40.870 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:40.870 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:40.870 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:40.870 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:40.871 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:40.871 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:40.871 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.871 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.871 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.871 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.871 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.871 INFO:teuthology.orchestra.run.smithi069.stdout: "log_refresh_metadata": { 2024-04-25T11:37:40.871 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_refresh_metadata", 2024-04-25T11:37:40.871 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.871 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.871 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.871 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:40.871 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.872 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.872 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.872 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T11:37:40.872 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.872 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.872 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.872 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.872 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:40.872 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:40.872 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.872 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.872 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.872 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:40.873 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.873 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.873 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.873 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T11:37:40.873 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.873 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.873 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.873 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.873 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:40.873 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:40.873 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.873 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.873 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.873 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:40.874 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.874 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.874 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:40.874 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:40.874 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:40.874 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:40.874 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:40.874 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:40.874 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:40.874 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:40.874 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.874 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.874 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.875 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.875 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.875 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:40.875 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:40.875 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.875 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.875 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.875 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:40.875 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.875 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.875 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.875 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.875 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.875 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.876 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.876 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.876 INFO:teuthology.orchestra.run.smithi069.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T11:37:40.876 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T11:37:40.876 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.876 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.876 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.876 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:40.876 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.876 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.876 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.876 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T11:37:40.876 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.876 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.877 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.877 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.877 INFO:teuthology.orchestra.run.smithi069.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T11:37:40.877 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T11:37:40.877 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.877 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.877 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.877 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "*", 2024-04-25T11:37:40.877 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.877 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.877 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.877 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T11:37:40.877 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.877 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.877 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.878 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.878 INFO:teuthology.orchestra.run.smithi069.stdout: "max_count_per_host": { 2024-04-25T11:37:40.878 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_count_per_host", 2024-04-25T11:37:40.878 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:40.878 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.878 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.878 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-25T11:37:40.878 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.878 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.878 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.878 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "max number of daemons per service per host", 2024-04-25T11:37:40.878 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.878 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.879 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.879 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.879 INFO:teuthology.orchestra.run.smithi069.stdout: "max_osd_draining_count": { 2024-04-25T11:37:40.879 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_osd_draining_count", 2024-04-25T11:37:40.879 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:40.879 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.879 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.879 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-25T11:37:40.879 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.879 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.879 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.879 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T11:37:40.879 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.879 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.880 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.880 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.880 INFO:teuthology.orchestra.run.smithi069.stdout: "migration_current": { 2024-04-25T11:37:40.880 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "migration_current", 2024-04-25T11:37:40.880 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:40.880 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.880 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.880 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:40.880 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.880 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.880 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.880 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "internal - do not modify", 2024-04-25T11:37:40.880 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.880 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.881 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.881 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.881 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": { 2024-04-25T11:37:40.881 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mode", 2024-04-25T11:37:40.881 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.881 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.881 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.881 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "root", 2024-04-25T11:37:40.881 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.881 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.881 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:40.881 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm-package", 2024-04-25T11:37:40.881 INFO:teuthology.orchestra.run.smithi069.stdout: "root" 2024-04-25T11:37:40.881 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:40.882 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T11:37:40.882 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.882 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.882 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.882 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.882 INFO:teuthology.orchestra.run.smithi069.stdout: "oob_default_addr": { 2024-04-25T11:37:40.882 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "oob_default_addr", 2024-04-25T11:37:40.882 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.882 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.882 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.882 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "169.254.1.1", 2024-04-25T11:37:40.882 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.882 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.883 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.883 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T11:37:40.883 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.883 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.883 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.883 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.883 INFO:teuthology.orchestra.run.smithi069.stdout: "prometheus_alerts_path": { 2024-04-25T11:37:40.883 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prometheus_alerts_path", 2024-04-25T11:37:40.883 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.883 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.883 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.883 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T11:37:40.883 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.883 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.883 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.884 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T11:37:40.884 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.884 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.884 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.884 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.884 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_insecure": { 2024-04-25T11:37:40.884 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_insecure", 2024-04-25T11:37:40.884 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.884 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.884 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.884 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:40.884 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.884 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.885 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.885 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T11:37:40.885 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.885 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.885 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.885 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.885 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_password": { 2024-04-25T11:37:40.885 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_password", 2024-04-25T11:37:40.885 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.885 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.885 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.885 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:40.885 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.885 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.886 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.886 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T11:37:40.886 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.886 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.886 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.886 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.886 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_url": { 2024-04-25T11:37:40.886 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_url", 2024-04-25T11:37:40.886 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.886 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.886 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.886 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:40.886 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.886 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.887 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.887 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T11:37:40.887 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.887 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.887 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.887 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.887 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_username": { 2024-04-25T11:37:40.887 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_username", 2024-04-25T11:37:40.887 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.887 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.887 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.887 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:40.887 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.888 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.888 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.888 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T11:37:40.888 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.888 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.888 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.888 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.888 INFO:teuthology.orchestra.run.smithi069.stdout: "secure_monitoring_stack": { 2024-04-25T11:37:40.888 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "secure_monitoring_stack", 2024-04-25T11:37:40.888 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.888 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.888 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.888 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:40.888 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.888 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.889 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.889 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T11:37:40.889 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.889 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.889 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.889 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.889 INFO:teuthology.orchestra.run.smithi069.stdout: "service_discovery_port": { 2024-04-25T11:37:40.889 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "service_discovery_port", 2024-04-25T11:37:40.889 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:40.889 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.889 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.889 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8765", 2024-04-25T11:37:40.889 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.889 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.890 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.890 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "cephadm service discovery port", 2024-04-25T11:37:40.890 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.890 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.890 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.890 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.890 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:40.890 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:40.890 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:40.890 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:40.890 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.890 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:40.890 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.890 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.891 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.891 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.891 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.891 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.891 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.891 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.891 INFO:teuthology.orchestra.run.smithi069.stdout: "ssh_config_file": { 2024-04-25T11:37:40.891 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssh_config_file", 2024-04-25T11:37:40.891 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.891 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.891 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.891 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:40.891 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.891 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.892 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.892 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T11:37:40.892 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.892 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.892 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.892 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.892 INFO:teuthology.orchestra.run.smithi069.stdout: "use_agent": { 2024-04-25T11:37:40.892 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "use_agent", 2024-04-25T11:37:40.892 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.892 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.892 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.892 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:40.892 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.892 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.892 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.892 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T11:37:40.893 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.893 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.893 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.893 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.893 INFO:teuthology.orchestra.run.smithi069.stdout: "use_repo_digest": { 2024-04-25T11:37:40.893 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "use_repo_digest", 2024-04-25T11:37:40.893 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.893 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.893 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.893 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:40.893 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.893 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.893 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.893 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T11:37:40.894 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.894 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.894 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.894 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.894 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_failed_host_check": { 2024-04-25T11:37:40.894 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_failed_host_check", 2024-04-25T11:37:40.894 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.894 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.894 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.894 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:40.894 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.894 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.895 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.895 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T11:37:40.895 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.895 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.895 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.895 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.895 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_stray_daemons": { 2024-04-25T11:37:40.895 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_stray_daemons", 2024-04-25T11:37:40.895 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.895 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.895 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.895 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:40.895 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.895 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.896 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.896 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T11:37:40.896 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.896 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.896 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.896 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.896 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_stray_hosts": { 2024-04-25T11:37:40.896 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_stray_hosts", 2024-04-25T11:37:40.896 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.896 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.896 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.896 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:40.896 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.896 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.897 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.897 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T11:37:40.897 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.897 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.897 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.897 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:40.897 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:40.897 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.897 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:40.897 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crash", 2024-04-25T11:37:40.897 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:40.897 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:40.897 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:40.897 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:40.897 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:40.898 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.898 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.898 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.898 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:40.898 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.898 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.898 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:40.898 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:40.898 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:40.898 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:40.898 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:40.898 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:40.898 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:40.898 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:40.899 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.899 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.899 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.899 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.899 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.899 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:40.899 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:40.899 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.899 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.899 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.899 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:40.899 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.899 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.899 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.900 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.900 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.900 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.900 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.900 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.900 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:40.900 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:40.900 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.900 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.900 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.900 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:40.900 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.900 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.900 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:40.901 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:40.901 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:40.901 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:40.901 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:40.901 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:40.901 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:40.901 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:40.901 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.901 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.901 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.901 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.901 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.901 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:40.901 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:40.902 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.902 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.902 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.902 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:40.902 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.902 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.902 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.902 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.902 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.902 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.902 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.902 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.902 INFO:teuthology.orchestra.run.smithi069.stdout: "retain_interval": { 2024-04-25T11:37:40.902 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "retain_interval", 2024-04-25T11:37:40.902 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:40.903 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.903 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.903 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "31536000", 2024-04-25T11:37:40.903 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.903 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.903 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.903 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T11:37:40.903 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.903 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.903 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.903 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.903 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:40.903 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:40.903 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:40.903 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:40.904 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.904 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:40.904 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.904 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.904 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.904 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.904 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.904 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.904 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.904 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.904 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_recent_interval": { 2024-04-25T11:37:40.904 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_recent_interval", 2024-04-25T11:37:40.904 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:40.904 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.905 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.905 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1209600", 2024-04-25T11:37:40.905 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.905 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.905 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.905 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T11:37:40.905 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.905 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.905 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.905 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:40.905 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:40.905 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.905 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:40.905 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "dashboard", 2024-04-25T11:37:40.905 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:40.906 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:40.906 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:40.906 INFO:teuthology.orchestra.run.smithi069.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T11:37:40.906 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T11:37:40.906 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:40.906 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.906 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.906 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-25T11:37:40.906 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.906 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.906 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.906 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.906 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.906 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.906 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.907 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.907 INFO:teuthology.orchestra.run.smithi069.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T11:37:40.907 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T11:37:40.907 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.907 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.907 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.907 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:40.907 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.907 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.907 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.907 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.907 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.907 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.907 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.908 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.908 INFO:teuthology.orchestra.run.smithi069.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T11:37:40.908 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T11:37:40.908 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.908 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.908 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.908 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:40.908 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.908 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.908 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.908 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.908 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.908 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.908 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.909 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.909 INFO:teuthology.orchestra.run.smithi069.stdout: "AUDIT_API_ENABLED": { 2024-04-25T11:37:40.909 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T11:37:40.909 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.909 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.909 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.909 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:40.909 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.909 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.909 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.909 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.909 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.909 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.910 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.910 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.910 INFO:teuthology.orchestra.run.smithi069.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T11:37:40.910 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T11:37:40.910 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.910 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.910 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.910 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:40.910 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.910 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.910 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.910 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.910 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.910 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.911 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.911 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.911 INFO:teuthology.orchestra.run.smithi069.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T11:37:40.911 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T11:37:40.911 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.911 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.911 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.911 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:40.911 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.911 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.911 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.911 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.911 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.911 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.912 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.912 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.912 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T11:37:40.912 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T11:37:40.912 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.912 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.912 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.912 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:40.912 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.912 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.912 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.912 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.912 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.912 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.913 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.913 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.913 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T11:37:40.913 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T11:37:40.913 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.913 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.913 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.913 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:40.913 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.913 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.913 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.913 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.913 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.913 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.913 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.914 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.914 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T11:37:40.914 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T11:37:40.914 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.914 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.914 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.914 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:40.914 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.914 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.914 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.914 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.914 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.914 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.914 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.915 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.915 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T11:37:40.915 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T11:37:40.915 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.915 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.915 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.915 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:40.915 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.915 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.915 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.915 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.915 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.915 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.915 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.915 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.916 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T11:37:40.916 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T11:37:40.916 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.916 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.916 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.916 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:40.916 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.916 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.916 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.916 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.916 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.916 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.916 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.916 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.916 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T11:37:40.917 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T11:37:40.917 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.917 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.917 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.917 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:40.917 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.917 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.917 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.917 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.917 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.917 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.917 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.917 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.917 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T11:37:40.917 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T11:37:40.918 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.918 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.918 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.918 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:40.918 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.918 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.918 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.918 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.918 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.918 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.918 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.918 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.918 INFO:teuthology.orchestra.run.smithi069.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T11:37:40.918 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T11:37:40.918 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.919 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.919 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.919 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:40.919 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.919 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.919 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.919 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.919 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.919 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.919 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.919 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.919 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T11:37:40.919 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T11:37:40.919 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.920 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.920 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.920 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2024-04-25T11:37:40.920 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.920 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.920 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.920 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.920 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.920 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.920 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.920 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.920 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T11:37:40.920 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T11:37:40.920 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.920 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.921 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.921 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:40.921 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.921 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.921 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.921 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.921 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.921 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.921 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.921 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.921 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_URL": { 2024-04-25T11:37:40.921 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_URL", 2024-04-25T11:37:40.921 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.921 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.922 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.922 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:40.922 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.922 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.922 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.922 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.922 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.922 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.922 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.922 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.922 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T11:37:40.922 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T11:37:40.922 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.922 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.923 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.923 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2024-04-25T11:37:40.923 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.923 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.923 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.923 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.923 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.923 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.923 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.923 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.923 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T11:37:40.923 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T11:37:40.923 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.923 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.923 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.924 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:40.924 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.924 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.924 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.924 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.924 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.924 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.924 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.924 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.924 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T11:37:40.924 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T11:37:40.924 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.924 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.924 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.924 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:40.924 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.925 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.925 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.925 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.925 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.925 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.925 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.925 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.925 INFO:teuthology.orchestra.run.smithi069.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T11:37:40.925 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T11:37:40.925 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.925 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.925 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.925 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:40.925 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.925 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.926 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.926 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.926 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.926 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.926 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.926 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.926 INFO:teuthology.orchestra.run.smithi069.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T11:37:40.926 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T11:37:40.926 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.926 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.926 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.926 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:40.926 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.926 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.927 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.927 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.927 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.927 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.927 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.927 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.927 INFO:teuthology.orchestra.run.smithi069.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T11:37:40.927 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T11:37:40.927 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.927 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.927 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.927 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "[]", 2024-04-25T11:37:40.927 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.928 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.928 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.928 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.928 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.928 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.928 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.928 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.928 INFO:teuthology.orchestra.run.smithi069.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T11:37:40.928 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T11:37:40.928 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.928 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.928 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.928 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "{}", 2024-04-25T11:37:40.928 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.929 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.929 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.929 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.929 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.929 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.929 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.929 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.929 INFO:teuthology.orchestra.run.smithi069.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T11:37:40.929 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T11:37:40.929 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.929 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.929 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.929 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:40.929 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.930 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.930 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.930 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.930 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.930 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.930 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.930 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.930 INFO:teuthology.orchestra.run.smithi069.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T11:37:40.930 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T11:37:40.930 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.930 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.930 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.930 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:40.930 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.931 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.931 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.931 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.931 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.931 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.931 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.931 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.931 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T11:37:40.931 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T11:37:40.931 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.931 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.931 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.931 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:40.931 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.932 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.932 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.932 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.932 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.932 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.932 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.932 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.932 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T11:37:40.932 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T11:37:40.932 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.932 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.932 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.932 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:40.932 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.932 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.933 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.933 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.933 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.933 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.933 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.933 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.933 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T11:37:40.933 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T11:37:40.933 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.933 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.933 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.933 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:40.933 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.933 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.934 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.934 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.934 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.934 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.934 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.934 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.934 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T11:37:40.934 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T11:37:40.934 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.934 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.934 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.934 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:40.934 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.934 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.934 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.935 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.935 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.935 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.935 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.935 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.935 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T11:37:40.935 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T11:37:40.935 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.935 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.935 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.935 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:40.935 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.935 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.935 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.936 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.936 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.936 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.936 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.936 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.936 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T11:37:40.936 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T11:37:40.936 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.936 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.936 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.936 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:40.936 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.936 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.936 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.936 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.937 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.937 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.937 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.937 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.937 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T11:37:40.937 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T11:37:40.937 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.937 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.937 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.937 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:40.937 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.937 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.937 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.937 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.937 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.938 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.938 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.938 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.938 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T11:37:40.938 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T11:37:40.938 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.938 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.938 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.938 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:40.938 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.938 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.938 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.938 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.938 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.939 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.939 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.939 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.939 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T11:37:40.939 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T11:37:40.939 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.939 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.939 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.939 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T11:37:40.939 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.939 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.939 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.939 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.940 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.940 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.940 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.940 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.940 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T11:37:40.940 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T11:37:40.940 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:40.940 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.940 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.940 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-25T11:37:40.940 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.940 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.940 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.940 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.941 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.941 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.941 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.941 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.941 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T11:37:40.941 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T11:37:40.941 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:40.941 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.941 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.941 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8", 2024-04-25T11:37:40.941 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.941 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.941 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.941 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.941 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.942 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.942 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.942 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.942 INFO:teuthology.orchestra.run.smithi069.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T11:37:40.942 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T11:37:40.942 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:40.942 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.942 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.942 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "45", 2024-04-25T11:37:40.942 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.942 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.942 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.942 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.942 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.943 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.943 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.943 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.943 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T11:37:40.943 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T11:37:40.943 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.943 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.943 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.943 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:40.943 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.943 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.943 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.943 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.943 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.944 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.944 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.944 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.944 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T11:37:40.944 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T11:37:40.944 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.944 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.944 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.944 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2024-04-25T11:37:40.944 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.944 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.944 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.944 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.944 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.944 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.945 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.945 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.945 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T11:37:40.945 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T11:37:40.945 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.945 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.945 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.945 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:40.945 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.945 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.945 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.945 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.945 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.945 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.945 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.946 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.946 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T11:37:40.946 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T11:37:40.946 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.946 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.946 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.946 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:40.946 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.946 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.946 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.946 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.946 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.947 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.947 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.947 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.947 INFO:teuthology.orchestra.run.smithi069.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T11:37:40.947 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T11:37:40.947 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.947 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.947 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.947 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:40.947 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.947 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.947 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.947 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.947 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.947 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.948 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.948 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.948 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T11:37:40.948 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T11:37:40.948 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:40.948 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.948 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.948 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:40.948 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.948 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.948 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.948 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.948 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.948 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.949 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.949 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.949 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T11:37:40.949 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T11:37:40.949 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:40.949 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.949 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.949 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-25T11:37:40.949 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.949 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.949 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.949 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.949 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.949 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.950 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.950 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.950 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T11:37:40.950 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T11:37:40.950 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:40.950 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.950 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.950 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-04-25T11:37:40.950 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.950 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.950 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.950 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.950 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.951 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.951 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.951 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.951 INFO:teuthology.orchestra.run.smithi069.stdout: "cross_origin_url": { 2024-04-25T11:37:40.951 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cross_origin_url", 2024-04-25T11:37:40.951 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.951 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.951 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.951 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:40.951 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.951 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.951 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.951 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.951 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.952 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.952 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.952 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.952 INFO:teuthology.orchestra.run.smithi069.stdout: "crt_file": { 2024-04-25T11:37:40.952 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crt_file", 2024-04-25T11:37:40.952 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.952 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.952 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.952 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:40.952 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.952 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.952 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.952 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.952 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.953 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.953 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.953 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.953 INFO:teuthology.orchestra.run.smithi069.stdout: "debug": { 2024-04-25T11:37:40.953 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "debug", 2024-04-25T11:37:40.953 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.953 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.953 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.953 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:40.953 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.953 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.953 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.953 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable/disable debug options", 2024-04-25T11:37:40.953 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.954 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.954 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.954 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.954 INFO:teuthology.orchestra.run.smithi069.stdout: "jwt_token_ttl": { 2024-04-25T11:37:40.954 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "jwt_token_ttl", 2024-04-25T11:37:40.954 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:40.954 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.954 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.954 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "28800", 2024-04-25T11:37:40.954 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.954 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.954 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.954 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.955 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.955 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.955 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.955 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.955 INFO:teuthology.orchestra.run.smithi069.stdout: "key_file": { 2024-04-25T11:37:40.955 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "key_file", 2024-04-25T11:37:40.955 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.955 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.955 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.955 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:40.955 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.955 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.955 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.955 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.955 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.956 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.956 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.956 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.956 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:40.956 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:40.956 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.956 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.956 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.956 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:40.956 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.956 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.956 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:40.956 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:40.956 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:40.957 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:40.957 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:40.957 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:40.957 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:40.957 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:40.957 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.957 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.957 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.957 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.957 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.957 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:40.957 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:40.957 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.957 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.957 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.958 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:40.958 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.958 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.958 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.958 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.958 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.958 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.958 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.958 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.958 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:40.958 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:40.958 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.958 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.958 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.958 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:40.959 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.959 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.959 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:40.959 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:40.959 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:40.959 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:40.959 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:40.959 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:40.959 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:40.959 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:40.959 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.959 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.959 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.960 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.960 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.960 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:40.960 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:40.960 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.960 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.960 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.960 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:40.960 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.960 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.960 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.960 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.960 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.961 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.961 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.961 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.961 INFO:teuthology.orchestra.run.smithi069.stdout: "motd": { 2024-04-25T11:37:40.961 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "motd", 2024-04-25T11:37:40.961 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.961 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.961 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.961 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:40.961 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.961 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.961 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.961 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "The message of the day", 2024-04-25T11:37:40.962 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.962 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.962 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.962 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.962 INFO:teuthology.orchestra.run.smithi069.stdout: "redirect_resolve_ip_addr": { 2024-04-25T11:37:40.962 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T11:37:40.962 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.962 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.962 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.962 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:40.962 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.962 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.962 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.962 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.963 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.963 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.963 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.963 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.963 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2024-04-25T11:37:40.963 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2024-04-25T11:37:40.963 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.963 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.963 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.963 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "::", 2024-04-25T11:37:40.963 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.963 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.963 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.963 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.964 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.964 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.964 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.964 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.964 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2024-04-25T11:37:40.964 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2024-04-25T11:37:40.964 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:40.964 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.964 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.964 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8080", 2024-04-25T11:37:40.964 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.964 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.964 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.964 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.964 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.965 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.965 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.965 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.965 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:40.965 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:40.965 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:40.965 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:40.965 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.965 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:40.965 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.965 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.965 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.965 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.965 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.965 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.965 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.965 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.966 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl": { 2024-04-25T11:37:40.966 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl", 2024-04-25T11:37:40.966 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.966 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.966 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.966 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:40.966 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.966 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.966 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.966 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.966 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.966 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.966 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.967 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.967 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl_server_port": { 2024-04-25T11:37:40.967 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl_server_port", 2024-04-25T11:37:40.967 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:40.967 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.967 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.967 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8443", 2024-04-25T11:37:40.967 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.967 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.967 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.967 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.967 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.967 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.968 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.968 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.968 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_behaviour": { 2024-04-25T11:37:40.968 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_behaviour", 2024-04-25T11:37:40.968 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.968 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.968 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.968 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "redirect", 2024-04-25T11:37:40.968 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.968 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.968 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:40.968 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:40.968 INFO:teuthology.orchestra.run.smithi069.stdout: "redirect" 2024-04-25T11:37:40.968 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:40.969 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.969 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.969 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.969 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.969 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.969 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_error_status_code": { 2024-04-25T11:37:40.969 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_error_status_code", 2024-04-25T11:37:40.969 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:40.969 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.969 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.969 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "500", 2024-04-25T11:37:40.969 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "400", 2024-04-25T11:37:40.969 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "599", 2024-04-25T11:37:40.969 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.970 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.970 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.970 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.970 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.970 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.970 INFO:teuthology.orchestra.run.smithi069.stdout: "url_prefix": { 2024-04-25T11:37:40.970 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "url_prefix", 2024-04-25T11:37:40.970 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.970 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.970 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.970 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:40.970 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.970 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.971 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.971 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.971 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.971 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.971 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.971 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:40.971 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:40.971 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.971 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:40.971 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "devicehealth", 2024-04-25T11:37:40.971 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:40.971 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:40.971 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:40.971 INFO:teuthology.orchestra.run.smithi069.stdout: "enable_monitoring": { 2024-04-25T11:37:40.972 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enable_monitoring", 2024-04-25T11:37:40.972 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.972 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.972 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.972 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:40.972 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.972 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.972 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.972 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "monitor device health metrics", 2024-04-25T11:37:40.972 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.972 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.972 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.972 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.973 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:40.973 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:40.973 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.973 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.973 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.973 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:40.973 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.973 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.973 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:40.973 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:40.973 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:40.973 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:40.973 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:40.973 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:40.974 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:40.974 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:40.974 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.974 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.974 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.974 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.974 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.974 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:40.974 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:40.974 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.974 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.974 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.974 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:40.974 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.975 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.975 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.975 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.975 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.975 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.975 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.975 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.975 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:40.975 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:40.975 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.975 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.975 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.975 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:40.975 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.976 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.976 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:40.976 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:40.976 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:40.976 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:40.976 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:40.976 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:40.976 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:40.976 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:40.976 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.976 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.976 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.976 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.976 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.976 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:40.977 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:40.977 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.977 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.977 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.977 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:40.977 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.977 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.977 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.977 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.977 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.977 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.977 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.977 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.977 INFO:teuthology.orchestra.run.smithi069.stdout: "mark_out_threshold": { 2024-04-25T11:37:40.978 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mark_out_threshold", 2024-04-25T11:37:40.978 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:40.978 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.978 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.978 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "2419200", 2024-04-25T11:37:40.978 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.978 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.978 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.978 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T11:37:40.978 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.978 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.978 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.978 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.978 INFO:teuthology.orchestra.run.smithi069.stdout: "pool_name": { 2024-04-25T11:37:40.979 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pool_name", 2024-04-25T11:37:40.979 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.979 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.979 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.979 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "device_health_metrics", 2024-04-25T11:37:40.979 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.979 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.979 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.979 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T11:37:40.979 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.979 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.979 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.979 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.980 INFO:teuthology.orchestra.run.smithi069.stdout: "retention_period": { 2024-04-25T11:37:40.980 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "retention_period", 2024-04-25T11:37:40.980 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:40.980 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.980 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.980 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15552000", 2024-04-25T11:37:40.980 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.980 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.980 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.980 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long to retain device health metrics", 2024-04-25T11:37:40.980 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.980 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.980 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.980 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.981 INFO:teuthology.orchestra.run.smithi069.stdout: "scrape_frequency": { 2024-04-25T11:37:40.981 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "scrape_frequency", 2024-04-25T11:37:40.981 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:40.981 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.981 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.981 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "86400", 2024-04-25T11:37:40.981 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.981 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.981 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.981 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T11:37:40.981 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.981 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.981 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.981 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.981 INFO:teuthology.orchestra.run.smithi069.stdout: "self_heal": { 2024-04-25T11:37:40.981 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "self_heal", 2024-04-25T11:37:40.982 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.982 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.982 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.982 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:40.982 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.982 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.982 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.982 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T11:37:40.982 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.982 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.982 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.982 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.982 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-04-25T11:37:40.983 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-04-25T11:37:40.983 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:40.983 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.983 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.983 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-04-25T11:37:40.983 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.983 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.983 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.983 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T11:37:40.983 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.983 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.983 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.983 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.983 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:40.983 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:40.984 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:40.984 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:40.984 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.984 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:40.984 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.984 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.984 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.984 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.984 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.984 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.984 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.984 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.984 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_threshold": { 2024-04-25T11:37:40.985 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_threshold", 2024-04-25T11:37:40.985 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:40.985 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.985 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.985 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "7257600", 2024-04-25T11:37:40.985 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.985 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.985 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.985 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T11:37:40.985 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.985 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.985 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.985 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:40.985 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:40.986 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.986 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:40.986 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "diskprediction_local", 2024-04-25T11:37:40.986 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:40.986 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:40.986 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:40.986 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:40.986 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:40.986 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.986 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.986 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.986 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:40.986 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.987 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.987 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:40.987 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:40.987 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:40.987 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:40.987 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:40.987 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:40.987 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:40.987 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:40.987 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.987 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.987 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.987 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.987 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.988 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:40.988 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:40.988 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.988 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.988 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.988 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:40.988 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.988 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.988 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.988 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.988 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.988 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.988 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.988 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.988 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:40.989 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:40.989 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.989 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.989 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.989 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:40.989 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.989 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.989 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:40.989 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:40.989 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:40.989 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:40.989 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:40.989 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:40.990 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:40.990 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:40.990 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.990 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.990 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.990 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.990 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.990 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:40.990 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:40.990 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:40.990 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.990 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.990 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:40.990 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.991 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.991 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.991 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.991 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.991 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.991 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.991 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.991 INFO:teuthology.orchestra.run.smithi069.stdout: "predict_interval": { 2024-04-25T11:37:40.991 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "predict_interval", 2024-04-25T11:37:40.991 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.991 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.991 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.991 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "86400", 2024-04-25T11:37:40.991 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.992 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.992 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.992 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.992 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.992 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.992 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.992 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.992 INFO:teuthology.orchestra.run.smithi069.stdout: "predictor_model": { 2024-04-25T11:37:40.992 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "predictor_model", 2024-04-25T11:37:40.992 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.992 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.992 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.992 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "prophetstor", 2024-04-25T11:37:40.992 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.993 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.993 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.993 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.993 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.993 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.993 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.993 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.993 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-04-25T11:37:40.993 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-04-25T11:37:40.993 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.993 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.993 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.993 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-04-25T11:37:40.993 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.993 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.993 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.994 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.994 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.994 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.994 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.994 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.994 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:40.994 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:40.994 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:40.994 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:40.994 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:40.994 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:40.994 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.994 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.995 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.995 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:40.995 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.995 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.995 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.995 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:40.995 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:40.995 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.995 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:40.995 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "influx", 2024-04-25T11:37:40.995 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": false, 2024-04-25T11:37:40.995 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "influxdb python module not found", 2024-04-25T11:37:40.995 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:40.995 INFO:teuthology.orchestra.run.smithi069.stdout: "batch_size": { 2024-04-25T11:37:40.995 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "batch_size", 2024-04-25T11:37:40.996 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:40.996 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.996 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.996 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5000", 2024-04-25T11:37:40.996 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.996 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.996 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.996 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T11:37:40.996 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.996 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.996 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.996 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.996 INFO:teuthology.orchestra.run.smithi069.stdout: "database": { 2024-04-25T11:37:40.996 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "database", 2024-04-25T11:37:40.997 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.997 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.997 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.997 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "ceph", 2024-04-25T11:37:40.997 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.997 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.997 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.997 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T11:37:40.997 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.997 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.997 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.997 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.997 INFO:teuthology.orchestra.run.smithi069.stdout: "hostname": { 2024-04-25T11:37:40.997 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "hostname", 2024-04-25T11:37:40.997 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:40.998 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.998 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.998 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:40.998 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:40.998 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.998 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.998 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB server hostname", 2024-04-25T11:37:40.998 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.998 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.998 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.998 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.998 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-04-25T11:37:40.999 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-04-25T11:37:40.999 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:40.999 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:40.999 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:40.999 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "30", 2024-04-25T11:37:40.999 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "5", 2024-04-25T11:37:40.999 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:40.999 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:40.999 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T11:37:40.999 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:40.999 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:40.999 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:40.999 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:40.999 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:40.999 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.000 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.000 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.000 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.000 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.000 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.000 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.000 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.000 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.000 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.000 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.000 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.000 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.000 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.001 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.001 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.001 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.001 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.001 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.001 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.001 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.001 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.001 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.001 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.001 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.001 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.001 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.001 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.002 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.002 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.002 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.002 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.002 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.002 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.002 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.002 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.002 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.002 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.002 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.002 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.002 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.003 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.003 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.003 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.003 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.003 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.003 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.003 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.003 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.003 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.003 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.003 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.003 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.003 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.003 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.003 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.004 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.004 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.004 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.004 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.004 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.004 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.004 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.004 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.004 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.004 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.004 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.004 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.004 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.004 INFO:teuthology.orchestra.run.smithi069.stdout: "password": { 2024-04-25T11:37:41.005 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "password", 2024-04-25T11:37:41.005 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.005 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.005 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.005 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.005 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.005 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.005 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.005 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "password of InfluxDB server user", 2024-04-25T11:37:41.005 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.005 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.005 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.005 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.006 INFO:teuthology.orchestra.run.smithi069.stdout: "port": { 2024-04-25T11:37:41.006 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "port", 2024-04-25T11:37:41.006 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.006 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.006 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.006 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8086", 2024-04-25T11:37:41.006 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.006 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.006 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.006 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB server port", 2024-04-25T11:37:41.006 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.006 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.006 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.007 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.007 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.007 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.007 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.007 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.007 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.007 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.007 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.007 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.007 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.007 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.007 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.007 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.008 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.008 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.008 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl": { 2024-04-25T11:37:41.008 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl", 2024-04-25T11:37:41.008 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.008 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.008 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.008 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "false", 2024-04-25T11:37:41.008 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.008 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.008 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.008 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T11:37:41.008 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.009 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.009 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.009 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.009 INFO:teuthology.orchestra.run.smithi069.stdout: "threads": { 2024-04-25T11:37:41.009 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "threads", 2024-04-25T11:37:41.009 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.009 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.009 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.009 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-04-25T11:37:41.009 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-04-25T11:37:41.009 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "32", 2024-04-25T11:37:41.009 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.009 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T11:37:41.009 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.010 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.010 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.010 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.010 INFO:teuthology.orchestra.run.smithi069.stdout: "username": { 2024-04-25T11:37:41.010 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "username", 2024-04-25T11:37:41.010 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.010 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.010 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.010 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.010 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.010 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.010 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.010 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "username of InfluxDB server user", 2024-04-25T11:37:41.010 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.011 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.011 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.011 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.011 INFO:teuthology.orchestra.run.smithi069.stdout: "verify_ssl": { 2024-04-25T11:37:41.011 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "verify_ssl", 2024-04-25T11:37:41.011 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.011 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.011 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.011 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "true", 2024-04-25T11:37:41.011 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.011 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.011 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.011 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T11:37:41.011 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.012 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.012 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.012 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.012 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.012 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.012 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.012 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "insights", 2024-04-25T11:37:41.012 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.012 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.012 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.012 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.012 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.012 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.012 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.013 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.013 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.013 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.013 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.013 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.013 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.013 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.013 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.013 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.013 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.013 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.013 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.013 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.014 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.014 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.014 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.014 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.014 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.014 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.014 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.014 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.014 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.014 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.014 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.014 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.014 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.015 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.015 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.015 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.015 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.015 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.015 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.015 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.015 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.015 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.015 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.015 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.015 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.015 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.015 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.016 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.016 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.016 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.016 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.016 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.016 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.016 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.016 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.016 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.016 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.016 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.016 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.016 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.016 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.017 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.017 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.017 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.017 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.017 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.017 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.017 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.017 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.017 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.017 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.017 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.017 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.017 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.017 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.017 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.018 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.018 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.018 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.018 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.018 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.018 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.018 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.018 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.018 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.018 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.018 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.018 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.018 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.019 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.019 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "iostat", 2024-04-25T11:37:41.019 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.019 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.019 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.019 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.019 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.019 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.019 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.019 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.019 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.019 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.019 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.019 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.020 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.020 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.020 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.020 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.020 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.020 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.020 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.020 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.020 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.020 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.020 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.020 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.020 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.020 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.021 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.021 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.021 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.021 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.021 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.021 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.021 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.021 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.021 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.021 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.021 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.021 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.021 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.021 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.021 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.021 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.022 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.022 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.022 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.022 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.022 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.022 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.022 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.022 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.022 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.022 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.022 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.022 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.022 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.023 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.023 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.023 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.023 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.023 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.023 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.023 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.023 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.023 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.023 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.023 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.023 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.023 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.024 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.024 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.024 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.024 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.024 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.024 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.024 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.024 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.024 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.024 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.024 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.024 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.024 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.024 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.024 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.025 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.025 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.025 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.025 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.025 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.025 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.025 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.025 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "k8sevents", 2024-04-25T11:37:41.025 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.025 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.025 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.025 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_event_retention_days": { 2024-04-25T11:37:41.025 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ceph_event_retention_days", 2024-04-25T11:37:41.025 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.025 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.026 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.026 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "7", 2024-04-25T11:37:41.026 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.026 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.026 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.026 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T11:37:41.026 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.026 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.026 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.026 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.026 INFO:teuthology.orchestra.run.smithi069.stdout: "config_check_secs": { 2024-04-25T11:37:41.026 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_check_secs", 2024-04-25T11:37:41.026 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.026 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.026 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.027 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-25T11:37:41.027 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "10", 2024-04-25T11:37:41.027 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.027 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.027 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T11:37:41.027 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.027 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.027 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.027 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.027 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.027 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.027 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.027 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.027 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.027 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.028 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.028 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.028 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.028 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.028 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.028 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.028 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.028 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.028 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.028 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.028 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.028 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.028 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.028 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.029 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.029 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.029 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.029 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.029 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.029 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.029 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.029 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.029 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.029 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.029 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.029 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.029 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.030 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.030 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.030 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.030 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.030 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.030 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.030 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.030 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.030 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.030 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.030 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.030 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.030 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.030 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.031 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.031 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.031 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.031 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.031 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.031 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.031 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.031 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.031 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.031 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.031 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.031 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.031 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.031 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.031 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.031 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.032 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.032 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.032 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.032 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.032 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.032 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.032 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.032 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.032 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.032 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.032 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.032 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.032 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.032 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.033 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.033 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.033 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.033 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.033 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.033 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.033 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.033 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.033 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.033 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.033 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "localpool", 2024-04-25T11:37:41.033 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.033 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.033 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.033 INFO:teuthology.orchestra.run.smithi069.stdout: "failure_domain": { 2024-04-25T11:37:41.034 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "failure_domain", 2024-04-25T11:37:41.034 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.034 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.034 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.034 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "host", 2024-04-25T11:37:41.034 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.034 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.034 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.034 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "failure domain for any created local pool", 2024-04-25T11:37:41.034 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T11:37:41.034 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.034 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.034 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.034 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.035 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.035 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.035 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.035 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.035 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.035 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.035 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.035 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.035 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.035 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.035 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.035 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.035 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.035 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.035 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.036 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.036 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.036 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.036 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.036 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.036 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.036 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.036 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.036 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.036 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.036 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.036 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.036 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.036 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.036 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.037 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.037 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.037 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.037 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.037 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.037 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.037 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.037 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.037 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.037 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.037 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.037 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.037 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.037 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.038 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.038 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.038 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.038 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.038 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.038 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.038 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.038 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.038 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.038 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.038 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.038 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.038 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.038 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.039 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.039 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.039 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.039 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.039 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.039 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.039 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.039 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.039 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.039 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.039 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.039 INFO:teuthology.orchestra.run.smithi069.stdout: "min_size": { 2024-04-25T11:37:41.039 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "min_size", 2024-04-25T11:37:41.039 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.040 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.040 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.040 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.040 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.040 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.040 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.040 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default min_size for any created local pool", 2024-04-25T11:37:41.040 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T11:37:41.040 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.040 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.040 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.040 INFO:teuthology.orchestra.run.smithi069.stdout: "num_rep": { 2024-04-25T11:37:41.040 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "num_rep", 2024-04-25T11:37:41.040 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.041 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.041 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.041 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3", 2024-04-25T11:37:41.041 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.041 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.041 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.041 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default replica count for any created local pool", 2024-04-25T11:37:41.041 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.041 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.041 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.041 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.041 INFO:teuthology.orchestra.run.smithi069.stdout: "pg_num": { 2024-04-25T11:37:41.041 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pg_num", 2024-04-25T11:37:41.041 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.042 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.042 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.042 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "128", 2024-04-25T11:37:41.042 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.042 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.042 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.042 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T11:37:41.042 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.042 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.042 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.042 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.042 INFO:teuthology.orchestra.run.smithi069.stdout: "prefix": { 2024-04-25T11:37:41.042 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prefix", 2024-04-25T11:37:41.042 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.043 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.043 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.043 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.043 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.043 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.043 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.043 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "name prefix for any created local pool", 2024-04-25T11:37:41.043 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.043 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.043 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.043 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.043 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.043 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.043 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.044 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.044 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.044 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.044 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.044 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.044 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.044 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.044 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.044 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.044 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.044 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.044 INFO:teuthology.orchestra.run.smithi069.stdout: "subtree": { 2024-04-25T11:37:41.044 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "subtree", 2024-04-25T11:37:41.044 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.044 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.045 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.045 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "rack", 2024-04-25T11:37:41.045 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.045 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.045 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.045 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T11:37:41.045 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T11:37:41.045 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.045 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.045 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.045 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.045 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.045 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.045 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mds_autoscaler", 2024-04-25T11:37:41.045 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.046 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.046 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.046 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.046 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.046 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.046 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.046 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.046 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.046 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.046 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.046 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.046 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.046 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.046 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.047 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.047 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.047 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.047 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.047 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.047 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.047 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.047 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.047 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.047 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.047 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.047 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.047 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.047 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.047 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.048 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.048 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.048 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.048 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.048 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.048 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.048 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.048 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.048 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.048 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.048 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.048 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.048 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.048 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.049 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.049 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.049 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.049 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.049 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.049 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.049 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.049 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.049 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.049 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.049 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.049 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.049 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.049 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.050 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.050 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.050 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.050 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.050 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.050 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.050 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.050 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.050 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.050 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.050 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.050 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.050 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.050 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.050 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.051 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.051 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.051 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.051 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.051 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.051 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.051 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.051 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.051 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.051 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.051 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.051 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.051 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.051 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.052 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.052 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.052 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.052 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mirroring", 2024-04-25T11:37:41.052 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.052 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.052 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.052 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.052 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.052 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.052 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.052 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.052 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.052 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.052 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.052 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.052 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.053 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.053 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.053 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.053 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.053 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.053 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.053 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.053 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.053 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.053 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.053 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.053 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.053 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.053 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.053 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.054 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.054 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.054 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.054 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.054 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.054 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.054 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.054 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.054 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.054 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.054 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.054 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.054 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.055 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.055 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.055 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.055 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.055 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.055 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.055 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.055 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.055 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.055 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.055 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.055 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.055 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.055 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.056 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.056 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.056 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.056 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.056 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.056 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.056 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.056 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.056 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.056 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.056 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.056 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.056 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.056 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.056 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.056 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.057 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.057 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.057 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.057 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.057 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.057 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.057 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.057 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.057 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.057 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.057 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.057 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.057 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.058 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.058 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.058 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.058 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.058 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.058 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.058 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "nfs", 2024-04-25T11:37:41.058 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.058 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.058 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.058 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.058 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.058 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.059 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.059 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.059 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.059 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.059 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.059 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.059 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.059 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.059 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.059 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.059 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.059 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.059 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.059 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.059 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.060 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.060 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.060 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.060 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.060 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.060 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.060 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.060 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.060 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.060 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.060 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.060 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.061 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.061 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.061 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.061 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.061 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.061 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.061 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.061 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.061 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.061 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.061 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.061 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.061 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.061 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.061 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.062 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.062 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.062 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.062 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.062 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.062 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.062 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.062 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.062 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.062 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.062 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.062 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.062 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.063 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.063 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.063 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.063 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.063 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.063 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.063 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.063 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.063 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.063 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.063 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.063 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.063 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.063 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.064 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.064 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.064 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.064 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.064 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.064 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.064 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.064 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.064 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.064 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.064 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.064 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.064 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.065 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.065 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.065 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "orchestrator", 2024-04-25T11:37:41.065 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.065 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.065 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.065 INFO:teuthology.orchestra.run.smithi069.stdout: "fail_fs": { 2024-04-25T11:37:41.065 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "fail_fs", 2024-04-25T11:37:41.065 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.065 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.065 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.065 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.065 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.065 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.065 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.066 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T11:37:41.066 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.066 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.066 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.066 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.066 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.066 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.066 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.066 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.066 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.066 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.066 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.066 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.066 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.067 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.067 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.067 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.067 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.067 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.067 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.067 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.067 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.067 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.067 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.067 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.067 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.067 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.067 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.068 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.068 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.068 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.068 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.068 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.068 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.068 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.068 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.068 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.068 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.068 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.068 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.068 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.068 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.069 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.069 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.069 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.069 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.069 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.069 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.069 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.069 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.069 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.069 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.069 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.069 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.070 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.070 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.070 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.070 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.070 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.070 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.070 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.070 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.070 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.070 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.070 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.070 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.070 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.070 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.071 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.071 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.071 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.071 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.071 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.071 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.071 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.071 INFO:teuthology.orchestra.run.smithi069.stdout: "orchestrator": { 2024-04-25T11:37:41.071 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "orchestrator", 2024-04-25T11:37:41.071 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.071 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.071 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.071 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.071 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.072 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.072 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.072 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm", 2024-04-25T11:37:41.072 INFO:teuthology.orchestra.run.smithi069.stdout: "rook", 2024-04-25T11:37:41.072 INFO:teuthology.orchestra.run.smithi069.stdout: "test_orchestrator" 2024-04-25T11:37:41.072 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.072 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Orchestrator backend", 2024-04-25T11:37:41.072 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.072 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.072 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.072 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.072 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.072 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.072 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.072 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.073 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.073 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.073 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.073 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.073 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.073 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.073 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.073 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.073 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.073 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.073 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.073 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.073 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.074 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "osd_perf_query", 2024-04-25T11:37:41.074 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.074 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.074 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.074 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.074 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.074 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.074 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.074 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.074 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.074 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.074 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.074 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.074 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.075 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.075 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.075 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.075 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.075 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.075 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.075 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.075 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.075 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.075 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.075 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.075 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.075 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.075 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.076 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.076 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.076 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.076 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.076 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.076 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.076 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.076 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.076 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.076 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.076 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.076 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.076 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.076 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.077 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.077 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.077 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.077 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.077 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.077 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.077 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.077 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.077 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.077 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.077 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.077 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.077 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.077 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.078 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.078 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.078 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.078 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.078 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.078 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.078 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.078 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.078 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.078 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.078 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.078 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.078 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.079 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.079 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.079 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.079 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.079 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.079 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.079 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.079 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.079 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.079 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.079 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.079 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.079 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.079 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.079 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.080 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.080 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.080 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.080 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.080 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.080 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.080 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.080 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "osd_support", 2024-04-25T11:37:41.080 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.080 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.080 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.080 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.080 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.080 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.080 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.081 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.081 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.081 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.081 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.081 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.081 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.081 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.081 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.081 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.081 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.081 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.081 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.081 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.081 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.081 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.082 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.082 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.082 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.082 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.082 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.082 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.082 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.082 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.082 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.082 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.082 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.082 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.083 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.083 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.083 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.083 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.083 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.083 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.083 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.083 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.083 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.083 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.083 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.083 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.083 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.083 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.083 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.083 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.084 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.084 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.084 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.084 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.084 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.084 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.084 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.084 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.084 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.084 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.084 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.084 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.084 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.085 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.085 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.085 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.085 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.085 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.085 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.085 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.085 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.085 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.085 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.085 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.085 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.085 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.085 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.085 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.085 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.086 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.086 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.086 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.086 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.086 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.086 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.086 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.086 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.086 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.086 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.086 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.086 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pg_autoscaler", 2024-04-25T11:37:41.086 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.087 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.087 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.087 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.087 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.087 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.087 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.087 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.087 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.087 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.087 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.087 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.087 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.087 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.087 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.087 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.088 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.088 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.088 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.088 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.088 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.088 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.088 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.088 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.088 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.088 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.088 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.088 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.088 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.089 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.089 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.089 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.089 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.089 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.089 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.089 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.089 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.089 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.089 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.089 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.089 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.089 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.090 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.090 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.090 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.090 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.090 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.090 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.090 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.090 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.090 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.090 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.090 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.090 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.090 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.090 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.090 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.091 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.091 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.091 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.091 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.091 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.091 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.091 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.091 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.091 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.091 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.091 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.091 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.091 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.091 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.091 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.091 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.092 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-04-25T11:37:41.092 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-04-25T11:37:41.092 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:41.092 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.092 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.092 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-04-25T11:37:41.092 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.092 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.092 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.092 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.092 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.092 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.092 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.092 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.093 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.093 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.093 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.093 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.093 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.093 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.093 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.093 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.093 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.093 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.093 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.093 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.093 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.093 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.093 INFO:teuthology.orchestra.run.smithi069.stdout: "threshold": { 2024-04-25T11:37:41.093 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "threshold", 2024-04-25T11:37:41.094 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-04-25T11:37:41.094 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.094 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.094 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3.0", 2024-04-25T11:37:41.094 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1.0", 2024-04-25T11:37:41.094 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.094 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.094 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "scaling threshold", 2024-04-25T11:37:41.094 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T11:37:41.094 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.094 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.094 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.094 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.094 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.095 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.095 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "progress", 2024-04-25T11:37:41.095 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.095 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.095 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.095 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_pg_recovery_event": { 2024-04-25T11:37:41.095 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_pg_recovery_event", 2024-04-25T11:37:41.095 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.095 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.095 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.095 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.095 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.095 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.095 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.095 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T11:37:41.096 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.096 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.096 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.096 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.096 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": { 2024-04-25T11:37:41.096 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enabled", 2024-04-25T11:37:41.096 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.096 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.096 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.096 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.096 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.096 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.096 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.096 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.096 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.096 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.097 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.097 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.097 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.097 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.097 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.097 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.097 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.097 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.097 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.097 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.097 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.097 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.097 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.097 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.097 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.097 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.098 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.098 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.098 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.098 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.098 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.098 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.098 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.098 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.098 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.098 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.098 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.098 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.098 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.098 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.099 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.099 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.099 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.099 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.099 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.099 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.099 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.099 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.099 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.099 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.099 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.099 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.099 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.099 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.100 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.100 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.100 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.100 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.100 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.100 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.100 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.100 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.100 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.100 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.100 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.100 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.100 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.100 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.100 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.101 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.101 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.101 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.101 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.101 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.101 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.101 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.101 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.101 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.101 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.101 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.101 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.101 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.101 INFO:teuthology.orchestra.run.smithi069.stdout: "max_completed_events": { 2024-04-25T11:37:41.101 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_completed_events", 2024-04-25T11:37:41.102 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.102 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.102 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.102 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "50", 2024-04-25T11:37:41.102 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.102 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.102 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.102 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "number of past completed events to remember", 2024-04-25T11:37:41.102 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.102 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.102 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.102 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.102 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-04-25T11:37:41.102 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-04-25T11:37:41.103 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:41.103 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.103 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.103 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-04-25T11:37:41.103 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.103 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.103 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.103 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long the module is going to sleep", 2024-04-25T11:37:41.103 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.103 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.103 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.103 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.103 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.103 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.103 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.103 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.104 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.104 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.104 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.104 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.104 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.104 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.104 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.104 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.104 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.104 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.104 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.104 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.104 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.104 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prometheus", 2024-04-25T11:37:41.104 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.104 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.105 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.105 INFO:teuthology.orchestra.run.smithi069.stdout: "cache": { 2024-04-25T11:37:41.105 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cache", 2024-04-25T11:37:41.105 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.105 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.105 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.105 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.105 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.105 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.105 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.105 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.105 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.105 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.106 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.106 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.106 INFO:teuthology.orchestra.run.smithi069.stdout: "exclude_perf_counters": { 2024-04-25T11:37:41.106 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "exclude_perf_counters", 2024-04-25T11:37:41.106 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.106 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.106 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.106 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.106 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.106 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.106 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.106 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T11:37:41.106 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T11:37:41.107 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.107 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.107 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.107 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.107 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.107 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.107 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.107 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.107 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.107 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.107 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.107 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.107 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.107 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.108 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.108 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.108 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.108 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.108 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.108 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.108 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.108 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.108 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.108 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.108 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.109 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.109 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.109 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.109 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.109 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.109 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.109 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.109 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.109 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.109 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.109 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.109 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.109 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.110 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.110 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.110 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.110 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.110 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.110 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.110 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.110 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.110 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.110 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.110 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.110 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.110 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.110 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.110 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.111 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.111 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.111 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.111 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.111 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.111 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.111 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.111 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.111 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.111 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.111 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.111 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.111 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.111 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.111 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.112 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.112 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.112 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.112 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.112 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.112 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_stats_pools": { 2024-04-25T11:37:41.112 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_stats_pools", 2024-04-25T11:37:41.112 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.112 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.112 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.112 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.112 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.112 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.112 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.112 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.112 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.113 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.113 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.113 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.113 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T11:37:41.113 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T11:37:41.113 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.113 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.113 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.113 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "300", 2024-04-25T11:37:41.113 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.113 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.113 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.113 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.113 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.113 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.114 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.114 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.114 INFO:teuthology.orchestra.run.smithi069.stdout: "scrape_interval": { 2024-04-25T11:37:41.114 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "scrape_interval", 2024-04-25T11:37:41.114 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-04-25T11:37:41.114 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.114 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.114 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15.0", 2024-04-25T11:37:41.114 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.114 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.114 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.114 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.114 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.114 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.115 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.115 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.115 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2024-04-25T11:37:41.115 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2024-04-25T11:37:41.115 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.115 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.115 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.115 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "::", 2024-04-25T11:37:41.115 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.115 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.115 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.115 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T11:37:41.115 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.115 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.115 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.116 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.116 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2024-04-25T11:37:41.116 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2024-04-25T11:37:41.116 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.116 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.116 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.116 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "9283", 2024-04-25T11:37:41.116 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.116 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.116 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.116 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T11:37:41.116 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.116 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.116 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.116 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.116 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.116 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.117 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.117 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.117 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.117 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.117 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.117 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.117 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.117 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.117 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.117 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.117 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.117 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.117 INFO:teuthology.orchestra.run.smithi069.stdout: "stale_cache_strategy": { 2024-04-25T11:37:41.117 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "stale_cache_strategy", 2024-04-25T11:37:41.117 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.118 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.118 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.118 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "log", 2024-04-25T11:37:41.118 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.118 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.118 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.118 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.118 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.118 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.118 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.118 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.118 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_behaviour": { 2024-04-25T11:37:41.118 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_behaviour", 2024-04-25T11:37:41.118 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.119 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.119 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.119 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "default", 2024-04-25T11:37:41.119 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.119 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.119 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.119 INFO:teuthology.orchestra.run.smithi069.stdout: "default", 2024-04-25T11:37:41.119 INFO:teuthology.orchestra.run.smithi069.stdout: "error" 2024-04-25T11:37:41.119 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.119 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.119 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.119 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.119 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.119 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.119 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_error_status_code": { 2024-04-25T11:37:41.119 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_error_status_code", 2024-04-25T11:37:41.119 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.120 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.120 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.120 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "500", 2024-04-25T11:37:41.120 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "400", 2024-04-25T11:37:41.120 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "599", 2024-04-25T11:37:41.120 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.120 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.120 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.120 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.120 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.120 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.120 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.120 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.120 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.121 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_support", 2024-04-25T11:37:41.121 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.121 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.121 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.121 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.121 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.121 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.121 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.121 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.121 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.121 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.121 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.121 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.121 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.121 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.122 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.122 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.122 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.122 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.122 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.122 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.122 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.122 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.122 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.122 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.122 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.122 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.122 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.122 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.122 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.123 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.123 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.123 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.123 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.123 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.123 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.123 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.123 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.123 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.123 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.123 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.123 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.123 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.123 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.123 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.124 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.124 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.124 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.124 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.124 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.124 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.124 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.124 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.124 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.124 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.124 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.124 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.124 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.124 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.124 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.124 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.125 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.125 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.125 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.125 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.125 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.125 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.125 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.125 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.125 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.125 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.125 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.125 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.125 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.125 INFO:teuthology.orchestra.run.smithi069.stdout: "max_concurrent_snap_create": { 2024-04-25T11:37:41.126 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_concurrent_snap_create", 2024-04-25T11:37:41.126 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.126 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.126 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.126 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-25T11:37:41.126 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.126 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.126 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.126 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.126 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.126 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.126 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.126 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.126 INFO:teuthology.orchestra.run.smithi069.stdout: "mirror_snapshot_schedule": { 2024-04-25T11:37:41.126 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T11:37:41.127 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.127 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.127 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.127 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.127 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.127 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.127 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.127 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.127 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.127 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.127 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.127 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.127 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.127 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.127 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.127 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.128 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.128 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.128 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.128 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.128 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.128 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.128 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.128 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.128 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.128 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.128 INFO:teuthology.orchestra.run.smithi069.stdout: "trash_purge_schedule": { 2024-04-25T11:37:41.128 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "trash_purge_schedule", 2024-04-25T11:37:41.128 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.128 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.128 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.128 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.129 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.129 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.129 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.129 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.129 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.129 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.129 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.129 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.129 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.129 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.129 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.129 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "restful", 2024-04-25T11:37:41.129 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.129 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.130 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.130 INFO:teuthology.orchestra.run.smithi069.stdout: "enable_auth": { 2024-04-25T11:37:41.130 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enable_auth", 2024-04-25T11:37:41.130 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.130 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.130 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.130 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.130 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.130 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.130 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.130 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.130 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.130 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.130 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.131 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.131 INFO:teuthology.orchestra.run.smithi069.stdout: "key_file": { 2024-04-25T11:37:41.131 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "key_file", 2024-04-25T11:37:41.131 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.131 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.131 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.131 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.131 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.131 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.131 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.131 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.131 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.131 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.131 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.131 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.131 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.132 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.132 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.132 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.132 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.132 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.132 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.132 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.132 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.132 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.132 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.132 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.132 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.132 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.132 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.132 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.132 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.133 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.133 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.133 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.133 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.133 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.133 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.133 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.133 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.133 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.133 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.133 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.133 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.133 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.133 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.133 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.134 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.134 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.134 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.134 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.134 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.134 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.134 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.134 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.134 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.134 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.134 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.134 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.134 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.134 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.134 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.134 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.135 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.135 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.135 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.135 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.135 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.135 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.135 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.135 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.135 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.135 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.135 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.135 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.135 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.135 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.135 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.135 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.136 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.136 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.136 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.136 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.136 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.136 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.136 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2024-04-25T11:37:41.136 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2024-04-25T11:37:41.136 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.136 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.136 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.136 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.136 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.136 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.136 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.137 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.137 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.137 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.137 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.137 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.137 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2024-04-25T11:37:41.137 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2024-04-25T11:37:41.137 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.137 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.137 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.137 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.137 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.137 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.137 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.137 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.138 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.138 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.138 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.138 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.138 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.138 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.138 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.138 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.138 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.138 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.138 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.138 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.138 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.139 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.139 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.139 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.139 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.139 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.139 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.139 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.139 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.139 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rgw", 2024-04-25T11:37:41.139 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.139 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.139 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.139 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.140 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.140 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.140 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.140 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.140 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.140 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.140 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.140 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.140 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.140 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.140 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.140 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.140 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.140 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.141 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.141 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.141 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.141 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.141 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.141 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.141 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.141 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.141 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.141 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.141 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.141 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.141 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.141 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.141 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.142 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.142 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.142 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.142 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.142 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.142 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.142 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.142 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.142 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.142 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.142 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.142 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.142 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.143 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.143 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.143 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.143 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.143 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.143 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.143 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.143 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.143 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.143 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.143 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.143 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.143 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.143 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.144 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.144 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.144 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.144 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.144 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.144 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.144 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.144 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.144 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.144 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.144 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.144 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.144 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.144 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.145 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.145 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.145 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.145 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.145 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.145 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.145 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.145 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.145 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.145 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.145 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.145 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.145 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.145 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.145 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.146 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.146 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rook", 2024-04-25T11:37:41.146 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.146 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.146 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.146 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.146 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.146 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.146 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.146 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.146 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.146 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.146 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.147 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.147 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.147 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.147 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.147 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.147 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.147 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.147 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.147 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.147 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.147 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.147 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.148 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.148 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.148 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.148 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.148 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.148 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.148 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.148 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.148 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.148 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.148 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.148 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.148 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.149 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.149 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.149 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.149 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.149 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.149 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.149 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.149 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.149 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.149 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.149 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.149 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.149 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.149 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.150 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.150 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.150 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.150 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.150 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.150 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.150 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.150 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.150 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.150 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.150 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.150 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.150 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.150 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.151 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.151 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.151 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.151 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.151 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.151 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.151 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.151 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.151 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.151 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.151 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.151 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.151 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.151 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.152 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.152 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.152 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.152 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.152 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.152 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.152 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.152 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.152 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.152 INFO:teuthology.orchestra.run.smithi069.stdout: "storage_class": { 2024-04-25T11:37:41.152 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "storage_class", 2024-04-25T11:37:41.152 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.152 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.152 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.152 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "local", 2024-04-25T11:37:41.153 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.153 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.153 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.153 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T11:37:41.153 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.153 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.153 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.153 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.153 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.153 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.153 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.153 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "selftest", 2024-04-25T11:37:41.153 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.153 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.153 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.154 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.154 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.154 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.154 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.154 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.154 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.154 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.154 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.154 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.154 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.154 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.154 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.154 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.155 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.155 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.155 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.155 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.155 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.155 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.155 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.155 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.155 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.155 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.155 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.155 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.155 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.155 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.155 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.155 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.156 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.156 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.156 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.156 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.156 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.156 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.156 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.156 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.156 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.156 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.156 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.156 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.156 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.156 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.157 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.157 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.157 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.157 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.157 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.157 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.157 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.157 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.157 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.157 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.157 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.157 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.157 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.157 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.157 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.158 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.158 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.158 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.158 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.158 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.158 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.158 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.158 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.158 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.158 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.158 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.158 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.158 INFO:teuthology.orchestra.run.smithi069.stdout: "roption1": { 2024-04-25T11:37:41.159 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "roption1", 2024-04-25T11:37:41.159 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.159 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.159 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.159 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.159 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.159 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.159 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.159 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.159 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.159 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.159 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.159 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.159 INFO:teuthology.orchestra.run.smithi069.stdout: "roption2": { 2024-04-25T11:37:41.160 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "roption2", 2024-04-25T11:37:41.160 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.160 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.160 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.160 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "xyz", 2024-04-25T11:37:41.160 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.160 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.160 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.160 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.160 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.160 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.160 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.160 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.161 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption1": { 2024-04-25T11:37:41.161 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption1", 2024-04-25T11:37:41.161 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.161 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.161 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.161 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.161 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.161 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.161 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.161 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.161 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.161 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.161 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.162 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.162 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption2": { 2024-04-25T11:37:41.162 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption2", 2024-04-25T11:37:41.162 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.162 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.162 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.162 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.162 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.162 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.162 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.162 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.162 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.162 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.162 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.163 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.163 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption3": { 2024-04-25T11:37:41.163 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption3", 2024-04-25T11:37:41.163 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-04-25T11:37:41.163 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.163 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.163 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.163 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.163 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.163 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.163 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.163 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.163 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.163 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.164 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.164 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption4": { 2024-04-25T11:37:41.164 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption4", 2024-04-25T11:37:41.164 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.164 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.164 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.164 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.164 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.164 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.164 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.164 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.164 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.164 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.164 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.164 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.165 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption5": { 2024-04-25T11:37:41.165 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption5", 2024-04-25T11:37:41.165 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.165 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.165 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.165 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.165 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.165 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.165 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.165 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.165 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.165 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.165 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.165 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.166 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption6": { 2024-04-25T11:37:41.166 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption6", 2024-04-25T11:37:41.166 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.166 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.166 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.166 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.166 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.166 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.166 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.166 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.166 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.166 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.166 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.166 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.167 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption7": { 2024-04-25T11:37:41.167 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption7", 2024-04-25T11:37:41.167 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.167 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.167 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.167 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.167 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-04-25T11:37:41.167 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "42", 2024-04-25T11:37:41.167 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.167 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.167 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.167 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.167 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.167 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.168 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.168 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.168 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.168 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.168 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.168 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.168 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.168 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.168 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.168 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.168 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.168 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.168 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.168 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.169 INFO:teuthology.orchestra.run.smithi069.stdout: "testkey": { 2024-04-25T11:37:41.169 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testkey", 2024-04-25T11:37:41.169 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.169 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.169 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.169 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.169 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.169 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.169 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.169 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.169 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.169 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.169 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.169 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.170 INFO:teuthology.orchestra.run.smithi069.stdout: "testlkey": { 2024-04-25T11:37:41.170 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testlkey", 2024-04-25T11:37:41.170 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.170 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.170 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.170 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.170 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.170 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.170 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.170 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.170 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.170 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.170 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.171 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.171 INFO:teuthology.orchestra.run.smithi069.stdout: "testnewline": { 2024-04-25T11:37:41.171 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testnewline", 2024-04-25T11:37:41.171 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.171 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.171 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.171 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.171 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.171 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.171 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.171 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.171 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.171 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.171 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.171 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.172 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.172 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.172 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.172 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snap_schedule", 2024-04-25T11:37:41.172 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.172 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.172 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.172 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_m_granularity": { 2024-04-25T11:37:41.172 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_m_granularity", 2024-04-25T11:37:41.172 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.172 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.172 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.172 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.172 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.172 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.172 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.173 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T11:37:41.173 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.173 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.173 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.173 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.173 INFO:teuthology.orchestra.run.smithi069.stdout: "dump_on_update": { 2024-04-25T11:37:41.173 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "dump_on_update", 2024-04-25T11:37:41.173 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.173 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.173 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.173 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.173 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.173 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.173 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.173 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "dump database to debug log on update", 2024-04-25T11:37:41.174 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.174 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.174 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.174 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.174 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.174 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.174 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.174 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.174 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.174 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.174 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.174 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.174 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.174 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.175 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.175 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.175 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.175 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.175 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.175 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.175 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.175 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.175 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.175 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.175 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.175 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.175 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.176 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.176 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.176 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.176 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.176 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.176 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.176 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.176 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.176 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.176 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.176 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.176 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.176 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.176 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.176 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.177 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.177 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.177 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.177 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.177 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.177 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.177 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.177 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.177 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.177 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.177 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.177 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.177 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.177 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.178 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.178 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.178 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.178 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.178 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.178 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.178 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.178 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.178 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.178 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.178 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.178 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.178 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.178 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.179 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.179 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.179 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.179 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.179 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.179 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.179 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.179 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.179 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.179 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.179 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.179 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.179 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.179 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.180 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.180 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.180 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.180 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.180 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.180 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.180 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.180 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "stats", 2024-04-25T11:37:41.180 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.180 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.180 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.180 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.180 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.180 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.181 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.181 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.181 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.181 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.181 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.181 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.181 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.181 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.181 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.181 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.181 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.181 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.181 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.181 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.182 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.182 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.182 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.182 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.182 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.182 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.182 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.182 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.182 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.182 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.182 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.182 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.182 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.182 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.183 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.183 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.183 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.183 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.183 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.183 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.183 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.183 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.183 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.183 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.183 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.183 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.183 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.183 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.184 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.184 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.184 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.184 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.184 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.184 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.184 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.184 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.184 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.184 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.184 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.184 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.184 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.184 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.184 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.185 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.185 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.185 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.185 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.185 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.185 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.185 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.185 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.185 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.185 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.185 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.185 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.185 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.185 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.185 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.186 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.186 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.186 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.186 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.186 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.186 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.186 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.186 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.186 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.186 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.186 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.186 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.186 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "status", 2024-04-25T11:37:41.186 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.187 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.187 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.187 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.187 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.187 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.187 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.187 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.187 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.187 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.187 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.187 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.187 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.187 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.187 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.187 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.188 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.188 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.188 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.188 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.188 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.188 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.188 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.188 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.188 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.188 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.188 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.188 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.188 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.188 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.189 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.189 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.189 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.189 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.189 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.189 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.189 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.189 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.189 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.189 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.189 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.189 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.189 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.189 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.189 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.190 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.190 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.190 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.190 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.190 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.190 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.190 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.190 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.190 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.190 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.190 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.190 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.190 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.191 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.191 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.191 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.191 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.191 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.191 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.191 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.191 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.191 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.191 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.191 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.191 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.191 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.192 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.192 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.192 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.192 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.192 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.192 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.192 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.192 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.192 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.192 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.192 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.192 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.193 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.193 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.193 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.193 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.193 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.193 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.193 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.193 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "telegraf", 2024-04-25T11:37:41.193 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.193 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.193 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.193 INFO:teuthology.orchestra.run.smithi069.stdout: "address": { 2024-04-25T11:37:41.193 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "address", 2024-04-25T11:37:41.193 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.194 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.194 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.194 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T11:37:41.194 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.194 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.194 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.194 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.194 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.194 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.194 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.194 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.194 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-04-25T11:37:41.194 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-04-25T11:37:41.194 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:41.195 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.195 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.195 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15", 2024-04-25T11:37:41.195 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.195 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.195 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.195 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.195 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.195 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.195 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.195 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.195 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.195 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.195 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.196 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.196 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.196 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.196 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.196 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.196 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.196 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.196 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.196 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.196 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.196 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.196 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.196 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.196 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.196 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.197 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.197 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.197 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.197 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.197 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.197 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.197 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.197 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.197 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.197 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.197 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.197 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.197 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.197 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.198 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.198 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.198 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.198 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.198 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.198 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.198 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.198 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.198 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.198 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.198 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.198 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.198 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.198 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.199 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.199 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.199 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.199 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.199 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.199 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.199 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.199 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.199 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.199 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.199 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.199 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.199 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.199 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.200 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.200 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.200 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.200 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.200 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.200 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.200 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.200 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.200 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.200 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.200 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.200 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.200 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.200 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.200 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.201 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.201 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.201 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.201 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.201 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.201 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.201 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.201 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.201 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.201 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.201 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.201 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.201 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "telemetry", 2024-04-25T11:37:41.201 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.201 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.202 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.202 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_basic": { 2024-04-25T11:37:41.202 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_basic", 2024-04-25T11:37:41.202 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.202 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.202 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.202 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.202 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.202 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.202 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.202 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T11:37:41.202 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.202 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.203 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.203 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.203 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_crash": { 2024-04-25T11:37:41.203 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_crash", 2024-04-25T11:37:41.203 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.203 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.203 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.203 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.203 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.203 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.203 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.203 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T11:37:41.203 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.204 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.204 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.204 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.204 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_device": { 2024-04-25T11:37:41.204 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_device", 2024-04-25T11:37:41.204 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.204 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.204 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.204 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.204 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.204 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.204 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.204 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T11:37:41.204 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.204 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.205 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.205 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.205 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_ident": { 2024-04-25T11:37:41.205 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_ident", 2024-04-25T11:37:41.205 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.205 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.205 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.205 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.205 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.205 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.205 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.205 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T11:37:41.205 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.205 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.206 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.206 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.206 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_perf": { 2024-04-25T11:37:41.206 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_perf", 2024-04-25T11:37:41.206 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.206 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.206 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.206 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.206 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.206 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.206 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.206 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T11:37:41.206 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.206 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.207 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.207 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.207 INFO:teuthology.orchestra.run.smithi069.stdout: "contact": { 2024-04-25T11:37:41.207 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "contact", 2024-04-25T11:37:41.207 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.207 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.207 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.207 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.207 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.207 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.207 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.207 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.207 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.207 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.208 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.208 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.208 INFO:teuthology.orchestra.run.smithi069.stdout: "description": { 2024-04-25T11:37:41.208 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "description", 2024-04-25T11:37:41.208 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.208 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.208 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.208 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.208 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.208 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.208 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.208 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.208 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.208 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.208 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.209 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.209 INFO:teuthology.orchestra.run.smithi069.stdout: "device_url": { 2024-04-25T11:37:41.209 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_url", 2024-04-25T11:37:41.209 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.209 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.209 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.209 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T11:37:41.209 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.209 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.209 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.209 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.209 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.210 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.210 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.210 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.210 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": { 2024-04-25T11:37:41.210 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enabled", 2024-04-25T11:37:41.210 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.210 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.210 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.210 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.210 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.210 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.210 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.210 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.210 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.211 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.211 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.211 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.211 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-04-25T11:37:41.211 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-04-25T11:37:41.211 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.211 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.211 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.211 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "24", 2024-04-25T11:37:41.211 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "8", 2024-04-25T11:37:41.211 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.211 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.211 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.211 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.211 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.212 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.212 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.212 INFO:teuthology.orchestra.run.smithi069.stdout: "last_opt_revision": { 2024-04-25T11:37:41.212 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "last_opt_revision", 2024-04-25T11:37:41.212 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.212 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.212 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.212 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1", 2024-04-25T11:37:41.212 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.212 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.212 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.212 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.212 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.212 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.212 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.213 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.213 INFO:teuthology.orchestra.run.smithi069.stdout: "leaderboard": { 2024-04-25T11:37:41.213 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "leaderboard", 2024-04-25T11:37:41.213 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.213 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.213 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.213 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.213 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.213 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.213 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.213 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.213 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.213 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.213 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.213 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.214 INFO:teuthology.orchestra.run.smithi069.stdout: "leaderboard_description": { 2024-04-25T11:37:41.214 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "leaderboard_description", 2024-04-25T11:37:41.214 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.214 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.214 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.214 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.214 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.214 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.214 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.214 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.214 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.214 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.214 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.214 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.215 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.215 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.215 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.215 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.215 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.215 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.215 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.215 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.215 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.215 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.215 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.215 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.215 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.215 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.215 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.216 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.216 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.216 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.216 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.216 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.216 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.216 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.216 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.216 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.216 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.216 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.216 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.216 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.216 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.217 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.217 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.217 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.217 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.217 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.217 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.217 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.217 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.217 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.217 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.217 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.217 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.217 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.217 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.218 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.218 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.218 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.218 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.218 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.218 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.218 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.218 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.218 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.218 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.218 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.218 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.218 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.218 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.219 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.219 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.219 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.219 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.219 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.219 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.219 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.219 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.219 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.219 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.219 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.219 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.219 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.219 INFO:teuthology.orchestra.run.smithi069.stdout: "organization": { 2024-04-25T11:37:41.219 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "organization", 2024-04-25T11:37:41.220 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.220 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.220 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.220 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.220 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.220 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.220 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.220 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.220 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.220 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.220 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.220 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.220 INFO:teuthology.orchestra.run.smithi069.stdout: "proxy": { 2024-04-25T11:37:41.220 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "proxy", 2024-04-25T11:37:41.220 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.221 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.221 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.221 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.221 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.221 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.221 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.221 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.221 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.221 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.221 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.221 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.221 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.221 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.221 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.221 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.221 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.222 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.222 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.222 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.222 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.222 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.222 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.222 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.222 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.222 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.222 INFO:teuthology.orchestra.run.smithi069.stdout: "url": { 2024-04-25T11:37:41.222 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "url", 2024-04-25T11:37:41.222 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.222 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.222 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.223 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T11:37:41.223 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.223 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.223 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.223 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.223 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.223 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.223 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.223 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.223 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.223 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.223 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.223 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "test_orchestrator", 2024-04-25T11:37:41.223 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.223 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.224 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.224 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.224 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.224 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.224 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.224 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.224 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.224 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.224 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.224 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.224 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.224 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.224 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.224 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.224 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.224 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.225 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.225 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.225 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.225 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.225 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.225 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.225 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.225 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.225 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.225 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.225 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.225 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.225 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.225 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.225 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.226 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.226 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.226 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.226 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.226 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.226 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.226 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.226 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.226 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.226 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.226 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.226 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.226 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.226 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.227 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.227 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.227 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.227 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.227 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.227 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.227 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.227 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.227 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.227 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.227 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.227 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.227 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.227 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.227 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.228 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.228 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.228 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.228 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.228 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.228 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.228 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.228 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.228 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.228 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.228 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.228 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.228 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.228 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.228 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.228 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.229 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.229 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.229 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.229 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.229 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.229 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.229 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.229 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.229 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.229 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.229 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.229 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.229 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "volumes", 2024-04-25T11:37:41.229 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.230 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.230 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.230 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.230 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.230 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.230 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.230 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.230 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.230 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.230 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.230 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.230 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.230 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.231 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.231 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.231 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.231 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.231 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.231 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.231 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.231 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.231 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.231 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.231 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.231 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.231 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.231 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.231 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.231 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.231 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.232 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.232 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.232 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.232 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.232 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.232 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.232 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.232 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.232 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.232 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.232 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.232 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.232 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.232 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.232 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.233 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.233 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.233 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.233 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.233 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.233 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.233 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.233 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.233 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.233 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.233 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.233 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.233 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.233 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.233 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.234 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.234 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.234 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.234 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.234 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.234 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.234 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.234 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.234 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.234 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.234 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.234 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.234 INFO:teuthology.orchestra.run.smithi069.stdout: "max_concurrent_clones": { 2024-04-25T11:37:41.235 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_concurrent_clones", 2024-04-25T11:37:41.235 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.235 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.235 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.235 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "4", 2024-04-25T11:37:41.235 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.235 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.235 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.235 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T11:37:41.235 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.235 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.235 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.235 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.235 INFO:teuthology.orchestra.run.smithi069.stdout: "periodic_async_work": { 2024-04-25T11:37:41.236 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "periodic_async_work", 2024-04-25T11:37:41.236 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.236 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.236 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.236 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.236 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.236 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.236 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.236 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Periodically check for async work", 2024-04-25T11:37:41.236 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.236 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.236 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.236 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.236 INFO:teuthology.orchestra.run.smithi069.stdout: "snapshot_clone_delay": { 2024-04-25T11:37:41.237 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snapshot_clone_delay", 2024-04-25T11:37:41.237 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.237 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.237 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.237 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.237 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.237 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.237 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.237 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T11:37:41.237 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.237 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.237 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.237 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.237 INFO:teuthology.orchestra.run.smithi069.stdout: "snapshot_clone_no_wait": { 2024-04-25T11:37:41.238 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T11:37:41.238 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.238 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.238 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.238 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.238 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.238 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.238 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.238 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T11:37:41.238 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.238 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.238 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.238 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.238 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.239 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.239 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.239 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.239 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.239 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.239 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.239 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.239 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.239 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.239 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.239 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.239 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.239 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.239 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.239 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.239 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.240 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix", 2024-04-25T11:37:41.240 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.240 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.240 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.240 INFO:teuthology.orchestra.run.smithi069.stdout: "discovery_interval": { 2024-04-25T11:37:41.240 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "discovery_interval", 2024-04-25T11:37:41.240 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-04-25T11:37:41.240 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.240 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.240 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "100", 2024-04-25T11:37:41.240 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.240 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.240 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.240 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.241 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.241 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.241 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.241 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.241 INFO:teuthology.orchestra.run.smithi069.stdout: "identifier": { 2024-04-25T11:37:41.241 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "identifier", 2024-04-25T11:37:41.241 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.241 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.241 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.241 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.241 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.241 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.241 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.241 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.241 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.241 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.242 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.242 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.242 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-04-25T11:37:41.242 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-04-25T11:37:41.242 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:41.242 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.242 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.242 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-04-25T11:37:41.242 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.242 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.242 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.242 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.242 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.243 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.243 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.243 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.243 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.243 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.243 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.243 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.243 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.243 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.243 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.243 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.243 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.243 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.243 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.243 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.243 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.244 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.244 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.244 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.244 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.244 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.244 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.244 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.244 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.244 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.244 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.244 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.244 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.244 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.245 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.245 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.245 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.245 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.245 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.245 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.245 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.245 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.245 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.245 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.245 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.245 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.245 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.245 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.246 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.246 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.246 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.246 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.246 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.246 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.246 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.246 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.246 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.246 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.246 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.246 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.246 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.247 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.247 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.247 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.247 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.247 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.247 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.247 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.247 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.247 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.247 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.247 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.247 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.247 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.248 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.248 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.248 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.248 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.248 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.248 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.248 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.248 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.248 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.248 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.248 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.248 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.248 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.248 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.249 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.249 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.249 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.249 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.249 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_host": { 2024-04-25T11:37:41.249 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_host", 2024-04-25T11:37:41.249 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.249 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.249 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.249 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.249 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.249 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.249 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.249 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.250 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.250 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.250 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.250 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.250 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_port": { 2024-04-25T11:37:41.250 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_port", 2024-04-25T11:37:41.250 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.250 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.250 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.250 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10051", 2024-04-25T11:37:41.250 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.250 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.250 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.250 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.251 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.251 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.251 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.251 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.251 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_sender": { 2024-04-25T11:37:41.251 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_sender", 2024-04-25T11:37:41.251 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.251 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.251 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.251 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T11:37:41.251 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.251 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.251 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.252 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.252 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.252 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.252 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.252 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.252 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.252 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.252 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-04-25T11:37:41.252 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.252 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.252 INFO:teuthology.orchestra.run.smithi069.stdout: "gid": 24110, 2024-04-25T11:37:41.252 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "y", 2024-04-25T11:37:41.252 INFO:teuthology.orchestra.run.smithi069.stdout: "mgr_features": 4540701547738038271, 2024-04-25T11:37:41.252 INFO:teuthology.orchestra.run.smithi069.stdout: "available_modules": [ 2024-04-25T11:37:41.252 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.253 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "alerts", 2024-04-25T11:37:41.253 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.253 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.253 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.253 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-04-25T11:37:41.253 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-04-25T11:37:41.253 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:41.253 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.253 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.253 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-04-25T11:37:41.253 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.253 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.253 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.253 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T11:37:41.254 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.254 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.254 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.254 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.254 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.254 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.254 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.254 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.254 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.254 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.254 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.254 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.254 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.254 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.255 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.255 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.255 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.255 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.255 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.255 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.255 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.255 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.255 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.255 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.255 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.255 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.255 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.255 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.255 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.256 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.256 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.256 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.256 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.256 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.256 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.256 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.256 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.256 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.256 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.256 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.256 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.256 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.256 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.257 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.257 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.257 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.257 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.257 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.257 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.257 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.257 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.257 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.257 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.257 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.257 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.257 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.257 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.258 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.258 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.258 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.258 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.258 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.258 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.258 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.258 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.258 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.258 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.258 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.258 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.258 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.258 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.259 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.259 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.259 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.259 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_destination": { 2024-04-25T11:37:41.259 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_destination", 2024-04-25T11:37:41.259 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.259 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.259 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.259 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.259 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.259 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.259 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.259 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Email address to send alerts to", 2024-04-25T11:37:41.259 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.260 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.260 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.260 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.260 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_from_name": { 2024-04-25T11:37:41.260 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_from_name", 2024-04-25T11:37:41.260 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.260 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.260 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.260 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "Ceph", 2024-04-25T11:37:41.260 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.260 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.260 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.260 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Email From: name", 2024-04-25T11:37:41.260 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.261 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.261 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.261 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.261 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_host": { 2024-04-25T11:37:41.261 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_host", 2024-04-25T11:37:41.261 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.261 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.261 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.261 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.261 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.261 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.261 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.261 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP server", 2024-04-25T11:37:41.261 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.262 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.262 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.262 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.262 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_password": { 2024-04-25T11:37:41.262 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_password", 2024-04-25T11:37:41.262 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.262 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.262 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.262 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.262 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.262 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.262 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.262 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Password to authenticate with", 2024-04-25T11:37:41.262 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.263 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.263 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.263 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.263 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_port": { 2024-04-25T11:37:41.263 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_port", 2024-04-25T11:37:41.263 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.263 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.263 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.263 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "465", 2024-04-25T11:37:41.263 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.263 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.263 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.263 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP port", 2024-04-25T11:37:41.263 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.264 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.264 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.264 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.264 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_sender": { 2024-04-25T11:37:41.264 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_sender", 2024-04-25T11:37:41.264 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.264 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.264 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.264 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.264 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.264 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.264 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.264 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP envelope sender", 2024-04-25T11:37:41.264 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.264 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.265 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.265 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.265 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_ssl": { 2024-04-25T11:37:41.265 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_ssl", 2024-04-25T11:37:41.265 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.265 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.265 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.265 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.265 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.265 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.265 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.265 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T11:37:41.265 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.265 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.265 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.265 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.266 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_user": { 2024-04-25T11:37:41.266 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_user", 2024-04-25T11:37:41.266 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.266 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.266 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.266 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.266 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.266 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.266 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.266 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "User to authenticate as", 2024-04-25T11:37:41.266 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.266 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.266 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.266 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.266 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.267 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.267 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.267 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.267 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.267 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.267 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.267 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.267 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.267 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.267 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.267 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.267 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.267 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.267 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.267 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.267 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.268 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "balancer", 2024-04-25T11:37:41.268 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.268 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.268 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.268 INFO:teuthology.orchestra.run.smithi069.stdout: "active": { 2024-04-25T11:37:41.268 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "active", 2024-04-25T11:37:41.268 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.268 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.268 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.268 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.268 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.268 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.268 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.268 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T11:37:41.268 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.269 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.269 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.269 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.269 INFO:teuthology.orchestra.run.smithi069.stdout: "begin_time": { 2024-04-25T11:37:41.269 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "begin_time", 2024-04-25T11:37:41.269 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.269 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.269 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.269 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0000", 2024-04-25T11:37:41.269 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.269 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.269 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.269 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T11:37:41.270 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T11:37:41.270 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.270 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.270 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.270 INFO:teuthology.orchestra.run.smithi069.stdout: "begin_weekday": { 2024-04-25T11:37:41.270 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "begin_weekday", 2024-04-25T11:37:41.270 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-04-25T11:37:41.270 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.270 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.270 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.270 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0", 2024-04-25T11:37:41.270 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "6", 2024-04-25T11:37:41.270 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.270 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T11:37:41.271 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T11:37:41.271 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.271 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.271 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.271 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_max_iterations": { 2024-04-25T11:37:41.271 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_max_iterations", 2024-04-25T11:37:41.271 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-04-25T11:37:41.271 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.271 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.271 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "25", 2024-04-25T11:37:41.271 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-04-25T11:37:41.271 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "250", 2024-04-25T11:37:41.271 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.271 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T11:37:41.272 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.272 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.272 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.272 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.272 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_metrics": { 2024-04-25T11:37:41.272 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_metrics", 2024-04-25T11:37:41.272 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.272 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.272 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.272 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T11:37:41.272 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.272 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.272 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.273 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T11:37:41.273 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T11:37:41.273 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.273 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.273 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.273 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_step": { 2024-04-25T11:37:41.273 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_step", 2024-04-25T11:37:41.273 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-04-25T11:37:41.273 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.273 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.273 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0.5", 2024-04-25T11:37:41.273 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0.001", 2024-04-25T11:37:41.273 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "0.999", 2024-04-25T11:37:41.273 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.273 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "aggressiveness of optimization", 2024-04-25T11:37:41.274 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T11:37:41.274 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.274 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.274 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.274 INFO:teuthology.orchestra.run.smithi069.stdout: "end_time": { 2024-04-25T11:37:41.274 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "end_time", 2024-04-25T11:37:41.274 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.274 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.274 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.274 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "2359", 2024-04-25T11:37:41.274 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.274 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.274 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.275 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T11:37:41.275 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T11:37:41.275 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.275 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.275 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.275 INFO:teuthology.orchestra.run.smithi069.stdout: "end_weekday": { 2024-04-25T11:37:41.275 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "end_weekday", 2024-04-25T11:37:41.275 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-04-25T11:37:41.275 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.275 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.275 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.275 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0", 2024-04-25T11:37:41.275 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "6", 2024-04-25T11:37:41.275 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.275 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T11:37:41.276 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T11:37:41.276 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.276 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.276 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.276 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.276 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.276 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.276 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.276 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.276 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.276 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.276 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.276 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.277 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.277 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.277 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.277 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.277 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.277 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.277 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.277 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.277 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.277 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.277 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.277 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.277 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.277 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.277 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.278 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.278 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.278 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.278 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.278 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.278 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.278 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.278 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.278 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.278 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.278 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.278 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.278 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.278 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.279 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.279 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.279 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.279 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.279 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.279 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.279 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.279 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.279 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.279 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.279 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.279 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.279 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.279 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.280 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.280 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.280 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.280 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.280 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.280 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.280 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.280 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.280 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.280 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.280 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.280 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.280 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.280 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.280 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.281 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.281 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.281 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.281 INFO:teuthology.orchestra.run.smithi069.stdout: "min_score": { 2024-04-25T11:37:41.281 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "min_score", 2024-04-25T11:37:41.281 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-04-25T11:37:41.281 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.281 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.281 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.281 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.281 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.281 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.281 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T11:37:41.281 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.282 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.282 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.282 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.282 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": { 2024-04-25T11:37:41.282 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mode", 2024-04-25T11:37:41.282 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.282 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.282 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.282 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "upmap", 2024-04-25T11:37:41.282 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.282 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.282 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.282 INFO:teuthology.orchestra.run.smithi069.stdout: "crush-compat", 2024-04-25T11:37:41.282 INFO:teuthology.orchestra.run.smithi069.stdout: "none", 2024-04-25T11:37:41.282 INFO:teuthology.orchestra.run.smithi069.stdout: "read", 2024-04-25T11:37:41.283 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap", 2024-04-25T11:37:41.283 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap-read" 2024-04-25T11:37:41.283 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.283 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Balancer mode", 2024-04-25T11:37:41.283 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.283 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.283 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.283 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.283 INFO:teuthology.orchestra.run.smithi069.stdout: "pool_ids": { 2024-04-25T11:37:41.283 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pool_ids", 2024-04-25T11:37:41.283 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.283 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.283 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.283 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.284 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.284 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.284 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.284 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T11:37:41.284 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.284 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.284 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.284 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.284 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-04-25T11:37:41.284 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-04-25T11:37:41.284 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:41.284 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.284 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.284 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-04-25T11:37:41.284 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.285 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.285 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.285 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T11:37:41.285 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.285 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.285 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.285 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.285 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.285 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.285 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.285 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.285 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.285 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.285 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.285 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.286 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.286 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.286 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.286 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.286 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.286 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.286 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap_max_deviation": { 2024-04-25T11:37:41.286 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "upmap_max_deviation", 2024-04-25T11:37:41.286 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.286 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.286 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.286 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-04-25T11:37:41.286 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-04-25T11:37:41.286 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.287 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.287 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T11:37:41.287 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T11:37:41.287 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.287 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.287 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.287 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap_max_optimizations": { 2024-04-25T11:37:41.287 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "upmap_max_optimizations", 2024-04-25T11:37:41.287 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-04-25T11:37:41.287 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.287 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.287 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-25T11:37:41.287 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.287 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.287 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.288 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T11:37:41.288 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.288 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.288 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.288 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.288 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.288 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.288 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.288 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cephadm", 2024-04-25T11:37:41.288 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.288 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.288 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.288 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_down_multiplier": { 2024-04-25T11:37:41.288 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_down_multiplier", 2024-04-25T11:37:41.288 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-04-25T11:37:41.289 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.289 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.289 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3.0", 2024-04-25T11:37:41.289 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.289 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.289 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.289 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T11:37:41.289 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.289 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.289 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.289 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.289 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_refresh_rate": { 2024-04-25T11:37:41.289 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_refresh_rate", 2024-04-25T11:37:41.289 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:41.289 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.290 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.290 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "20", 2024-04-25T11:37:41.290 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.290 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.290 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.290 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T11:37:41.290 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.290 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.290 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.290 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.290 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_starting_port": { 2024-04-25T11:37:41.290 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_starting_port", 2024-04-25T11:37:41.290 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.291 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.291 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.291 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "4721", 2024-04-25T11:37:41.291 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.291 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.291 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.291 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T11:37:41.291 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.291 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.291 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.291 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.291 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_ptrace": { 2024-04-25T11:37:41.291 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_ptrace", 2024-04-25T11:37:41.291 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.291 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.291 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.292 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.292 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.292 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.292 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.292 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T11:37:41.292 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T11:37:41.292 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.292 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.292 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.292 INFO:teuthology.orchestra.run.smithi069.stdout: "autotune_interval": { 2024-04-25T11:37:41.292 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "autotune_interval", 2024-04-25T11:37:41.292 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:41.292 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.292 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.292 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-04-25T11:37:41.292 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.293 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.293 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.293 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T11:37:41.293 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.293 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.293 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.293 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.293 INFO:teuthology.orchestra.run.smithi069.stdout: "autotune_memory_target_ratio": { 2024-04-25T11:37:41.293 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T11:37:41.293 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-04-25T11:37:41.293 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.293 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.293 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0.7", 2024-04-25T11:37:41.294 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.294 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.294 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.294 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T11:37:41.294 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.294 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.294 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.294 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.294 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm_log_destination": { 2024-04-25T11:37:41.294 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cephadm_log_destination", 2024-04-25T11:37:41.294 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.294 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.294 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.294 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.295 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.295 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.295 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.295 INFO:teuthology.orchestra.run.smithi069.stdout: "file", 2024-04-25T11:37:41.295 INFO:teuthology.orchestra.run.smithi069.stdout: "file,syslog", 2024-04-25T11:37:41.295 INFO:teuthology.orchestra.run.smithi069.stdout: "syslog" 2024-04-25T11:37:41.295 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.295 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T11:37:41.295 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.295 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.295 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.295 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.295 INFO:teuthology.orchestra.run.smithi069.stdout: "cgroups_split": { 2024-04-25T11:37:41.295 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cgroups_split", 2024-04-25T11:37:41.295 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.296 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.296 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.296 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.296 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.296 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.296 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.296 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T11:37:41.296 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.296 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.296 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.296 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.296 INFO:teuthology.orchestra.run.smithi069.stdout: "config_checks_enabled": { 2024-04-25T11:37:41.296 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_checks_enabled", 2024-04-25T11:37:41.296 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.296 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.297 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.297 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.297 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.297 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.297 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.297 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T11:37:41.297 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.297 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.297 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.297 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.297 INFO:teuthology.orchestra.run.smithi069.stdout: "config_dashboard": { 2024-04-25T11:37:41.297 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_dashboard", 2024-04-25T11:37:41.297 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.297 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.297 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.298 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.298 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.298 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.298 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.298 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T11:37:41.298 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.298 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.298 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.298 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.298 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_alertmanager": { 2024-04-25T11:37:41.298 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_alertmanager", 2024-04-25T11:37:41.298 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.298 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.298 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.299 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T11:37:41.299 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.299 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.299 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.299 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-04-25T11:37:41.299 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.299 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.299 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.299 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.299 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_base": { 2024-04-25T11:37:41.299 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_base", 2024-04-25T11:37:41.299 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.299 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.299 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.300 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T11:37:41.300 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.300 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.300 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.300 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Container image name, without the tag", 2024-04-25T11:37:41.300 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.300 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.300 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.300 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.300 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_elasticsearch": { 2024-04-25T11:37:41.300 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_elasticsearch", 2024-04-25T11:37:41.300 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.300 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.301 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.301 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T11:37:41.301 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.301 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.301 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.301 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "elasticsearch container image", 2024-04-25T11:37:41.301 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.301 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.301 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.301 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.301 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_grafana": { 2024-04-25T11:37:41.301 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_grafana", 2024-04-25T11:37:41.301 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.301 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.301 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.302 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T11:37:41.302 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.302 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.302 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.302 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-04-25T11:37:41.302 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.302 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.302 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.302 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.302 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_haproxy": { 2024-04-25T11:37:41.302 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_haproxy", 2024-04-25T11:37:41.302 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.302 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.302 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.303 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T11:37:41.303 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.303 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.303 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.303 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "HAproxy container image", 2024-04-25T11:37:41.303 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.303 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.303 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.303 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.303 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_agent": { 2024-04-25T11:37:41.303 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_agent", 2024-04-25T11:37:41.303 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.303 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.303 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.303 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T11:37:41.303 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.304 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.304 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.304 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger agent container image", 2024-04-25T11:37:41.304 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.304 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.304 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.304 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.304 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_collector": { 2024-04-25T11:37:41.304 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_collector", 2024-04-25T11:37:41.304 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.304 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.304 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.304 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T11:37:41.304 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.304 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.305 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.305 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger collector container image", 2024-04-25T11:37:41.305 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.305 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.305 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.305 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.305 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_query": { 2024-04-25T11:37:41.305 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_query", 2024-04-25T11:37:41.305 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.305 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.305 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.305 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T11:37:41.305 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.306 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.306 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.306 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger query container image", 2024-04-25T11:37:41.306 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.306 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.306 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.306 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.306 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_keepalived": { 2024-04-25T11:37:41.306 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_keepalived", 2024-04-25T11:37:41.306 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.306 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.306 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.306 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T11:37:41.306 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.307 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.307 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.307 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Keepalived container image", 2024-04-25T11:37:41.307 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.307 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.307 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.307 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.307 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_loki": { 2024-04-25T11:37:41.307 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_loki", 2024-04-25T11:37:41.307 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.307 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.307 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.307 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T11:37:41.307 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.307 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.308 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.308 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Loki container image", 2024-04-25T11:37:41.308 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.308 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.308 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.308 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.308 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_node_exporter": { 2024-04-25T11:37:41.308 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_node_exporter", 2024-04-25T11:37:41.308 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.308 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.308 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.308 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T11:37:41.308 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.308 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.308 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.308 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-04-25T11:37:41.308 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.309 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.309 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.309 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.309 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_nvmeof": { 2024-04-25T11:37:41.309 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_nvmeof", 2024-04-25T11:37:41.309 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.309 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.309 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.309 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T11:37:41.309 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.309 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.309 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.309 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Nvme-of container image", 2024-04-25T11:37:41.309 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.310 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.310 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.310 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.310 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_prometheus": { 2024-04-25T11:37:41.310 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_prometheus", 2024-04-25T11:37:41.310 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.310 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.310 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.310 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T11:37:41.310 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.310 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.310 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.310 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-04-25T11:37:41.311 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.311 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.311 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.311 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.311 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_promtail": { 2024-04-25T11:37:41.311 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_promtail", 2024-04-25T11:37:41.311 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.311 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.311 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.311 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T11:37:41.311 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.311 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.311 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.311 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Promtail container image", 2024-04-25T11:37:41.311 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.311 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.312 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.312 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.312 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_samba": { 2024-04-25T11:37:41.312 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_samba", 2024-04-25T11:37:41.312 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.312 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.312 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.312 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T11:37:41.312 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.312 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.312 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.312 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Samba/SMB container image", 2024-04-25T11:37:41.312 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.312 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.312 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.313 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.313 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_snmp_gateway": { 2024-04-25T11:37:41.313 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_snmp_gateway", 2024-04-25T11:37:41.313 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.313 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.313 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.313 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T11:37:41.313 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.313 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.313 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.313 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SNMP Gateway container image", 2024-04-25T11:37:41.313 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.313 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.313 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.314 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.314 INFO:teuthology.orchestra.run.smithi069.stdout: "container_init": { 2024-04-25T11:37:41.314 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_init", 2024-04-25T11:37:41.314 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.314 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.314 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.314 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.314 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.314 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.314 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.314 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T11:37:41.314 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.314 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.314 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.314 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.315 INFO:teuthology.orchestra.run.smithi069.stdout: "daemon_cache_timeout": { 2024-04-25T11:37:41.315 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "daemon_cache_timeout", 2024-04-25T11:37:41.315 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:41.315 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.315 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.315 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-04-25T11:37:41.315 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.315 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.315 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.315 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T11:37:41.315 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.315 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.315 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.315 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.316 INFO:teuthology.orchestra.run.smithi069.stdout: "default_cephadm_command_timeout": { 2024-04-25T11:37:41.316 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T11:37:41.316 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.316 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.316 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.316 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "900", 2024-04-25T11:37:41.316 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.316 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.316 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.316 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T11:37:41.316 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.316 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.316 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.316 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.316 INFO:teuthology.orchestra.run.smithi069.stdout: "default_registry": { 2024-04-25T11:37:41.317 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "default_registry", 2024-04-25T11:37:41.317 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.317 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.317 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.317 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io", 2024-04-25T11:37:41.317 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.317 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.317 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.317 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T11:37:41.317 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.317 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.317 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.317 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.317 INFO:teuthology.orchestra.run.smithi069.stdout: "device_cache_timeout": { 2024-04-25T11:37:41.317 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_cache_timeout", 2024-04-25T11:37:41.318 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:41.318 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.318 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.318 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1800", 2024-04-25T11:37:41.318 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.318 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.318 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.318 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache device inventory", 2024-04-25T11:37:41.318 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.318 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.318 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.318 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.318 INFO:teuthology.orchestra.run.smithi069.stdout: "device_enhanced_scan": { 2024-04-25T11:37:41.319 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_enhanced_scan", 2024-04-25T11:37:41.319 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.319 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.319 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.319 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.319 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.319 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.319 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.319 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T11:37:41.319 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.319 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.319 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.319 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.319 INFO:teuthology.orchestra.run.smithi069.stdout: "facts_cache_timeout": { 2024-04-25T11:37:41.320 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "facts_cache_timeout", 2024-04-25T11:37:41.320 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:41.320 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.320 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.320 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-04-25T11:37:41.320 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.320 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.320 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.320 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache host facts data", 2024-04-25T11:37:41.320 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.320 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.320 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.320 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.320 INFO:teuthology.orchestra.run.smithi069.stdout: "grafana_dashboards_path": { 2024-04-25T11:37:41.320 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "grafana_dashboards_path", 2024-04-25T11:37:41.320 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.321 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.321 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.321 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T11:37:41.321 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.321 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.321 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.321 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T11:37:41.321 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.321 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.321 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.321 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.321 INFO:teuthology.orchestra.run.smithi069.stdout: "host_check_interval": { 2024-04-25T11:37:41.321 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "host_check_interval", 2024-04-25T11:37:41.321 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:41.321 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.321 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.322 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-04-25T11:37:41.322 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.322 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.322 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.322 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to perform a host check", 2024-04-25T11:37:41.322 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.322 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.322 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.322 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.322 INFO:teuthology.orchestra.run.smithi069.stdout: "hw_monitoring": { 2024-04-25T11:37:41.322 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "hw_monitoring", 2024-04-25T11:37:41.322 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.322 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.322 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.323 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.323 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.323 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.323 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.323 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T11:37:41.323 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.323 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.323 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.323 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.323 INFO:teuthology.orchestra.run.smithi069.stdout: "inventory_list_all": { 2024-04-25T11:37:41.323 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "inventory_list_all", 2024-04-25T11:37:41.323 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.323 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.323 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.323 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.323 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.323 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.324 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.324 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T11:37:41.324 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.324 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.324 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.324 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.324 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.324 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.324 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.324 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.324 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.324 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.324 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.325 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.325 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.325 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.325 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.325 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.325 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.325 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.325 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.325 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.325 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.325 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.325 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.325 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.325 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.326 INFO:teuthology.orchestra.run.smithi069.stdout: "log_refresh_metadata": { 2024-04-25T11:37:41.326 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_refresh_metadata", 2024-04-25T11:37:41.326 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.326 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.326 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.326 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.326 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.326 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.326 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.326 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T11:37:41.326 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.326 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.326 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.327 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.327 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.327 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.327 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.327 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.327 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.327 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.327 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.327 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.327 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.327 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T11:37:41.327 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.327 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.327 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.327 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.328 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.328 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.328 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.328 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.328 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.328 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.328 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.328 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.328 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.328 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.328 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.328 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.328 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.329 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.329 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.329 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.329 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.329 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.329 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.329 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.329 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.329 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.329 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.329 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.329 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.329 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.329 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.330 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.330 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.330 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.330 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.330 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.330 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.330 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.330 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.330 INFO:teuthology.orchestra.run.smithi069.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T11:37:41.330 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T11:37:41.330 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.330 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.330 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.330 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.331 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.331 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.331 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.331 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T11:37:41.331 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.331 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.331 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.331 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.331 INFO:teuthology.orchestra.run.smithi069.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T11:37:41.331 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T11:37:41.331 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.331 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.331 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.331 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "*", 2024-04-25T11:37:41.332 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.332 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.332 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.332 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T11:37:41.332 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.332 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.332 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.332 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.332 INFO:teuthology.orchestra.run.smithi069.stdout: "max_count_per_host": { 2024-04-25T11:37:41.332 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_count_per_host", 2024-04-25T11:37:41.332 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.332 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.332 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.332 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-25T11:37:41.332 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.333 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.333 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.333 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "max number of daemons per service per host", 2024-04-25T11:37:41.333 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.333 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.333 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.333 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.333 INFO:teuthology.orchestra.run.smithi069.stdout: "max_osd_draining_count": { 2024-04-25T11:37:41.333 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_osd_draining_count", 2024-04-25T11:37:41.333 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.333 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.333 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.333 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-25T11:37:41.333 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.333 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.334 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.334 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T11:37:41.334 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.334 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.334 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.334 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.334 INFO:teuthology.orchestra.run.smithi069.stdout: "migration_current": { 2024-04-25T11:37:41.334 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "migration_current", 2024-04-25T11:37:41.334 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.334 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.334 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.334 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.334 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.334 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.335 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.335 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "internal - do not modify", 2024-04-25T11:37:41.335 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.335 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.335 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.335 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.335 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": { 2024-04-25T11:37:41.335 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mode", 2024-04-25T11:37:41.335 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.335 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.335 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.335 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "root", 2024-04-25T11:37:41.335 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.335 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.336 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.336 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm-package", 2024-04-25T11:37:41.336 INFO:teuthology.orchestra.run.smithi069.stdout: "root" 2024-04-25T11:37:41.336 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.336 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T11:37:41.336 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.336 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.336 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.336 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.336 INFO:teuthology.orchestra.run.smithi069.stdout: "oob_default_addr": { 2024-04-25T11:37:41.336 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "oob_default_addr", 2024-04-25T11:37:41.336 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.336 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.336 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.337 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "169.254.1.1", 2024-04-25T11:37:41.337 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.337 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.337 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.337 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T11:37:41.337 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.337 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.337 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.337 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.337 INFO:teuthology.orchestra.run.smithi069.stdout: "prometheus_alerts_path": { 2024-04-25T11:37:41.337 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prometheus_alerts_path", 2024-04-25T11:37:41.337 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.337 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.337 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.337 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T11:37:41.338 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.338 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.338 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.338 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T11:37:41.338 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.338 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.338 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.338 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.338 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_insecure": { 2024-04-25T11:37:41.338 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_insecure", 2024-04-25T11:37:41.338 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.338 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.338 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.338 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.339 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.339 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.339 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.339 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T11:37:41.339 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.339 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.339 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.339 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.339 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_password": { 2024-04-25T11:37:41.339 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_password", 2024-04-25T11:37:41.339 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.339 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.339 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.339 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.340 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.340 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.340 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.340 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T11:37:41.340 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.340 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.340 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.340 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.340 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_url": { 2024-04-25T11:37:41.340 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_url", 2024-04-25T11:37:41.340 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.340 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.340 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.340 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.341 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.341 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.341 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.341 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T11:37:41.341 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.341 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.341 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.341 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.341 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_username": { 2024-04-25T11:37:41.341 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_username", 2024-04-25T11:37:41.341 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.341 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.341 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.341 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.342 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.342 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.342 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.342 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T11:37:41.342 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.342 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.342 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.342 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.342 INFO:teuthology.orchestra.run.smithi069.stdout: "secure_monitoring_stack": { 2024-04-25T11:37:41.342 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "secure_monitoring_stack", 2024-04-25T11:37:41.342 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.342 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.342 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.342 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.343 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.343 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.343 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.343 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T11:37:41.343 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.343 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.343 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.343 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.343 INFO:teuthology.orchestra.run.smithi069.stdout: "service_discovery_port": { 2024-04-25T11:37:41.343 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "service_discovery_port", 2024-04-25T11:37:41.343 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.343 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.343 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.343 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8765", 2024-04-25T11:37:41.344 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.344 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.344 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.344 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "cephadm service discovery port", 2024-04-25T11:37:41.344 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.344 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.344 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.344 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.344 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.344 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.344 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.344 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.344 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.344 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.345 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.345 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.345 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.345 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.345 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.345 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.345 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.345 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.345 INFO:teuthology.orchestra.run.smithi069.stdout: "ssh_config_file": { 2024-04-25T11:37:41.345 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssh_config_file", 2024-04-25T11:37:41.345 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.345 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.345 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.345 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.345 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.345 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.346 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.346 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T11:37:41.346 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.346 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.346 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.346 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.346 INFO:teuthology.orchestra.run.smithi069.stdout: "use_agent": { 2024-04-25T11:37:41.346 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "use_agent", 2024-04-25T11:37:41.346 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.346 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.346 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.346 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.346 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.346 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.347 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.347 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T11:37:41.347 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.347 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.347 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.347 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.347 INFO:teuthology.orchestra.run.smithi069.stdout: "use_repo_digest": { 2024-04-25T11:37:41.347 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "use_repo_digest", 2024-04-25T11:37:41.347 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.347 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.347 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.347 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.347 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.347 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.347 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.348 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T11:37:41.348 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.348 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.348 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.348 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.348 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_failed_host_check": { 2024-04-25T11:37:41.348 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_failed_host_check", 2024-04-25T11:37:41.348 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.348 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.348 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.348 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.348 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.348 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.348 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.348 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T11:37:41.348 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.349 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.349 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.349 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.349 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_stray_daemons": { 2024-04-25T11:37:41.349 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_stray_daemons", 2024-04-25T11:37:41.349 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.349 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.349 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.349 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.349 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.349 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.349 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.349 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T11:37:41.350 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.350 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.350 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.350 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.350 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_stray_hosts": { 2024-04-25T11:37:41.350 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_stray_hosts", 2024-04-25T11:37:41.350 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.350 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.350 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.350 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.350 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.350 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.350 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.351 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T11:37:41.351 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.351 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.351 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.351 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.351 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.351 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.351 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.351 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crash", 2024-04-25T11:37:41.351 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.351 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.351 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.351 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.351 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.352 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.352 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.352 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.352 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.352 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.352 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.352 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.352 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.352 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.352 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.352 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.352 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.352 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.352 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.352 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.353 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.353 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.353 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.353 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.353 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.353 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.353 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.353 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.353 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.353 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.353 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.353 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.353 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.353 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.353 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.354 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.354 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.354 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.354 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.354 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.354 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.354 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.354 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.354 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.354 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.354 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.354 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.354 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.355 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.355 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.355 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.355 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.355 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.355 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.355 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.355 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.355 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.355 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.355 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.355 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.355 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.355 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.355 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.356 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.356 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.356 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.356 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.356 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.356 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.356 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.356 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.356 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.356 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.356 INFO:teuthology.orchestra.run.smithi069.stdout: "retain_interval": { 2024-04-25T11:37:41.356 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "retain_interval", 2024-04-25T11:37:41.356 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:41.356 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.357 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.357 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "31536000", 2024-04-25T11:37:41.357 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.357 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.357 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.357 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T11:37:41.357 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.357 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.357 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.357 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.357 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.357 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.357 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.357 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.357 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.358 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.358 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.358 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.358 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.358 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.358 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.358 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.358 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.358 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.358 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_recent_interval": { 2024-04-25T11:37:41.358 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_recent_interval", 2024-04-25T11:37:41.358 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:41.358 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.358 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.359 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1209600", 2024-04-25T11:37:41.359 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.359 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.359 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.359 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T11:37:41.359 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.359 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.359 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.359 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.359 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.359 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.359 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.359 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "dashboard", 2024-04-25T11:37:41.359 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.360 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.360 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.360 INFO:teuthology.orchestra.run.smithi069.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T11:37:41.360 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T11:37:41.360 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.360 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.360 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.360 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-25T11:37:41.360 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.360 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.360 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.360 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.360 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.360 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.361 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.361 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.361 INFO:teuthology.orchestra.run.smithi069.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T11:37:41.361 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T11:37:41.361 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.361 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.361 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.361 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.361 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.361 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.361 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.361 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.361 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.361 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.362 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.362 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.362 INFO:teuthology.orchestra.run.smithi069.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T11:37:41.362 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T11:37:41.362 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.362 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.362 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.362 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.362 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.362 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.362 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.362 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.362 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.362 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.363 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.363 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.363 INFO:teuthology.orchestra.run.smithi069.stdout: "AUDIT_API_ENABLED": { 2024-04-25T11:37:41.363 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T11:37:41.363 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.363 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.363 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.363 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.363 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.363 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.363 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.363 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.363 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.363 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.364 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.364 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.364 INFO:teuthology.orchestra.run.smithi069.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T11:37:41.364 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T11:37:41.364 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.364 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.364 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.364 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.364 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.364 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.364 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.364 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.364 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.364 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.364 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.365 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.365 INFO:teuthology.orchestra.run.smithi069.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T11:37:41.365 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T11:37:41.365 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.365 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.365 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.365 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.365 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.365 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.365 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.365 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.365 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.365 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.365 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.365 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.366 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T11:37:41.366 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T11:37:41.366 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.366 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.366 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.366 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.366 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.366 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.366 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.366 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.366 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.366 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.366 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.367 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.367 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T11:37:41.367 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T11:37:41.367 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.367 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.367 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.367 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.367 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.367 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.367 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.367 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.367 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.367 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.367 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.367 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.368 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T11:37:41.368 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T11:37:41.368 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.368 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.368 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.368 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.368 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.368 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.368 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.368 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.368 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.368 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.368 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.368 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.368 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T11:37:41.369 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T11:37:41.369 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.369 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.369 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.369 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.369 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.369 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.369 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.369 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.369 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.369 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.369 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.369 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.370 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T11:37:41.370 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T11:37:41.370 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.370 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.370 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.370 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.370 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.370 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.370 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.370 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.370 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.370 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.370 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.370 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.371 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T11:37:41.371 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T11:37:41.371 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.371 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.371 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.371 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.371 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.371 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.371 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.371 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.371 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.371 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.371 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.372 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.372 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T11:37:41.372 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T11:37:41.372 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.372 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.372 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.372 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.372 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.372 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.372 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.372 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.372 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.372 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.372 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.372 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.372 INFO:teuthology.orchestra.run.smithi069.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T11:37:41.373 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T11:37:41.373 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.373 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.373 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.373 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.373 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.373 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.373 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.373 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.373 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.373 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.373 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.373 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.373 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T11:37:41.373 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T11:37:41.373 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.374 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.374 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.374 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2024-04-25T11:37:41.374 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.374 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.374 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.374 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.374 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.374 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.374 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.374 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.374 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T11:37:41.374 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T11:37:41.374 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.374 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.375 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.375 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.375 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.375 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.375 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.375 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.375 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.375 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.375 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.375 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.375 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_URL": { 2024-04-25T11:37:41.375 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_URL", 2024-04-25T11:37:41.375 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.375 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.375 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.375 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.376 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.376 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.376 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.376 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.376 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.376 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.376 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.376 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.376 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T11:37:41.376 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T11:37:41.376 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.376 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.376 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.377 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2024-04-25T11:37:41.377 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.377 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.377 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.377 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.377 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.377 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.377 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.377 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.377 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T11:37:41.377 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T11:37:41.377 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.377 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.377 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.377 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.377 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.378 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.378 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.378 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.378 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.378 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.378 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.378 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.378 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T11:37:41.378 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T11:37:41.378 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.378 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.378 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.378 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.378 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.379 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.379 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.379 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.379 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.379 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.379 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.379 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.379 INFO:teuthology.orchestra.run.smithi069.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T11:37:41.379 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T11:37:41.379 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.379 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.379 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.379 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.380 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.380 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.380 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.380 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.380 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.380 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.380 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.380 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.380 INFO:teuthology.orchestra.run.smithi069.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T11:37:41.380 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T11:37:41.380 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.380 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.380 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.380 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.381 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.381 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.381 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.381 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.381 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.381 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.381 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.381 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.381 INFO:teuthology.orchestra.run.smithi069.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T11:37:41.381 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T11:37:41.381 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.381 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.381 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.381 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "[]", 2024-04-25T11:37:41.382 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.382 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.382 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.382 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.382 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.382 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.382 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.382 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.382 INFO:teuthology.orchestra.run.smithi069.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T11:37:41.382 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T11:37:41.382 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.382 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.382 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.382 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "{}", 2024-04-25T11:37:41.382 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.383 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.383 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.383 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.383 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.383 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.383 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.383 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.383 INFO:teuthology.orchestra.run.smithi069.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T11:37:41.383 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T11:37:41.383 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.383 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.383 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.383 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.383 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.384 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.384 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.384 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.384 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.384 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.384 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.384 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.384 INFO:teuthology.orchestra.run.smithi069.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T11:37:41.384 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T11:37:41.384 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.384 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.384 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.384 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.384 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.385 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.385 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.385 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.385 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.385 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.385 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.385 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.385 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T11:37:41.385 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T11:37:41.385 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.385 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.385 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.385 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.385 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.386 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.386 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.386 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.386 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.386 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.386 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.386 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.386 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T11:37:41.386 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T11:37:41.386 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.386 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.386 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.386 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.386 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.387 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.387 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.387 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.387 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.387 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.387 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.387 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.387 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T11:37:41.387 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T11:37:41.387 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.387 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.387 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.387 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.387 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.388 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.388 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.388 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.388 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.388 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.388 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.388 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.388 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T11:37:41.388 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T11:37:41.388 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.388 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.388 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.388 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.388 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.388 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.389 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.389 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.389 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.389 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.389 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.389 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.389 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T11:37:41.389 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T11:37:41.389 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.389 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.389 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.389 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.389 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.390 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.390 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.390 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.390 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.390 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.390 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.390 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.390 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T11:37:41.390 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T11:37:41.390 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.390 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.390 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.390 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.390 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.391 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.391 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.391 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.391 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.391 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.391 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.391 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.391 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T11:37:41.391 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T11:37:41.391 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.391 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.391 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.391 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.391 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.391 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.392 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.392 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.392 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.392 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.392 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.392 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.392 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T11:37:41.392 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T11:37:41.392 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.392 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.392 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.392 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.392 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.392 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.392 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.393 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.393 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.393 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.393 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.393 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.393 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T11:37:41.393 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T11:37:41.393 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.393 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.393 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.393 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T11:37:41.393 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.393 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.393 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.393 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.394 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.394 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.394 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.394 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.394 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T11:37:41.394 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T11:37:41.394 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.394 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.394 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.394 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-25T11:37:41.394 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.394 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.394 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.394 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.395 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.395 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.395 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.395 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.395 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T11:37:41.395 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T11:37:41.395 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.395 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.395 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.395 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8", 2024-04-25T11:37:41.395 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.395 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.395 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.395 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.395 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.396 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.396 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.396 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.396 INFO:teuthology.orchestra.run.smithi069.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T11:37:41.396 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T11:37:41.396 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.396 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.396 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.396 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "45", 2024-04-25T11:37:41.396 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.396 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.396 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.396 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.396 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.397 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.397 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.397 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.397 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T11:37:41.397 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T11:37:41.397 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.397 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.397 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.397 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.397 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.397 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.397 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.397 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.397 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.397 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.398 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.398 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.398 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T11:37:41.398 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T11:37:41.398 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.398 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.398 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.398 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2024-04-25T11:37:41.398 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.398 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.398 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.398 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.398 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.398 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.399 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.399 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.399 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T11:37:41.399 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T11:37:41.399 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.399 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.399 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.399 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.399 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.399 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.399 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.399 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.399 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.400 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.400 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.400 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.400 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T11:37:41.400 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T11:37:41.400 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.400 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.400 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.400 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.400 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.400 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.400 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.400 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.400 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.400 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.401 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.401 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.401 INFO:teuthology.orchestra.run.smithi069.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T11:37:41.401 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T11:37:41.401 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.401 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.401 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.401 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.401 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.401 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.401 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.401 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.401 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.401 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.401 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.402 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.402 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T11:37:41.402 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T11:37:41.402 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.402 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.402 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.402 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.402 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.402 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.402 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.402 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.402 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.402 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.402 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.403 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.403 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T11:37:41.403 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T11:37:41.403 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.403 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.403 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.403 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-25T11:37:41.403 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.403 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.403 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.403 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.403 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.403 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.403 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.403 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.404 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T11:37:41.404 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T11:37:41.404 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.404 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.404 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.404 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-04-25T11:37:41.404 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.404 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.404 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.404 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.404 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.404 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.404 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.405 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.405 INFO:teuthology.orchestra.run.smithi069.stdout: "cross_origin_url": { 2024-04-25T11:37:41.405 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cross_origin_url", 2024-04-25T11:37:41.405 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.405 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.405 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.405 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.405 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.405 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.405 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.405 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.405 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.405 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.405 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.405 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.406 INFO:teuthology.orchestra.run.smithi069.stdout: "crt_file": { 2024-04-25T11:37:41.406 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crt_file", 2024-04-25T11:37:41.406 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.406 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.406 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.406 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.406 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.406 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.406 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.406 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.406 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.406 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.406 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.406 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.407 INFO:teuthology.orchestra.run.smithi069.stdout: "debug": { 2024-04-25T11:37:41.407 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "debug", 2024-04-25T11:37:41.407 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.407 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.407 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.407 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.407 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.407 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.407 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.407 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable/disable debug options", 2024-04-25T11:37:41.407 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.407 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.407 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.407 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.407 INFO:teuthology.orchestra.run.smithi069.stdout: "jwt_token_ttl": { 2024-04-25T11:37:41.408 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "jwt_token_ttl", 2024-04-25T11:37:41.408 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.408 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.408 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.408 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "28800", 2024-04-25T11:37:41.408 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.408 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.408 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.408 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.408 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.408 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.408 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.408 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.408 INFO:teuthology.orchestra.run.smithi069.stdout: "key_file": { 2024-04-25T11:37:41.409 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "key_file", 2024-04-25T11:37:41.409 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.409 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.409 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.409 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.409 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.409 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.409 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.409 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.409 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.409 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.409 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.409 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.409 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.409 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.410 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.410 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.410 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.410 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.410 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.410 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.410 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.410 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.410 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.410 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.410 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.410 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.410 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.411 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.411 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.411 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.411 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.411 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.411 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.411 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.411 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.411 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.411 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.411 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.411 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.411 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.411 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.412 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.412 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.412 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.412 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.412 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.412 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.412 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.412 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.412 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.412 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.412 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.412 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.412 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.412 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.413 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.413 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.413 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.413 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.413 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.413 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.413 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.413 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.413 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.413 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.413 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.413 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.413 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.413 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.413 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.414 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.414 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.414 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.414 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.414 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.414 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.414 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.414 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.414 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.414 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.414 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.414 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.415 INFO:teuthology.orchestra.run.smithi069.stdout: "motd": { 2024-04-25T11:37:41.415 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "motd", 2024-04-25T11:37:41.415 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.415 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.415 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.415 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.415 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.415 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.415 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.415 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "The message of the day", 2024-04-25T11:37:41.415 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.415 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.415 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.415 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.415 INFO:teuthology.orchestra.run.smithi069.stdout: "redirect_resolve_ip_addr": { 2024-04-25T11:37:41.416 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T11:37:41.416 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.416 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.416 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.416 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.416 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.416 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.416 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.416 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.416 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.416 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.416 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.416 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.416 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2024-04-25T11:37:41.417 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2024-04-25T11:37:41.417 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.417 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.417 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.417 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "::", 2024-04-25T11:37:41.417 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.417 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.417 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.417 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.417 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.417 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.417 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.417 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.417 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2024-04-25T11:37:41.417 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2024-04-25T11:37:41.418 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.418 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.418 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.418 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8080", 2024-04-25T11:37:41.418 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.418 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.418 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.418 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.418 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.418 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.418 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.418 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.419 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.419 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.419 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.419 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.419 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.419 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.419 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.419 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.419 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.419 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.419 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.419 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.419 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.419 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.419 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl": { 2024-04-25T11:37:41.420 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl", 2024-04-25T11:37:41.420 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.420 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.420 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.420 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.420 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.420 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.420 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.420 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.420 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.420 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.420 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.420 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.420 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl_server_port": { 2024-04-25T11:37:41.420 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl_server_port", 2024-04-25T11:37:41.421 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.421 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.421 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.421 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8443", 2024-04-25T11:37:41.421 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.421 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.421 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.421 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.421 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.421 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.421 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.421 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.421 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_behaviour": { 2024-04-25T11:37:41.421 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_behaviour", 2024-04-25T11:37:41.421 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.421 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.422 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.422 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "redirect", 2024-04-25T11:37:41.422 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.422 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.422 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.422 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.422 INFO:teuthology.orchestra.run.smithi069.stdout: "redirect" 2024-04-25T11:37:41.422 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.422 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.422 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.422 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.422 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.422 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.422 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_error_status_code": { 2024-04-25T11:37:41.423 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_error_status_code", 2024-04-25T11:37:41.423 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.423 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.423 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.423 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "500", 2024-04-25T11:37:41.423 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "400", 2024-04-25T11:37:41.423 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "599", 2024-04-25T11:37:41.423 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.423 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.423 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.423 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.423 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.423 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.423 INFO:teuthology.orchestra.run.smithi069.stdout: "url_prefix": { 2024-04-25T11:37:41.423 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "url_prefix", 2024-04-25T11:37:41.424 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.424 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.424 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.424 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.424 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.424 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.424 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.424 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.424 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.424 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.424 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.424 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.424 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.424 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.424 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.425 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "devicehealth", 2024-04-25T11:37:41.425 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.425 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.425 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.425 INFO:teuthology.orchestra.run.smithi069.stdout: "enable_monitoring": { 2024-04-25T11:37:41.425 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enable_monitoring", 2024-04-25T11:37:41.425 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.425 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.425 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.425 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.425 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.425 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.425 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.425 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "monitor device health metrics", 2024-04-25T11:37:41.426 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.426 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.426 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.426 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.426 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.426 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.426 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.426 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.426 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.426 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.426 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.426 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.426 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.427 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.427 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.427 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.427 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.427 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.427 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.427 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.427 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.427 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.427 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.427 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.427 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.427 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.427 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.427 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.427 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.428 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.428 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.428 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.428 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.428 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.428 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.428 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.428 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.428 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.428 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.428 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.428 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.428 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.428 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.428 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.428 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.429 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.429 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.429 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.429 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.429 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.429 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.429 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.429 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.429 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.429 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.429 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.429 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.429 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.430 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.430 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.430 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.430 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.430 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.430 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.430 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.430 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.430 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.430 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.430 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.430 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.430 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.430 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.431 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.431 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.431 INFO:teuthology.orchestra.run.smithi069.stdout: "mark_out_threshold": { 2024-04-25T11:37:41.431 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mark_out_threshold", 2024-04-25T11:37:41.431 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:41.431 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.431 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.431 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "2419200", 2024-04-25T11:37:41.431 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.431 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.431 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.431 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T11:37:41.431 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.432 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.432 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.432 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.432 INFO:teuthology.orchestra.run.smithi069.stdout: "pool_name": { 2024-04-25T11:37:41.432 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pool_name", 2024-04-25T11:37:41.432 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.432 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.432 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.432 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "device_health_metrics", 2024-04-25T11:37:41.432 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.432 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.432 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.432 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T11:37:41.432 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.432 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.432 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.433 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.433 INFO:teuthology.orchestra.run.smithi069.stdout: "retention_period": { 2024-04-25T11:37:41.433 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "retention_period", 2024-04-25T11:37:41.433 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:41.433 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.433 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.433 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15552000", 2024-04-25T11:37:41.433 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.433 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.433 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.433 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long to retain device health metrics", 2024-04-25T11:37:41.433 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.433 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.433 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.433 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.433 INFO:teuthology.orchestra.run.smithi069.stdout: "scrape_frequency": { 2024-04-25T11:37:41.434 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "scrape_frequency", 2024-04-25T11:37:41.434 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:41.434 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.434 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.434 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "86400", 2024-04-25T11:37:41.434 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.434 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.434 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.434 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T11:37:41.434 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.434 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.434 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.434 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.434 INFO:teuthology.orchestra.run.smithi069.stdout: "self_heal": { 2024-04-25T11:37:41.434 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "self_heal", 2024-04-25T11:37:41.435 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.435 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.435 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.435 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.435 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.435 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.435 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.435 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T11:37:41.435 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.435 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.435 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.435 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.435 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-04-25T11:37:41.435 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-04-25T11:37:41.435 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:41.435 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.436 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.436 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-04-25T11:37:41.436 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.436 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.436 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.436 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T11:37:41.436 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.436 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.436 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.436 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.436 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.436 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.436 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.436 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.436 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.436 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.437 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.437 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.437 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.437 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.437 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.437 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.437 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.437 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.437 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_threshold": { 2024-04-25T11:37:41.437 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_threshold", 2024-04-25T11:37:41.437 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:41.437 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.437 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.437 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "7257600", 2024-04-25T11:37:41.437 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.437 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.438 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.438 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T11:37:41.438 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.438 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.438 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.438 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.438 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.438 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.438 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.438 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "diskprediction_local", 2024-04-25T11:37:41.438 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.438 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.438 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.438 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.438 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.439 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.439 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.439 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.439 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.439 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.439 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.439 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.439 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.439 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.439 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.439 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.439 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.439 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.439 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.439 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.440 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.440 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.440 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.440 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.440 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.440 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.440 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.440 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.440 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.440 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.440 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.440 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.440 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.440 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.440 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.440 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.441 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.441 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.441 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.441 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.441 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.441 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.441 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.441 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.441 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.441 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.441 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.441 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.441 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.441 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.441 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.442 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.442 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.442 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.442 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.442 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.442 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.442 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.442 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.442 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.442 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.442 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.442 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.442 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.442 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.443 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.443 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.443 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.443 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.443 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.443 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.443 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.443 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.443 INFO:teuthology.orchestra.run.smithi069.stdout: "predict_interval": { 2024-04-25T11:37:41.443 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "predict_interval", 2024-04-25T11:37:41.443 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.444 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.444 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.444 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "86400", 2024-04-25T11:37:41.444 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.444 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.444 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.444 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.444 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.444 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.444 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.444 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.444 INFO:teuthology.orchestra.run.smithi069.stdout: "predictor_model": { 2024-04-25T11:37:41.444 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "predictor_model", 2024-04-25T11:37:41.445 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.445 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.445 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.445 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "prophetstor", 2024-04-25T11:37:41.445 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.445 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.445 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.445 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.445 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.445 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.445 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.445 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.445 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-04-25T11:37:41.445 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-04-25T11:37:41.446 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.446 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.446 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.446 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-04-25T11:37:41.446 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.446 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.446 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.446 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.446 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.446 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.446 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.446 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.446 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.447 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.447 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.447 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.447 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.447 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.447 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.447 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.447 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.447 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.447 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.447 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.447 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.447 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.447 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.448 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.448 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.448 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "influx", 2024-04-25T11:37:41.448 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": false, 2024-04-25T11:37:41.448 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "influxdb python module not found", 2024-04-25T11:37:41.448 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.448 INFO:teuthology.orchestra.run.smithi069.stdout: "batch_size": { 2024-04-25T11:37:41.448 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "batch_size", 2024-04-25T11:37:41.448 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.448 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.448 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.448 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5000", 2024-04-25T11:37:41.448 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.448 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.448 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.449 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T11:37:41.449 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.449 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.449 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.449 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.449 INFO:teuthology.orchestra.run.smithi069.stdout: "database": { 2024-04-25T11:37:41.449 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "database", 2024-04-25T11:37:41.449 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.449 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.449 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.449 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "ceph", 2024-04-25T11:37:41.449 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.449 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.450 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.450 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T11:37:41.450 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.450 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.450 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.450 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.450 INFO:teuthology.orchestra.run.smithi069.stdout: "hostname": { 2024-04-25T11:37:41.450 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "hostname", 2024-04-25T11:37:41.450 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.450 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.450 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.450 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.450 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.450 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.451 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.451 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB server hostname", 2024-04-25T11:37:41.451 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.451 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.451 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.451 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.451 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-04-25T11:37:41.451 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-04-25T11:37:41.451 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:41.451 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.451 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.451 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "30", 2024-04-25T11:37:41.451 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "5", 2024-04-25T11:37:41.451 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.452 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.452 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T11:37:41.452 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.452 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.452 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.452 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.452 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.452 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.452 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.452 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.452 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.452 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.452 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.452 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.452 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.453 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.453 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.453 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.453 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.453 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.453 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.453 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.453 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.453 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.453 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.453 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.453 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.453 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.453 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.453 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.454 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.454 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.454 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.454 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.454 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.454 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.454 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.454 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.454 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.454 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.454 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.454 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.454 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.455 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.455 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.455 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.455 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.455 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.455 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.455 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.455 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.455 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.455 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.455 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.455 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.455 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.455 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.455 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.456 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.456 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.456 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.456 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.456 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.456 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.456 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.456 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.456 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.456 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.456 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.456 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.456 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.456 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.456 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.456 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.457 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.457 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.457 INFO:teuthology.orchestra.run.smithi069.stdout: "password": { 2024-04-25T11:37:41.457 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "password", 2024-04-25T11:37:41.457 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.457 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.457 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.457 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.457 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.457 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.457 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.457 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "password of InfluxDB server user", 2024-04-25T11:37:41.457 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.458 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.458 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.458 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.458 INFO:teuthology.orchestra.run.smithi069.stdout: "port": { 2024-04-25T11:37:41.458 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "port", 2024-04-25T11:37:41.458 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.458 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.458 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.458 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8086", 2024-04-25T11:37:41.458 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.458 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.458 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.458 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB server port", 2024-04-25T11:37:41.458 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.459 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.459 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.459 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.459 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.459 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.459 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.459 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.459 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.459 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.459 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.459 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.459 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.459 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.459 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.459 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.460 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.460 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.460 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl": { 2024-04-25T11:37:41.460 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl", 2024-04-25T11:37:41.460 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.460 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.460 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.460 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "false", 2024-04-25T11:37:41.460 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.460 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.460 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.460 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T11:37:41.460 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.460 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.460 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.461 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.461 INFO:teuthology.orchestra.run.smithi069.stdout: "threads": { 2024-04-25T11:37:41.461 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "threads", 2024-04-25T11:37:41.461 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.461 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.461 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.461 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-04-25T11:37:41.461 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-04-25T11:37:41.461 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "32", 2024-04-25T11:37:41.461 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.461 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T11:37:41.461 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.461 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.461 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.462 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.462 INFO:teuthology.orchestra.run.smithi069.stdout: "username": { 2024-04-25T11:37:41.462 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "username", 2024-04-25T11:37:41.462 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.462 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.462 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.462 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.462 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.462 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.462 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.462 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "username of InfluxDB server user", 2024-04-25T11:37:41.462 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.462 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.463 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.463 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.463 INFO:teuthology.orchestra.run.smithi069.stdout: "verify_ssl": { 2024-04-25T11:37:41.463 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "verify_ssl", 2024-04-25T11:37:41.463 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.463 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.463 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.463 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "true", 2024-04-25T11:37:41.463 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.463 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.463 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.463 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T11:37:41.463 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.463 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.463 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.464 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.464 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.464 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.464 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.464 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "insights", 2024-04-25T11:37:41.464 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.464 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.464 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.464 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.464 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.464 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.464 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.464 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.464 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.465 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.465 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.465 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.465 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.465 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.465 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.465 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.465 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.465 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.465 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.465 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.465 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.465 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.465 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.465 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.466 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.466 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.466 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.466 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.466 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.466 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.466 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.466 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.466 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.466 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.466 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.466 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.466 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.466 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.467 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.467 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.467 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.467 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.467 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.467 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.467 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.467 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.467 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.467 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.467 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.467 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.467 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.467 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.467 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.468 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.468 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.468 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.468 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.468 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.468 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.468 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.468 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.468 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.468 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.468 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.468 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.468 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.468 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.468 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.469 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.469 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.469 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.469 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.469 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.469 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.469 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.469 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.469 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.469 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.469 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.469 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.469 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.469 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.470 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.470 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.470 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.470 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.470 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.470 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.470 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.470 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.470 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "iostat", 2024-04-25T11:37:41.470 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.470 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.470 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.471 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.471 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.471 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.471 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.471 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.471 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.471 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.471 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.471 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.471 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.471 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.471 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.471 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.472 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.472 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.472 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.472 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.472 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.472 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.472 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.472 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.472 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.472 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.472 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.472 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.472 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.472 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.473 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.473 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.473 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.473 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.473 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.473 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.473 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.473 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.473 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.473 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.473 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.473 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.473 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.473 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.474 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.474 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.474 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.474 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.474 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.474 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.474 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.474 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.474 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.474 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.474 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.474 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.474 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.474 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.475 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.475 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.475 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.475 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.475 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.475 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.475 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.475 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.475 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.475 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.475 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.475 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.475 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.476 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.476 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.476 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.476 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.476 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.476 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.476 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.476 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.476 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.476 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.476 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.476 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.476 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.477 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.477 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.477 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.477 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.477 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.477 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.477 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "k8sevents", 2024-04-25T11:37:41.477 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.477 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.477 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.477 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_event_retention_days": { 2024-04-25T11:37:41.477 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ceph_event_retention_days", 2024-04-25T11:37:41.477 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.477 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.477 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.478 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "7", 2024-04-25T11:37:41.478 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.478 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.478 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.478 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T11:37:41.478 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.478 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.478 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.478 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.478 INFO:teuthology.orchestra.run.smithi069.stdout: "config_check_secs": { 2024-04-25T11:37:41.478 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_check_secs", 2024-04-25T11:37:41.478 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.478 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.478 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.479 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-25T11:37:41.479 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "10", 2024-04-25T11:37:41.479 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.479 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.479 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T11:37:41.479 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.479 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.479 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.479 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.479 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.479 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.479 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.479 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.479 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.480 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.480 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.480 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.480 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.480 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.480 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.480 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.480 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.480 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.480 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.480 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.480 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.480 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.480 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.480 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.480 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.481 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.481 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.481 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.481 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.481 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.481 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.481 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.481 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.481 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.481 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.481 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.481 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.481 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.481 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.482 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.482 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.482 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.482 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.482 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.482 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.482 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.482 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.482 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.482 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.482 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.482 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.482 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.482 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.482 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.483 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.483 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.483 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.483 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.483 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.483 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.483 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.483 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.483 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.483 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.483 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.483 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.483 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.483 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.483 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.484 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.484 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.484 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.484 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.484 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.484 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.484 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.484 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.484 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.484 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.484 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.484 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.484 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.484 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.485 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.485 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.485 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.485 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.485 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.485 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.485 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.485 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.485 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "localpool", 2024-04-25T11:37:41.485 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.485 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.485 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.485 INFO:teuthology.orchestra.run.smithi069.stdout: "failure_domain": { 2024-04-25T11:37:41.485 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "failure_domain", 2024-04-25T11:37:41.485 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.485 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.486 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.486 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "host", 2024-04-25T11:37:41.486 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.486 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.486 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.486 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "failure domain for any created local pool", 2024-04-25T11:37:41.486 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T11:37:41.486 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.486 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.486 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.486 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.486 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.486 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.487 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.487 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.487 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.487 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.487 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.487 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.487 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.487 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.487 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.487 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.487 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.487 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.487 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.487 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.488 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.488 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.488 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.488 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.488 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.488 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.488 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.488 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.488 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.488 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.488 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.488 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.488 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.488 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.489 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.489 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.489 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.489 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.489 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.489 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.489 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.489 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.489 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.489 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.489 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.489 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.490 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.490 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.490 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.490 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.490 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.490 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.490 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.490 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.490 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.490 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.490 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.490 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.490 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.491 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.491 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.491 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.491 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.491 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.491 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.491 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.491 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.491 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.491 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.491 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.491 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.491 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.492 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.492 INFO:teuthology.orchestra.run.smithi069.stdout: "min_size": { 2024-04-25T11:37:41.492 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "min_size", 2024-04-25T11:37:41.492 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.492 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.492 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.492 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.492 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.492 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.492 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.492 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default min_size for any created local pool", 2024-04-25T11:37:41.492 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T11:37:41.492 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.492 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.493 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.493 INFO:teuthology.orchestra.run.smithi069.stdout: "num_rep": { 2024-04-25T11:37:41.493 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "num_rep", 2024-04-25T11:37:41.493 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.493 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.493 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.493 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3", 2024-04-25T11:37:41.493 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.493 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.493 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.493 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default replica count for any created local pool", 2024-04-25T11:37:41.493 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.493 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.493 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.493 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.494 INFO:teuthology.orchestra.run.smithi069.stdout: "pg_num": { 2024-04-25T11:37:41.494 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pg_num", 2024-04-25T11:37:41.494 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.494 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.494 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.494 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "128", 2024-04-25T11:37:41.494 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.494 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.494 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.494 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T11:37:41.494 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.494 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.494 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.495 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.495 INFO:teuthology.orchestra.run.smithi069.stdout: "prefix": { 2024-04-25T11:37:41.495 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prefix", 2024-04-25T11:37:41.495 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.495 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.495 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.495 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.495 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.495 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.495 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.495 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "name prefix for any created local pool", 2024-04-25T11:37:41.495 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.495 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.495 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.495 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.496 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.496 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.496 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.496 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.496 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.496 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.496 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.496 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.496 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.496 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.496 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.496 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.496 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.496 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.497 INFO:teuthology.orchestra.run.smithi069.stdout: "subtree": { 2024-04-25T11:37:41.497 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "subtree", 2024-04-25T11:37:41.497 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.497 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.497 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.497 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "rack", 2024-04-25T11:37:41.497 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.497 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.497 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.497 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T11:37:41.497 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T11:37:41.497 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.497 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.497 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.498 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.498 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.498 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.498 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mds_autoscaler", 2024-04-25T11:37:41.498 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.498 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.498 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.498 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.498 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.498 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.498 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.498 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.498 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.498 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.499 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.499 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.499 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.499 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.499 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.499 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.499 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.499 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.499 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.499 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.499 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.499 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.499 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.499 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.500 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.500 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.500 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.500 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.500 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.500 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.500 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.500 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.500 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.500 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.500 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.500 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.500 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.500 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.501 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.501 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.501 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.501 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.501 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.501 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.501 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.501 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.501 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.501 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.501 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.501 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.501 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.501 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.501 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.502 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.502 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.502 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.502 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.502 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.502 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.502 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.502 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.502 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.502 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.502 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.502 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.502 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.502 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.503 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.503 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.503 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.503 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.503 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.503 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.503 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.503 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.503 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.503 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.503 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.503 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.503 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.503 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.503 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.504 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.504 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.504 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.504 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.504 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.504 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.504 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.504 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.504 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mirroring", 2024-04-25T11:37:41.504 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.504 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.504 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.504 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.504 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.504 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.505 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.505 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.505 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.505 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.505 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.505 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.505 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.505 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.505 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.505 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.505 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.505 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.505 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.505 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.505 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.506 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.506 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.506 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.506 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.506 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.506 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.506 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.506 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.506 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.506 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.506 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.506 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.506 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.507 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.507 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.507 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.507 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.507 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.507 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.507 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.507 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.507 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.507 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.507 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.507 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.507 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.507 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.508 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.508 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.508 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.508 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.508 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.508 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.508 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.508 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.508 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.508 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.508 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.508 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.508 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.508 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.508 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.509 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.509 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.509 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.509 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.509 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.509 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.509 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.509 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.509 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.509 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.509 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.509 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.509 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.510 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.510 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.510 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.510 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.510 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.510 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.510 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.510 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.510 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.510 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.510 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.510 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.510 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.510 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.511 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "nfs", 2024-04-25T11:37:41.511 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.511 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.511 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.511 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.511 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.511 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.511 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.511 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.511 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.511 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.511 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.511 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.512 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.512 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.512 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.512 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.512 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.512 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.512 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.512 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.512 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.512 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.512 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.512 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.512 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.512 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.513 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.513 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.513 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.513 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.513 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.513 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.513 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.513 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.513 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.513 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.513 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.513 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.513 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.514 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.514 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.514 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.514 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.514 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.514 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.514 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.514 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.514 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.514 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.514 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.514 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.514 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.514 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.515 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.515 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.515 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.515 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.515 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.515 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.515 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.515 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.515 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.515 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.515 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.515 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.515 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.515 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.516 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.516 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.516 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.516 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.516 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.516 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.516 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.516 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.516 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.516 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.516 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.516 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.516 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.516 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.517 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.517 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.517 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.517 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.517 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.517 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.517 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.517 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.517 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.517 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "orchestrator", 2024-04-25T11:37:41.517 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.517 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.517 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.517 INFO:teuthology.orchestra.run.smithi069.stdout: "fail_fs": { 2024-04-25T11:37:41.517 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "fail_fs", 2024-04-25T11:37:41.518 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.518 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.518 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.518 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.518 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.518 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.518 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.518 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T11:37:41.518 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.518 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.518 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.518 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.518 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.519 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.519 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.519 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.519 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.519 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.519 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.519 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.519 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.519 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.519 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.519 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.519 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.519 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.520 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.520 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.520 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.520 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.520 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.520 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.520 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.520 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.520 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.520 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.520 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.520 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.520 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.520 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.520 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.521 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.521 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.521 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.521 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.521 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.521 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.521 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.521 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.521 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.521 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.521 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.521 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.521 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.521 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.521 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.522 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.522 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.522 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.522 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.522 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.522 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.522 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.522 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.522 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.522 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.522 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.522 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.522 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.522 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.523 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.523 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.523 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.523 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.523 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.523 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.523 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.523 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.523 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.523 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.523 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.523 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.523 INFO:teuthology.orchestra.run.smithi069.stdout: "orchestrator": { 2024-04-25T11:37:41.523 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "orchestrator", 2024-04-25T11:37:41.523 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.524 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.524 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.524 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.524 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.524 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.524 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.524 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm", 2024-04-25T11:37:41.524 INFO:teuthology.orchestra.run.smithi069.stdout: "rook", 2024-04-25T11:37:41.524 INFO:teuthology.orchestra.run.smithi069.stdout: "test_orchestrator" 2024-04-25T11:37:41.524 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.524 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Orchestrator backend", 2024-04-25T11:37:41.524 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.524 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.524 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.524 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.525 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.525 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.525 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.525 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.525 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.525 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.525 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.525 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.525 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.525 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.525 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.525 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.525 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.525 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.526 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.526 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.526 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.526 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "osd_perf_query", 2024-04-25T11:37:41.526 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.526 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.526 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.526 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.526 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.526 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.526 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.526 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.526 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.526 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.527 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.527 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.527 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.527 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.527 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.527 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.527 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.527 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.527 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.527 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.527 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.527 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.527 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.528 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.528 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.528 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.528 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.528 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.528 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.528 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.528 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.528 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.528 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.528 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.528 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.528 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.528 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.528 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.529 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.529 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.529 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.529 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.529 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.529 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.529 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.529 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.529 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.529 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.529 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.529 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.529 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.529 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.530 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.530 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.530 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.530 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.530 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.530 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.530 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.530 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.530 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.530 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.530 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.530 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.530 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.531 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.531 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.531 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.531 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.531 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.531 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.531 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.531 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.531 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.531 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.531 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.531 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.531 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.532 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.532 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.532 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.532 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.532 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.532 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.532 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.532 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.532 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.532 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.532 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.532 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.532 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "osd_support", 2024-04-25T11:37:41.532 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.532 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.533 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.533 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.533 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.533 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.533 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.533 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.533 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.533 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.533 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.533 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.533 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.533 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.533 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.534 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.534 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.534 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.534 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.534 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.534 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.534 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.534 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.534 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.534 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.534 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.534 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.534 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.535 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.535 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.535 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.535 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.535 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.535 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.535 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.535 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.535 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.535 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.535 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.535 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.535 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.535 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.535 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.536 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.536 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.536 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.536 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.536 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.536 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.536 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.536 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.536 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.536 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.536 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.536 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.536 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.536 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.537 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.537 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.537 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.537 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.537 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.537 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.537 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.537 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.537 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.537 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.537 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.537 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.537 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.537 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.537 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.538 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.538 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.538 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.538 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.538 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.538 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.538 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.538 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.538 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.538 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.538 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.538 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.539 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.539 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.539 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.539 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.539 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.539 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.539 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pg_autoscaler", 2024-04-25T11:37:41.539 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.539 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.539 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.539 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.539 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.539 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.540 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.540 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.540 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.540 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.540 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.540 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.540 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.540 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.540 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.540 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.540 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.540 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.540 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.540 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.541 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.541 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.541 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.541 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.541 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.541 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.541 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.541 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.541 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.541 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.541 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.541 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.541 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.541 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.542 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.542 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.542 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.542 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.542 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.542 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.542 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.542 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.542 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.542 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.542 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.543 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.543 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.543 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.543 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.543 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.543 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.543 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.543 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.543 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.543 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.543 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.543 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.543 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.543 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.544 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.544 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.544 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.544 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.544 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.544 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.544 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.544 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.544 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.544 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.544 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.544 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.544 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.544 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.545 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-04-25T11:37:41.545 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-04-25T11:37:41.545 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:41.545 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.545 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.545 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-04-25T11:37:41.545 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.545 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.545 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.545 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.545 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.545 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.545 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.545 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.545 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.546 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.546 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.546 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.546 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.546 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.546 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.546 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.546 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.546 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.546 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.546 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.546 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.546 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.546 INFO:teuthology.orchestra.run.smithi069.stdout: "threshold": { 2024-04-25T11:37:41.546 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "threshold", 2024-04-25T11:37:41.546 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-04-25T11:37:41.547 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.547 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.547 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3.0", 2024-04-25T11:37:41.547 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1.0", 2024-04-25T11:37:41.547 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.547 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.547 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "scaling threshold", 2024-04-25T11:37:41.547 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T11:37:41.547 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.547 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.547 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.547 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.547 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.547 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.547 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "progress", 2024-04-25T11:37:41.547 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.548 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.548 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.548 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_pg_recovery_event": { 2024-04-25T11:37:41.548 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_pg_recovery_event", 2024-04-25T11:37:41.548 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.548 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.548 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.548 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.548 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.548 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.548 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.548 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T11:37:41.548 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.548 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.548 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.548 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.549 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": { 2024-04-25T11:37:41.549 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enabled", 2024-04-25T11:37:41.549 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.549 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.549 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.549 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.549 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.549 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.549 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.549 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.549 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.549 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.549 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.549 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.549 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.549 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.550 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.550 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.550 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.550 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.550 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.550 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.550 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.550 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.550 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.550 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.550 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.550 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.550 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.550 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.551 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.551 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.551 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.551 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.551 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.551 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.551 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.551 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.551 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.551 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.551 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.551 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.551 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.551 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.551 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.551 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.551 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.552 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.552 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.552 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.552 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.552 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.552 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.552 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.552 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.552 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.552 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.552 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.552 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.552 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.552 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.552 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.552 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.553 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.553 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.553 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.553 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.553 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.553 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.553 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.553 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.553 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.553 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.553 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.553 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.553 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.553 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.553 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.554 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.554 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.554 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.554 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.554 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.554 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.554 INFO:teuthology.orchestra.run.smithi069.stdout: "max_completed_events": { 2024-04-25T11:37:41.554 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_completed_events", 2024-04-25T11:37:41.554 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.554 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.554 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.554 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "50", 2024-04-25T11:37:41.554 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.554 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.554 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.555 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "number of past completed events to remember", 2024-04-25T11:37:41.555 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.555 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.555 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.555 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.555 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-04-25T11:37:41.555 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-04-25T11:37:41.555 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:41.555 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.555 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.555 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-04-25T11:37:41.555 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.555 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.555 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.556 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long the module is going to sleep", 2024-04-25T11:37:41.556 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.556 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.556 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.556 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.556 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.556 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.556 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.556 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.556 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.556 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.556 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.556 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.556 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.556 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.557 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.557 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.557 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.557 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.557 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.557 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.557 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.557 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prometheus", 2024-04-25T11:37:41.557 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.557 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.557 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.557 INFO:teuthology.orchestra.run.smithi069.stdout: "cache": { 2024-04-25T11:37:41.557 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cache", 2024-04-25T11:37:41.557 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.558 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.558 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.558 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.558 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.558 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.558 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.558 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.558 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.558 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.558 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.558 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.558 INFO:teuthology.orchestra.run.smithi069.stdout: "exclude_perf_counters": { 2024-04-25T11:37:41.558 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "exclude_perf_counters", 2024-04-25T11:37:41.558 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.558 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.559 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.559 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.559 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.559 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.559 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.559 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T11:37:41.559 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T11:37:41.559 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.559 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.559 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.559 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.559 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.559 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.560 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.560 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.560 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.560 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.560 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.560 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.560 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.560 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.560 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.560 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.560 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.560 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.560 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.561 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.561 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.561 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.561 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.561 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.561 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.561 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.561 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.561 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.561 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.561 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.561 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.561 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.561 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.561 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.562 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.562 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.562 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.562 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.562 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.562 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.562 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.562 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.562 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.562 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.562 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.562 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.562 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.562 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.563 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.563 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.563 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.563 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.563 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.563 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.563 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.563 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.563 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.563 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.563 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.563 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.563 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.563 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.563 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.564 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.564 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.564 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.564 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.564 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.564 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.564 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.564 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.564 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.564 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.564 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_stats_pools": { 2024-04-25T11:37:41.564 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_stats_pools", 2024-04-25T11:37:41.564 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.564 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.565 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.565 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.565 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.565 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.565 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.565 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.565 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.565 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.565 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.565 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.565 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T11:37:41.565 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T11:37:41.565 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.565 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.566 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.566 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "300", 2024-04-25T11:37:41.566 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.566 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.566 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.566 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.566 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.566 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.566 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.566 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.566 INFO:teuthology.orchestra.run.smithi069.stdout: "scrape_interval": { 2024-04-25T11:37:41.566 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "scrape_interval", 2024-04-25T11:37:41.566 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-04-25T11:37:41.566 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.567 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.567 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15.0", 2024-04-25T11:37:41.567 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.567 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.567 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.567 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.567 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.567 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.567 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.567 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.567 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2024-04-25T11:37:41.567 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2024-04-25T11:37:41.567 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.567 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.567 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.568 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "::", 2024-04-25T11:37:41.568 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.568 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.568 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.568 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T11:37:41.568 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.568 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.568 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.568 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.568 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2024-04-25T11:37:41.568 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2024-04-25T11:37:41.568 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.568 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.569 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.569 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "9283", 2024-04-25T11:37:41.569 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.569 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.569 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.569 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T11:37:41.569 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.569 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.569 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.569 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.569 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.569 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.569 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.570 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.570 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.570 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.570 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.570 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.570 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.570 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.570 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.570 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.570 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.570 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.570 INFO:teuthology.orchestra.run.smithi069.stdout: "stale_cache_strategy": { 2024-04-25T11:37:41.570 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "stale_cache_strategy", 2024-04-25T11:37:41.570 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.571 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.571 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.571 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "log", 2024-04-25T11:37:41.571 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.571 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.571 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.571 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.571 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.571 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.571 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.571 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.571 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_behaviour": { 2024-04-25T11:37:41.571 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_behaviour", 2024-04-25T11:37:41.571 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.572 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.572 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.572 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "default", 2024-04-25T11:37:41.572 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.572 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.572 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.572 INFO:teuthology.orchestra.run.smithi069.stdout: "default", 2024-04-25T11:37:41.572 INFO:teuthology.orchestra.run.smithi069.stdout: "error" 2024-04-25T11:37:41.572 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.572 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.572 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.572 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.572 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.572 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.572 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_error_status_code": { 2024-04-25T11:37:41.573 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_error_status_code", 2024-04-25T11:37:41.573 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.573 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.573 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.573 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "500", 2024-04-25T11:37:41.573 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "400", 2024-04-25T11:37:41.573 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "599", 2024-04-25T11:37:41.573 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.573 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.573 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.573 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.573 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.573 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.573 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.574 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.574 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.574 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_support", 2024-04-25T11:37:41.574 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.574 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.574 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.574 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.574 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.574 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.574 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.574 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.574 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.574 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.574 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.575 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.575 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.575 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.575 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.575 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.575 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.575 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.575 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.575 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.575 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.575 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.575 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.575 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.575 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.576 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.576 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.576 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.576 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.576 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.576 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.576 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.576 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.576 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.576 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.576 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.576 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.576 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.576 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.577 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.577 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.577 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.577 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.577 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.577 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.577 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.577 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.577 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.577 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.577 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.577 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.577 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.577 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.578 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.578 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.578 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.578 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.578 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.578 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.578 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.578 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.578 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.578 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.578 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.578 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.578 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.579 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.579 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.579 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.579 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.579 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.579 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.579 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.579 INFO:teuthology.orchestra.run.smithi069.stdout: "max_concurrent_snap_create": { 2024-04-25T11:37:41.579 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_concurrent_snap_create", 2024-04-25T11:37:41.579 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.579 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.579 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.579 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-25T11:37:41.580 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.580 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.580 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.580 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.580 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.580 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.580 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.580 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.580 INFO:teuthology.orchestra.run.smithi069.stdout: "mirror_snapshot_schedule": { 2024-04-25T11:37:41.580 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T11:37:41.580 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.580 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.580 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.581 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.581 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.581 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.581 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.581 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.581 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.581 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.581 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.581 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.581 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.581 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.581 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.581 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.581 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.582 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.582 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.582 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.582 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.582 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.582 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.582 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.582 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.582 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.582 INFO:teuthology.orchestra.run.smithi069.stdout: "trash_purge_schedule": { 2024-04-25T11:37:41.582 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "trash_purge_schedule", 2024-04-25T11:37:41.582 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.582 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.582 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.583 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.583 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.583 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.583 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.583 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.583 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.583 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.583 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.583 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.583 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.583 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.583 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.583 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "restful", 2024-04-25T11:37:41.583 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.584 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.584 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.584 INFO:teuthology.orchestra.run.smithi069.stdout: "enable_auth": { 2024-04-25T11:37:41.584 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enable_auth", 2024-04-25T11:37:41.584 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.584 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.584 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.584 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.584 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.584 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.584 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.584 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.584 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.584 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.585 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.585 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.585 INFO:teuthology.orchestra.run.smithi069.stdout: "key_file": { 2024-04-25T11:37:41.585 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "key_file", 2024-04-25T11:37:41.585 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.585 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.585 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.585 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.585 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.585 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.585 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.585 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.585 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.585 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.585 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.586 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.586 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.586 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.586 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.586 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.586 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.586 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.586 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.586 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.586 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.586 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.586 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.586 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.586 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.587 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.587 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.587 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.587 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.587 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.587 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.587 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.587 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.587 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.587 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.587 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.587 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.587 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.587 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.587 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.588 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.588 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.588 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.588 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.588 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.588 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.588 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.588 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.588 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.588 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.588 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.588 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.588 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.588 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.589 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.589 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.589 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.589 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.589 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.589 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.589 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.589 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.589 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.589 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.589 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.589 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.589 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.589 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.590 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.590 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.590 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.590 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.590 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.590 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.590 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.590 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.590 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.590 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.590 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.590 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.590 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.590 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.591 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2024-04-25T11:37:41.591 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2024-04-25T11:37:41.591 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.591 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.591 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.591 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.591 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.591 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.591 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.591 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.591 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.591 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.591 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.591 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.592 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2024-04-25T11:37:41.592 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2024-04-25T11:37:41.592 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.592 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.592 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.592 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.592 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.592 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.592 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.592 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.592 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.592 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.592 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.593 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.593 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.593 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.593 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.593 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.593 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.593 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.593 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.593 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.593 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.593 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.593 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.593 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.593 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.593 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.594 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.594 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.594 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.594 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rgw", 2024-04-25T11:37:41.594 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.594 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.594 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.594 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.594 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.594 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.594 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.594 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.594 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.595 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.595 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.595 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.595 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.595 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.595 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.595 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.595 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.595 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.595 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.595 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.595 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.595 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.595 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.596 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.596 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.596 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.596 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.596 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.596 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.596 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.596 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.596 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.596 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.596 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.596 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.596 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.597 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.597 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.597 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.597 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.597 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.597 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.597 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.597 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.597 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.597 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.597 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.597 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.597 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.597 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.597 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.597 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.598 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.598 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.598 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.598 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.598 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.598 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.598 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.598 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.598 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.598 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.598 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.598 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.598 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.598 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.599 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.599 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.599 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.599 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.599 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.599 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.599 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.599 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.599 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.599 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.599 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.599 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.599 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.599 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.600 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.600 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.600 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.600 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.600 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.600 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.600 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.600 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.600 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.600 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.600 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rook", 2024-04-25T11:37:41.600 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.600 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.601 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.601 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.601 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.601 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.601 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.601 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.601 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.601 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.601 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.601 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.601 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.601 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.601 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.601 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.602 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.602 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.602 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.602 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.602 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.602 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.602 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.602 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.602 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.602 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.602 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.602 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.602 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.602 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.603 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.603 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.603 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.603 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.603 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.603 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.603 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.603 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.603 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.603 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.603 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.603 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.604 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.604 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.604 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.604 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.604 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.604 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.604 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.604 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.604 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.604 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.604 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.604 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.604 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.604 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.605 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.605 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.605 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.605 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.605 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.605 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.605 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.605 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.605 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.605 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.605 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.605 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.605 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.605 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.606 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.606 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.606 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.606 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.606 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.606 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.606 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.606 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.606 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.606 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.606 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.606 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.606 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.606 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.607 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.607 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.607 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.607 INFO:teuthology.orchestra.run.smithi069.stdout: "storage_class": { 2024-04-25T11:37:41.607 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "storage_class", 2024-04-25T11:37:41.607 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.607 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.607 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.607 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "local", 2024-04-25T11:37:41.607 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.607 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.607 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.607 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T11:37:41.607 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.608 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.608 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.608 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.608 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.608 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.608 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.608 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "selftest", 2024-04-25T11:37:41.608 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.608 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.608 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.608 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.608 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.608 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.608 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.609 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.609 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.609 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.609 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.609 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.609 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.609 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.609 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.609 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.609 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.609 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.609 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.609 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.609 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.610 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.610 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.610 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.610 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.610 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.610 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.610 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.610 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.610 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.610 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.610 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.610 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.610 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.611 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.611 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.611 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.611 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.611 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.611 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.611 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.611 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.611 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.611 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.611 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.611 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.611 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.611 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.611 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.612 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.612 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.612 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.612 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.612 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.612 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.612 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.612 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.612 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.612 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.612 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.612 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.612 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.612 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.613 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.613 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.613 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.613 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.613 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.613 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.613 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.613 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.613 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.613 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.613 INFO:teuthology.orchestra.run.smithi069.stdout: "roption1": { 2024-04-25T11:37:41.613 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "roption1", 2024-04-25T11:37:41.613 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.613 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.614 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.614 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.614 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.614 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.614 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.614 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.614 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.614 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.614 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.614 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.614 INFO:teuthology.orchestra.run.smithi069.stdout: "roption2": { 2024-04-25T11:37:41.614 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "roption2", 2024-04-25T11:37:41.614 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.614 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.615 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.615 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "xyz", 2024-04-25T11:37:41.615 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.615 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.615 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.615 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.615 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.615 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.615 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.615 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.615 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption1": { 2024-04-25T11:37:41.615 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption1", 2024-04-25T11:37:41.615 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.615 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.615 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.616 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.616 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.616 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.616 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.616 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.616 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.616 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.616 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.616 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.616 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption2": { 2024-04-25T11:37:41.616 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption2", 2024-04-25T11:37:41.616 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.616 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.616 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.616 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.617 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.617 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.617 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.617 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.617 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.617 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.617 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.617 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.617 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption3": { 2024-04-25T11:37:41.617 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption3", 2024-04-25T11:37:41.617 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-04-25T11:37:41.617 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.617 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.617 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.617 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.618 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.618 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.618 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.618 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.618 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.618 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.618 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.618 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption4": { 2024-04-25T11:37:41.618 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption4", 2024-04-25T11:37:41.618 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.618 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.618 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.618 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.618 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.619 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.619 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.619 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.619 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.619 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.619 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.619 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.619 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption5": { 2024-04-25T11:37:41.619 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption5", 2024-04-25T11:37:41.619 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.619 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.619 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.619 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.619 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.620 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.620 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.620 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.620 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.620 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.620 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.620 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.620 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption6": { 2024-04-25T11:37:41.620 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption6", 2024-04-25T11:37:41.620 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.620 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.620 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.620 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.621 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.621 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.621 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.621 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.621 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.621 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.621 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.621 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.621 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption7": { 2024-04-25T11:37:41.621 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption7", 2024-04-25T11:37:41.621 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.621 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.621 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.621 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.622 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-04-25T11:37:41.622 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "42", 2024-04-25T11:37:41.622 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.622 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.622 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.622 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.622 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.622 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.622 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.622 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.622 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.622 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.622 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.622 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.623 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.623 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.623 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.623 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.623 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.623 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.623 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.623 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.623 INFO:teuthology.orchestra.run.smithi069.stdout: "testkey": { 2024-04-25T11:37:41.623 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testkey", 2024-04-25T11:37:41.623 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.623 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.623 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.623 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.623 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.624 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.624 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.624 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.624 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.624 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.624 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.624 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.624 INFO:teuthology.orchestra.run.smithi069.stdout: "testlkey": { 2024-04-25T11:37:41.624 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testlkey", 2024-04-25T11:37:41.624 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.624 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.624 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.624 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.624 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.624 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.625 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.625 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.625 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.625 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.625 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.625 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.625 INFO:teuthology.orchestra.run.smithi069.stdout: "testnewline": { 2024-04-25T11:37:41.625 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testnewline", 2024-04-25T11:37:41.625 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.625 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.625 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.625 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.625 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.626 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.626 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.626 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.626 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.626 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.626 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.626 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.626 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.626 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.626 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.626 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snap_schedule", 2024-04-25T11:37:41.626 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.626 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.626 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.627 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_m_granularity": { 2024-04-25T11:37:41.627 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_m_granularity", 2024-04-25T11:37:41.627 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.627 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.627 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.627 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.627 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.627 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.627 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.627 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T11:37:41.627 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.627 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.627 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.628 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.628 INFO:teuthology.orchestra.run.smithi069.stdout: "dump_on_update": { 2024-04-25T11:37:41.628 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "dump_on_update", 2024-04-25T11:37:41.628 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.628 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.628 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.628 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.628 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.628 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.628 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.628 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "dump database to debug log on update", 2024-04-25T11:37:41.628 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.628 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.628 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.628 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.629 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.629 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.629 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.629 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.629 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.629 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.629 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.629 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.629 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.629 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.629 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.629 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.629 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.629 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.630 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.630 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.630 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.630 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.630 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.630 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.630 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.630 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.630 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.630 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.630 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.630 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.630 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.630 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.631 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.631 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.631 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.631 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.631 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.631 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.631 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.631 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.631 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.631 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.631 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.631 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.631 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.631 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.631 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.631 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.632 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.632 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.632 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.632 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.632 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.632 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.632 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.632 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.632 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.632 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.632 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.632 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.632 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.632 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.632 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.633 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.633 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.633 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.633 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.633 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.633 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.633 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.633 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.633 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.633 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.633 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.633 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.633 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.633 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.634 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.634 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.634 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.634 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.634 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.634 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.634 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.634 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.634 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.634 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.634 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.634 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.634 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.635 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.635 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "stats", 2024-04-25T11:37:41.635 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.635 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.635 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.635 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.635 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.635 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.635 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.635 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.635 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.635 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.635 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.635 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.636 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.636 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.636 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.636 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.636 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.636 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.636 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.636 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.636 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.636 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.636 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.636 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.636 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.636 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.637 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.637 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.637 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.637 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.637 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.637 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.637 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.637 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.637 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.637 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.637 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.637 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.637 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.637 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.638 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.638 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.638 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.638 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.638 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.638 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.638 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.638 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.638 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.638 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.638 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.638 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.638 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.638 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.639 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.639 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.639 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.639 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.639 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.639 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.639 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.639 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.639 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.639 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.639 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.639 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.639 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.640 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.640 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.640 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.640 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.640 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.640 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.640 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.640 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.640 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.640 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.640 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.640 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.640 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.640 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.641 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.641 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.641 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.641 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.641 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.641 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.641 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.641 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.641 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.641 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "status", 2024-04-25T11:37:41.641 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.641 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.641 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.641 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.641 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.641 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.642 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.642 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.642 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.642 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.642 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.642 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.642 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.642 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.642 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.642 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.642 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.642 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.642 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.643 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.643 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.643 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.643 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.643 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.643 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.643 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.643 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.643 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.643 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.643 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.643 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.643 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.643 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.644 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.644 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.644 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.644 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.644 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.644 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.644 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.644 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.644 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.644 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.644 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.644 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.644 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.644 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.644 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.645 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.645 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.645 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.645 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.645 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.645 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.645 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.645 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.645 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.645 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.645 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.645 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.645 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.645 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.646 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.646 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.646 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.646 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.646 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.646 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.646 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.646 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.646 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.646 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.646 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.646 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.646 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.647 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.647 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.647 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.647 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.647 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.647 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.647 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.647 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.647 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.647 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.647 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.647 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.647 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.647 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.647 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.648 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "telegraf", 2024-04-25T11:37:41.648 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.648 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.648 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.648 INFO:teuthology.orchestra.run.smithi069.stdout: "address": { 2024-04-25T11:37:41.648 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "address", 2024-04-25T11:37:41.648 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.648 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.648 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.648 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T11:37:41.648 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.648 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.648 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.649 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.649 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.649 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.649 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.649 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.649 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-04-25T11:37:41.649 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-04-25T11:37:41.649 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:41.649 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.649 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.649 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15", 2024-04-25T11:37:41.649 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.649 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.649 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.650 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.650 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.650 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.650 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.650 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.650 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.650 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.650 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.650 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.650 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.650 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.650 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.650 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.650 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.651 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.651 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.651 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.651 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.651 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.651 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.651 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.651 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.651 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.651 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.651 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.651 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.651 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.651 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.651 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.652 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.652 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.652 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.652 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.652 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.652 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.652 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.652 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.652 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.652 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.652 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.652 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.652 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.653 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.653 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.653 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.653 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.653 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.653 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.653 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.653 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.653 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.653 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.653 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.653 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.653 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.653 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.653 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.653 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.654 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.654 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.654 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.654 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.654 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.654 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.654 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.654 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.654 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.654 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.654 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.654 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.655 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.655 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.655 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.655 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.655 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.655 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.655 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.655 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.655 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.655 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.655 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.655 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.655 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.655 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.655 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.656 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.656 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.656 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.656 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.656 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.656 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.656 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.656 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "telemetry", 2024-04-25T11:37:41.656 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.656 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.656 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.656 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_basic": { 2024-04-25T11:37:41.656 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_basic", 2024-04-25T11:37:41.656 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.657 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.657 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.657 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.657 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.657 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.657 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.657 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T11:37:41.657 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.657 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.657 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.657 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.657 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_crash": { 2024-04-25T11:37:41.657 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_crash", 2024-04-25T11:37:41.657 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.657 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.658 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.658 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.658 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.658 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.658 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.658 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T11:37:41.658 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.658 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.658 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.658 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.658 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_device": { 2024-04-25T11:37:41.658 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_device", 2024-04-25T11:37:41.658 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.658 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.659 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.659 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.659 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.659 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.659 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.659 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T11:37:41.659 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.659 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.659 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.659 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.659 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_ident": { 2024-04-25T11:37:41.659 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_ident", 2024-04-25T11:37:41.659 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.659 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.659 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.659 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.660 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.660 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.660 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.660 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T11:37:41.660 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.660 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.660 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.660 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.660 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_perf": { 2024-04-25T11:37:41.660 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_perf", 2024-04-25T11:37:41.660 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.660 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.660 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.660 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.660 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.661 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.661 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.661 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T11:37:41.661 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.661 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.661 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.661 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.661 INFO:teuthology.orchestra.run.smithi069.stdout: "contact": { 2024-04-25T11:37:41.661 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "contact", 2024-04-25T11:37:41.661 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.661 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.661 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.661 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.661 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.662 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.662 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.662 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.662 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.662 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.662 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.662 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.662 INFO:teuthology.orchestra.run.smithi069.stdout: "description": { 2024-04-25T11:37:41.662 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "description", 2024-04-25T11:37:41.662 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.662 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.662 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.662 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.663 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.663 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.663 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.663 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.663 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.663 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.663 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.663 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.663 INFO:teuthology.orchestra.run.smithi069.stdout: "device_url": { 2024-04-25T11:37:41.663 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_url", 2024-04-25T11:37:41.663 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.663 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.664 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.664 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T11:37:41.664 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.664 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.664 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.664 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.664 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.664 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.664 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.664 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.664 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": { 2024-04-25T11:37:41.664 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enabled", 2024-04-25T11:37:41.664 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.664 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.664 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.665 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.665 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.665 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.665 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.665 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.665 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.665 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.665 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.665 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.665 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-04-25T11:37:41.665 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-04-25T11:37:41.665 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.665 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.665 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.666 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "24", 2024-04-25T11:37:41.666 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "8", 2024-04-25T11:37:41.666 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.666 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.666 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.666 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.666 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.666 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.666 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.666 INFO:teuthology.orchestra.run.smithi069.stdout: "last_opt_revision": { 2024-04-25T11:37:41.666 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "last_opt_revision", 2024-04-25T11:37:41.666 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.666 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.667 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.667 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1", 2024-04-25T11:37:41.667 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.667 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.667 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.667 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.667 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.667 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.667 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.667 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.667 INFO:teuthology.orchestra.run.smithi069.stdout: "leaderboard": { 2024-04-25T11:37:41.667 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "leaderboard", 2024-04-25T11:37:41.667 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.667 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.668 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.668 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.668 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.668 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.668 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.668 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.668 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.668 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.668 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.668 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.668 INFO:teuthology.orchestra.run.smithi069.stdout: "leaderboard_description": { 2024-04-25T11:37:41.668 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "leaderboard_description", 2024-04-25T11:37:41.668 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.668 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.668 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.669 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.669 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.669 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.669 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.669 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.669 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.669 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.669 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.669 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.669 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.669 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.669 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.669 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.669 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.670 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.670 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.670 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.670 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.670 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.670 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.670 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.670 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.670 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.670 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.670 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.670 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.670 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.670 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.671 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.671 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.671 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.671 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.671 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.671 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.671 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.671 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.671 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.671 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.671 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.671 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.671 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.671 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.671 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.672 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.672 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.672 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.672 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.672 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.672 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.672 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.672 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.672 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.672 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.672 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.672 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.672 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.672 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.672 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.673 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.673 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.673 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.673 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.673 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.673 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.673 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.673 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.673 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.673 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.673 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.673 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.673 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.673 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.674 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.674 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.674 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.674 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.674 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.674 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.674 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.674 INFO:teuthology.orchestra.run.smithi069.stdout: "organization": { 2024-04-25T11:37:41.674 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "organization", 2024-04-25T11:37:41.674 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.674 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.674 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.674 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.674 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.675 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.675 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.675 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.675 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.675 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.675 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.675 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.675 INFO:teuthology.orchestra.run.smithi069.stdout: "proxy": { 2024-04-25T11:37:41.675 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "proxy", 2024-04-25T11:37:41.675 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.675 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.675 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.675 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.675 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.676 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.676 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.676 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.676 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.676 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.676 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.676 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.676 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.676 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.676 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.676 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.676 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.676 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.676 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.676 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.677 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.677 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.677 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.677 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.677 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.677 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.677 INFO:teuthology.orchestra.run.smithi069.stdout: "url": { 2024-04-25T11:37:41.677 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "url", 2024-04-25T11:37:41.677 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.677 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.677 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.677 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T11:37:41.677 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.677 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.677 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.678 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.678 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.678 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.678 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.678 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.678 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.678 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.678 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.678 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "test_orchestrator", 2024-04-25T11:37:41.678 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.678 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.678 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.678 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.679 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.679 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.679 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.679 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.679 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.679 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.679 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.679 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.679 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.679 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.679 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.679 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.679 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.679 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.680 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.680 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.680 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.680 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.680 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.680 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.680 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.680 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.680 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.680 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.680 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.680 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.680 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.680 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.680 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.681 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.681 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.681 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.681 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.681 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.681 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.681 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.681 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.681 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.681 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.681 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.681 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.681 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.681 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.682 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.682 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.682 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.682 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.682 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.682 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.682 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.682 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.682 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.682 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.682 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.682 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.682 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.682 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.682 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.683 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.683 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.683 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.683 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.683 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.683 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.683 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.683 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.683 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.683 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.683 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.683 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.683 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.683 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.685 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.685 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.685 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.685 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.685 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.685 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.685 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.685 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.685 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.685 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.685 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.686 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.686 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.686 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.686 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "volumes", 2024-04-25T11:37:41.686 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.686 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.686 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.686 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.686 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.686 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.686 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.686 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.686 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.686 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.687 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.687 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.687 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.687 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.687 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.687 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.687 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.687 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.687 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.687 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.687 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.687 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.687 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.687 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.688 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.688 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.688 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.688 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.688 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.688 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.688 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.688 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.688 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.688 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.688 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.688 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.688 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.688 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.689 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.689 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.689 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.689 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.689 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.689 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.689 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.689 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.689 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.689 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.689 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.689 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.689 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.689 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.690 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.690 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.690 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.690 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.690 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.690 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.690 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.690 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.690 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.690 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.690 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.690 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.690 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.691 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.691 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.691 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.691 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.691 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.691 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.691 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.691 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.691 INFO:teuthology.orchestra.run.smithi069.stdout: "max_concurrent_clones": { 2024-04-25T11:37:41.691 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_concurrent_clones", 2024-04-25T11:37:41.691 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.691 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.691 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.691 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "4", 2024-04-25T11:37:41.691 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.691 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.692 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.692 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T11:37:41.692 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.692 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.692 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.692 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.692 INFO:teuthology.orchestra.run.smithi069.stdout: "periodic_async_work": { 2024-04-25T11:37:41.692 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "periodic_async_work", 2024-04-25T11:37:41.692 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.692 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.692 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.692 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.692 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.693 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.693 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.693 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Periodically check for async work", 2024-04-25T11:37:41.693 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.693 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.693 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.693 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.693 INFO:teuthology.orchestra.run.smithi069.stdout: "snapshot_clone_delay": { 2024-04-25T11:37:41.693 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snapshot_clone_delay", 2024-04-25T11:37:41.693 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.693 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.693 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.693 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.693 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.694 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.694 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.694 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T11:37:41.694 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.694 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.694 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.694 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.694 INFO:teuthology.orchestra.run.smithi069.stdout: "snapshot_clone_no_wait": { 2024-04-25T11:37:41.694 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T11:37:41.694 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.694 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.694 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.694 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.695 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.695 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.695 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.695 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T11:37:41.695 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.695 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.695 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.695 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.695 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.695 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.695 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.695 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.695 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.695 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.695 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.696 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.696 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.696 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.696 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.696 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.696 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.696 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.696 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.696 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.696 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.696 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix", 2024-04-25T11:37:41.696 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.696 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.696 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.696 INFO:teuthology.orchestra.run.smithi069.stdout: "discovery_interval": { 2024-04-25T11:37:41.697 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "discovery_interval", 2024-04-25T11:37:41.697 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-04-25T11:37:41.697 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.697 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.697 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "100", 2024-04-25T11:37:41.697 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.697 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.697 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.697 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.697 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.697 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.697 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.697 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.697 INFO:teuthology.orchestra.run.smithi069.stdout: "identifier": { 2024-04-25T11:37:41.697 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "identifier", 2024-04-25T11:37:41.698 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.698 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.698 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.698 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.698 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.698 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.698 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.698 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.698 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.698 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.698 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.698 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.698 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-04-25T11:37:41.698 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-04-25T11:37:41.699 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:41.699 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.699 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.699 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-04-25T11:37:41.699 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.699 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.699 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.699 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.699 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.699 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.699 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.699 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.699 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.700 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.700 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.700 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.700 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.700 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.700 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.700 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.700 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.700 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.700 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.700 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.700 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.700 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.700 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.700 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.701 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.701 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.701 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.701 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.701 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.701 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.701 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.701 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.701 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.701 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.701 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.701 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.701 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.701 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.701 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.702 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.702 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.702 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.702 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.702 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.702 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.702 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.702 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.702 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.702 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.702 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.702 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.702 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.702 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.703 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.703 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.703 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.703 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.703 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.703 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.703 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.703 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.703 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.703 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.703 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.703 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.703 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.703 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.703 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.704 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.704 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.704 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.704 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.704 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.704 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.704 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.704 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.704 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.704 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.704 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.704 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.704 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.704 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.705 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.705 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.705 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.705 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.705 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.705 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.705 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.705 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.705 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.705 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.705 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_host": { 2024-04-25T11:37:41.705 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_host", 2024-04-25T11:37:41.705 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.706 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.706 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.706 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.706 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.706 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.706 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.706 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.706 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.706 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.706 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.706 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.706 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_port": { 2024-04-25T11:37:41.706 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_port", 2024-04-25T11:37:41.706 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.706 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.707 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.707 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10051", 2024-04-25T11:37:41.707 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.707 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.707 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.707 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.707 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.707 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.707 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.707 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.707 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_sender": { 2024-04-25T11:37:41.707 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_sender", 2024-04-25T11:37:41.707 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.707 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.708 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.708 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T11:37:41.708 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.708 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.708 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.708 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.708 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.708 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.708 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.708 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.708 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.708 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.708 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-04-25T11:37:41.708 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.709 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.709 INFO:teuthology.orchestra.run.smithi069.stdout: "modules": [ 2024-04-25T11:37:41.709 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm", 2024-04-25T11:37:41.709 INFO:teuthology.orchestra.run.smithi069.stdout: "dashboard", 2024-04-25T11:37:41.709 INFO:teuthology.orchestra.run.smithi069.stdout: "iostat", 2024-04-25T11:37:41.709 INFO:teuthology.orchestra.run.smithi069.stdout: "nfs", 2024-04-25T11:37:41.709 INFO:teuthology.orchestra.run.smithi069.stdout: "restful" 2024-04-25T11:37:41.709 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.709 INFO:teuthology.orchestra.run.smithi069.stdout: "available_modules": [ 2024-04-25T11:37:41.709 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.709 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "alerts", 2024-04-25T11:37:41.709 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.709 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.709 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.710 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-04-25T11:37:41.710 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-04-25T11:37:41.710 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:41.710 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.710 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.710 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-04-25T11:37:41.710 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.710 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.710 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.710 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T11:37:41.710 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.710 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.710 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.710 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.711 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.711 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.711 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.711 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.711 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.711 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.711 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.711 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.711 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.711 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.711 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.711 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.711 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.711 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.711 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.712 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.712 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.712 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.712 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.712 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.712 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.712 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.712 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.712 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.712 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.712 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.712 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.712 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.712 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.713 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.713 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.713 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.713 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.713 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.713 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.713 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.713 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.713 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.713 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.713 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.713 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.713 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.713 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.714 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.714 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.714 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.714 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.714 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.714 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.714 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.714 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.714 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.714 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.714 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.714 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.714 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.714 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.714 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.715 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.715 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.715 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.715 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.715 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.715 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.715 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.715 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.715 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.715 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.715 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.715 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.715 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_destination": { 2024-04-25T11:37:41.715 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_destination", 2024-04-25T11:37:41.715 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.716 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.716 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.716 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.716 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.716 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.716 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.716 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Email address to send alerts to", 2024-04-25T11:37:41.716 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.716 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.716 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.716 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.716 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_from_name": { 2024-04-25T11:37:41.716 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_from_name", 2024-04-25T11:37:41.716 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.717 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.717 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.717 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "Ceph", 2024-04-25T11:37:41.717 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.717 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.717 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.717 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Email From: name", 2024-04-25T11:37:41.717 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.717 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.717 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.717 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.717 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_host": { 2024-04-25T11:37:41.717 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_host", 2024-04-25T11:37:41.717 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.717 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.717 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.718 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.718 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.718 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.718 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.718 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP server", 2024-04-25T11:37:41.718 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.718 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.718 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.718 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.719 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_password": { 2024-04-25T11:37:41.719 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_password", 2024-04-25T11:37:41.719 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.719 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.719 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.719 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.719 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.719 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.720 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.720 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Password to authenticate with", 2024-04-25T11:37:41.720 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.720 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.720 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.720 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.720 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_port": { 2024-04-25T11:37:41.720 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_port", 2024-04-25T11:37:41.720 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.720 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.720 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.720 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "465", 2024-04-25T11:37:41.720 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.720 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.720 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.721 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP port", 2024-04-25T11:37:41.721 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.721 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.721 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.721 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.721 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_sender": { 2024-04-25T11:37:41.721 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_sender", 2024-04-25T11:37:41.721 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.721 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.721 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.721 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.721 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.721 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.721 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.721 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP envelope sender", 2024-04-25T11:37:41.722 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.722 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.722 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.722 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.722 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_ssl": { 2024-04-25T11:37:41.722 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_ssl", 2024-04-25T11:37:41.722 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.722 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.722 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.722 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.722 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.722 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.722 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.722 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T11:37:41.723 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.723 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.723 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.723 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.723 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_user": { 2024-04-25T11:37:41.723 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_user", 2024-04-25T11:37:41.723 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.723 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.723 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.723 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.723 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.723 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.723 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.723 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "User to authenticate as", 2024-04-25T11:37:41.724 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.724 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.724 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.724 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.724 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.724 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.724 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.724 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.724 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.724 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.724 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.724 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.724 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.724 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.724 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.725 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.725 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.725 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.725 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.725 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.725 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.725 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "balancer", 2024-04-25T11:37:41.725 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.725 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.725 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.725 INFO:teuthology.orchestra.run.smithi069.stdout: "active": { 2024-04-25T11:37:41.725 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "active", 2024-04-25T11:37:41.725 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.725 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.726 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.726 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.726 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.726 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.726 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.726 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T11:37:41.726 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.726 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.726 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.726 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.726 INFO:teuthology.orchestra.run.smithi069.stdout: "begin_time": { 2024-04-25T11:37:41.726 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "begin_time", 2024-04-25T11:37:41.726 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.726 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.727 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.727 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0000", 2024-04-25T11:37:41.727 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.727 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.727 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.727 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T11:37:41.727 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T11:37:41.727 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.727 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.727 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.727 INFO:teuthology.orchestra.run.smithi069.stdout: "begin_weekday": { 2024-04-25T11:37:41.727 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "begin_weekday", 2024-04-25T11:37:41.727 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-04-25T11:37:41.727 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.728 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.728 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.728 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0", 2024-04-25T11:37:41.728 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "6", 2024-04-25T11:37:41.728 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.728 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T11:37:41.728 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T11:37:41.728 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.728 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.728 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.728 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_max_iterations": { 2024-04-25T11:37:41.728 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_max_iterations", 2024-04-25T11:37:41.728 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-04-25T11:37:41.728 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.728 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.729 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "25", 2024-04-25T11:37:41.729 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-04-25T11:37:41.729 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "250", 2024-04-25T11:37:41.729 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.729 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T11:37:41.729 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.729 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.729 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.729 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.729 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_metrics": { 2024-04-25T11:37:41.729 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_metrics", 2024-04-25T11:37:41.729 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.729 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.729 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.730 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T11:37:41.730 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.730 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.730 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.730 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T11:37:41.730 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T11:37:41.730 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.730 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.730 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.730 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_step": { 2024-04-25T11:37:41.730 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_step", 2024-04-25T11:37:41.730 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-04-25T11:37:41.730 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.730 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.731 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0.5", 2024-04-25T11:37:41.731 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0.001", 2024-04-25T11:37:41.731 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "0.999", 2024-04-25T11:37:41.731 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.731 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "aggressiveness of optimization", 2024-04-25T11:37:41.731 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T11:37:41.731 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.731 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.731 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.731 INFO:teuthology.orchestra.run.smithi069.stdout: "end_time": { 2024-04-25T11:37:41.731 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "end_time", 2024-04-25T11:37:41.731 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.731 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.731 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.732 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "2359", 2024-04-25T11:37:41.732 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.732 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.732 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.732 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T11:37:41.732 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T11:37:41.732 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.732 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.732 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.732 INFO:teuthology.orchestra.run.smithi069.stdout: "end_weekday": { 2024-04-25T11:37:41.732 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "end_weekday", 2024-04-25T11:37:41.732 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-04-25T11:37:41.732 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.732 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.732 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.732 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0", 2024-04-25T11:37:41.733 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "6", 2024-04-25T11:37:41.733 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.733 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T11:37:41.733 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T11:37:41.733 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.733 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.733 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.733 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.733 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.733 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.733 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.733 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.733 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.733 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.733 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.733 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.734 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.734 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.734 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.734 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.734 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.734 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.734 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.734 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.734 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.734 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.734 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.734 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.734 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.734 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.735 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.735 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.735 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.735 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.735 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.735 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.735 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.735 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.735 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.735 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.735 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.735 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.735 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.735 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.735 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.735 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.736 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.736 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.736 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.736 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.736 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.736 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.736 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.736 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.736 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.736 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.736 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.736 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.736 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.736 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.737 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.737 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.737 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.737 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.737 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.737 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.737 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.737 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.737 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.737 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.737 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.737 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.737 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.737 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.737 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.738 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.738 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.738 INFO:teuthology.orchestra.run.smithi069.stdout: "min_score": { 2024-04-25T11:37:41.738 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "min_score", 2024-04-25T11:37:41.738 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-04-25T11:37:41.738 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.738 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.738 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.738 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.738 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.738 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.738 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T11:37:41.738 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.738 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.739 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.739 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.739 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": { 2024-04-25T11:37:41.739 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mode", 2024-04-25T11:37:41.739 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.739 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.739 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.739 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "upmap", 2024-04-25T11:37:41.739 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.739 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.739 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.739 INFO:teuthology.orchestra.run.smithi069.stdout: "crush-compat", 2024-04-25T11:37:41.739 INFO:teuthology.orchestra.run.smithi069.stdout: "none", 2024-04-25T11:37:41.739 INFO:teuthology.orchestra.run.smithi069.stdout: "read", 2024-04-25T11:37:41.740 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap", 2024-04-25T11:37:41.740 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap-read" 2024-04-25T11:37:41.740 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.740 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Balancer mode", 2024-04-25T11:37:41.740 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.740 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.740 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.740 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.740 INFO:teuthology.orchestra.run.smithi069.stdout: "pool_ids": { 2024-04-25T11:37:41.740 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pool_ids", 2024-04-25T11:37:41.740 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.740 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.740 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.740 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.740 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.740 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.741 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.741 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T11:37:41.741 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.741 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.741 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.741 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.741 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-04-25T11:37:41.741 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-04-25T11:37:41.741 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:41.741 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.741 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.741 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-04-25T11:37:41.741 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.741 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.741 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.741 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T11:37:41.741 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.742 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.742 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.742 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.742 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.742 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.742 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.742 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.742 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.742 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.742 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.742 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.742 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.742 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.742 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.743 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.743 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.743 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.743 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap_max_deviation": { 2024-04-25T11:37:41.743 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "upmap_max_deviation", 2024-04-25T11:37:41.743 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.743 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.743 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.743 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-04-25T11:37:41.743 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-04-25T11:37:41.743 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.743 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.743 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T11:37:41.743 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T11:37:41.743 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.743 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.744 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.744 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap_max_optimizations": { 2024-04-25T11:37:41.744 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "upmap_max_optimizations", 2024-04-25T11:37:41.744 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-04-25T11:37:41.744 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.744 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.744 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-25T11:37:41.744 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.744 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.744 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.744 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T11:37:41.744 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.744 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.744 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.744 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.744 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.744 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.745 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.745 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cephadm", 2024-04-25T11:37:41.745 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.745 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.745 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.745 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_down_multiplier": { 2024-04-25T11:37:41.745 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_down_multiplier", 2024-04-25T11:37:41.745 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-04-25T11:37:41.745 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.745 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.745 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3.0", 2024-04-25T11:37:41.745 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.745 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.745 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.745 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T11:37:41.746 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.746 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.746 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.746 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.746 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_refresh_rate": { 2024-04-25T11:37:41.746 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_refresh_rate", 2024-04-25T11:37:41.746 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:41.746 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.746 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.746 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "20", 2024-04-25T11:37:41.746 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.746 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.746 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.746 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T11:37:41.746 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.746 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.747 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.747 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.747 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_starting_port": { 2024-04-25T11:37:41.747 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_starting_port", 2024-04-25T11:37:41.747 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.747 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.747 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.747 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "4721", 2024-04-25T11:37:41.747 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.747 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.747 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.747 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T11:37:41.747 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.747 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.747 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.747 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.748 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_ptrace": { 2024-04-25T11:37:41.748 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_ptrace", 2024-04-25T11:37:41.748 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.748 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.748 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.748 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.748 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.748 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.748 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.748 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T11:37:41.748 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T11:37:41.748 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.748 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.748 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.748 INFO:teuthology.orchestra.run.smithi069.stdout: "autotune_interval": { 2024-04-25T11:37:41.748 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "autotune_interval", 2024-04-25T11:37:41.749 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:41.749 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.749 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.749 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-04-25T11:37:41.749 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.749 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.749 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.749 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T11:37:41.749 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.749 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.749 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.749 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.749 INFO:teuthology.orchestra.run.smithi069.stdout: "autotune_memory_target_ratio": { 2024-04-25T11:37:41.749 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T11:37:41.749 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-04-25T11:37:41.750 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.750 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.750 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0.7", 2024-04-25T11:37:41.750 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.750 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.750 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.750 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T11:37:41.750 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.750 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.750 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.750 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.750 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm_log_destination": { 2024-04-25T11:37:41.750 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cephadm_log_destination", 2024-04-25T11:37:41.750 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.750 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.750 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.751 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.751 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.751 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.751 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.751 INFO:teuthology.orchestra.run.smithi069.stdout: "file", 2024-04-25T11:37:41.751 INFO:teuthology.orchestra.run.smithi069.stdout: "file,syslog", 2024-04-25T11:37:41.751 INFO:teuthology.orchestra.run.smithi069.stdout: "syslog" 2024-04-25T11:37:41.751 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.751 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T11:37:41.751 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.751 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.751 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.751 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.751 INFO:teuthology.orchestra.run.smithi069.stdout: "cgroups_split": { 2024-04-25T11:37:41.751 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cgroups_split", 2024-04-25T11:37:41.752 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.752 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.752 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.752 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.752 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.752 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.752 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.752 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T11:37:41.752 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.752 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.752 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.752 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.752 INFO:teuthology.orchestra.run.smithi069.stdout: "config_checks_enabled": { 2024-04-25T11:37:41.752 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_checks_enabled", 2024-04-25T11:37:41.752 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.753 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.753 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.753 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.753 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.753 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.753 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.753 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T11:37:41.753 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.753 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.753 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.753 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.753 INFO:teuthology.orchestra.run.smithi069.stdout: "config_dashboard": { 2024-04-25T11:37:41.753 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_dashboard", 2024-04-25T11:37:41.753 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.753 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.754 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.754 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.754 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.754 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.754 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.754 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T11:37:41.754 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.754 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.754 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.754 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.754 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_alertmanager": { 2024-04-25T11:37:41.754 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_alertmanager", 2024-04-25T11:37:41.754 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.754 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.754 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.755 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T11:37:41.755 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.755 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.755 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.755 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-04-25T11:37:41.755 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.755 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.755 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.755 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.755 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_base": { 2024-04-25T11:37:41.755 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_base", 2024-04-25T11:37:41.755 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.755 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.755 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.756 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T11:37:41.756 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.756 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.756 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.756 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Container image name, without the tag", 2024-04-25T11:37:41.756 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.756 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.756 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.756 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.756 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_elasticsearch": { 2024-04-25T11:37:41.756 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_elasticsearch", 2024-04-25T11:37:41.756 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.756 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.756 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.756 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T11:37:41.757 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.757 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.757 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.757 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "elasticsearch container image", 2024-04-25T11:37:41.757 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.757 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.757 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.757 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.757 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_grafana": { 2024-04-25T11:37:41.757 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_grafana", 2024-04-25T11:37:41.757 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.757 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.757 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.757 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T11:37:41.758 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.758 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.758 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.758 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-04-25T11:37:41.758 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.758 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.758 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.758 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.758 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_haproxy": { 2024-04-25T11:37:41.758 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_haproxy", 2024-04-25T11:37:41.758 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.758 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.758 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.758 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T11:37:41.758 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.759 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.759 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.759 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "HAproxy container image", 2024-04-25T11:37:41.759 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.759 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.759 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.759 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.759 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_agent": { 2024-04-25T11:37:41.759 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_agent", 2024-04-25T11:37:41.759 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.759 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.759 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.759 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T11:37:41.759 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.760 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.760 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.760 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger agent container image", 2024-04-25T11:37:41.760 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.760 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.760 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.760 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.760 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_collector": { 2024-04-25T11:37:41.760 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_collector", 2024-04-25T11:37:41.760 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.760 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.760 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.760 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T11:37:41.760 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.760 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.761 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.761 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger collector container image", 2024-04-25T11:37:41.761 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.761 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.761 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.761 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.761 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_query": { 2024-04-25T11:37:41.761 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_query", 2024-04-25T11:37:41.761 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.761 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.761 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.761 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T11:37:41.762 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.762 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.762 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.762 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger query container image", 2024-04-25T11:37:41.762 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.762 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.762 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.762 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.762 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_keepalived": { 2024-04-25T11:37:41.762 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_keepalived", 2024-04-25T11:37:41.762 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.762 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.762 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.762 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T11:37:41.763 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.763 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.763 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.763 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Keepalived container image", 2024-04-25T11:37:41.763 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.763 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.763 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.763 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.763 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_loki": { 2024-04-25T11:37:41.763 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_loki", 2024-04-25T11:37:41.763 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.763 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.763 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.763 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T11:37:41.764 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.764 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.764 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.764 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Loki container image", 2024-04-25T11:37:41.764 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.764 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.764 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.764 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.764 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_node_exporter": { 2024-04-25T11:37:41.764 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_node_exporter", 2024-04-25T11:37:41.764 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.764 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.764 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.764 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T11:37:41.765 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.765 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.765 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.765 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-04-25T11:37:41.765 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.765 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.765 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.765 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.765 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_nvmeof": { 2024-04-25T11:37:41.765 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_nvmeof", 2024-04-25T11:37:41.765 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.765 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.765 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.765 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T11:37:41.765 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.766 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.766 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.766 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Nvme-of container image", 2024-04-25T11:37:41.766 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.766 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.766 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.766 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.766 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_prometheus": { 2024-04-25T11:37:41.766 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_prometheus", 2024-04-25T11:37:41.766 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.766 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.766 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.766 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T11:37:41.766 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.766 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.767 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.767 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-04-25T11:37:41.767 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.767 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.767 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.767 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.767 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_promtail": { 2024-04-25T11:37:41.767 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_promtail", 2024-04-25T11:37:41.767 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.767 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.767 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.767 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T11:37:41.767 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.767 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.767 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.767 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Promtail container image", 2024-04-25T11:37:41.768 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.768 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.768 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.768 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.768 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_samba": { 2024-04-25T11:37:41.768 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_samba", 2024-04-25T11:37:41.768 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.768 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.768 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.768 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T11:37:41.768 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.768 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.768 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.768 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Samba/SMB container image", 2024-04-25T11:37:41.768 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.769 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.769 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.769 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.769 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_snmp_gateway": { 2024-04-25T11:37:41.769 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_snmp_gateway", 2024-04-25T11:37:41.769 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.769 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.769 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.769 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T11:37:41.769 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.769 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.769 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.769 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SNMP Gateway container image", 2024-04-25T11:37:41.769 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.769 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.770 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.770 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.770 INFO:teuthology.orchestra.run.smithi069.stdout: "container_init": { 2024-04-25T11:37:41.770 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_init", 2024-04-25T11:37:41.770 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.770 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.770 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.770 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.770 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.770 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.770 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.770 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T11:37:41.770 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.770 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.770 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.771 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.771 INFO:teuthology.orchestra.run.smithi069.stdout: "daemon_cache_timeout": { 2024-04-25T11:37:41.771 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "daemon_cache_timeout", 2024-04-25T11:37:41.771 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:41.771 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.771 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.771 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-04-25T11:37:41.771 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.771 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.771 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.771 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T11:37:41.771 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.771 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.771 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.771 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.771 INFO:teuthology.orchestra.run.smithi069.stdout: "default_cephadm_command_timeout": { 2024-04-25T11:37:41.772 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T11:37:41.772 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.772 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.772 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.772 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "900", 2024-04-25T11:37:41.772 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.772 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.772 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.772 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T11:37:41.772 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.772 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.772 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.772 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.772 INFO:teuthology.orchestra.run.smithi069.stdout: "default_registry": { 2024-04-25T11:37:41.772 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "default_registry", 2024-04-25T11:37:41.773 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.773 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.773 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.773 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io", 2024-04-25T11:37:41.773 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.773 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.773 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.773 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T11:37:41.773 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.773 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.773 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.773 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.773 INFO:teuthology.orchestra.run.smithi069.stdout: "device_cache_timeout": { 2024-04-25T11:37:41.774 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_cache_timeout", 2024-04-25T11:37:41.774 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:41.774 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.774 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.774 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1800", 2024-04-25T11:37:41.774 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.774 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.774 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.774 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache device inventory", 2024-04-25T11:37:41.774 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.774 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.774 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.774 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.774 INFO:teuthology.orchestra.run.smithi069.stdout: "device_enhanced_scan": { 2024-04-25T11:37:41.774 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_enhanced_scan", 2024-04-25T11:37:41.775 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.775 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.775 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.775 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.775 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.775 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.775 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.775 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T11:37:41.775 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.775 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.775 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.775 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.775 INFO:teuthology.orchestra.run.smithi069.stdout: "facts_cache_timeout": { 2024-04-25T11:37:41.775 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "facts_cache_timeout", 2024-04-25T11:37:41.775 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:41.775 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.775 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.776 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-04-25T11:37:41.776 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.776 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.776 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.776 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache host facts data", 2024-04-25T11:37:41.776 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.776 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.776 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.776 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.776 INFO:teuthology.orchestra.run.smithi069.stdout: "grafana_dashboards_path": { 2024-04-25T11:37:41.776 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "grafana_dashboards_path", 2024-04-25T11:37:41.776 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.776 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.777 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.777 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T11:37:41.777 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.777 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.777 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.777 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T11:37:41.777 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.777 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.777 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.777 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.777 INFO:teuthology.orchestra.run.smithi069.stdout: "host_check_interval": { 2024-04-25T11:37:41.777 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "host_check_interval", 2024-04-25T11:37:41.777 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:41.777 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.777 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.777 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-04-25T11:37:41.778 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.778 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.778 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.778 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to perform a host check", 2024-04-25T11:37:41.778 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.778 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.778 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.778 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.778 INFO:teuthology.orchestra.run.smithi069.stdout: "hw_monitoring": { 2024-04-25T11:37:41.778 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "hw_monitoring", 2024-04-25T11:37:41.778 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.778 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.778 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.779 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.779 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.779 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.779 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.779 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T11:37:41.779 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.779 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.779 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.779 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.779 INFO:teuthology.orchestra.run.smithi069.stdout: "inventory_list_all": { 2024-04-25T11:37:41.779 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "inventory_list_all", 2024-04-25T11:37:41.779 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.779 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.779 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.779 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.780 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.780 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.780 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.780 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T11:37:41.780 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.780 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.780 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.780 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.780 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.780 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.780 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.780 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.780 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.781 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.781 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.781 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.781 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.781 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.781 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.781 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.781 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.781 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.781 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.781 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.781 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.781 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.781 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.781 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.782 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.782 INFO:teuthology.orchestra.run.smithi069.stdout: "log_refresh_metadata": { 2024-04-25T11:37:41.782 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_refresh_metadata", 2024-04-25T11:37:41.782 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.782 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.782 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.782 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.782 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.782 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.782 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.782 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T11:37:41.782 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.782 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.782 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.783 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.783 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.783 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.783 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.783 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.783 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.783 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.783 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.783 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.783 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.783 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T11:37:41.783 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.783 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.783 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.784 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.784 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.784 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.784 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.784 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.784 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.784 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.784 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.784 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.784 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.784 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.784 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.784 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.784 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.785 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.785 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.785 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.785 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.785 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.785 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.785 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.785 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.785 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.785 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.785 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.785 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.785 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.785 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.786 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.786 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.786 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.786 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.786 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.786 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.786 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.786 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.786 INFO:teuthology.orchestra.run.smithi069.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T11:37:41.786 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T11:37:41.786 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.786 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.786 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.786 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.787 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.787 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.787 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.787 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T11:37:41.787 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.787 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.787 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.787 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.787 INFO:teuthology.orchestra.run.smithi069.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T11:37:41.787 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T11:37:41.787 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.787 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.787 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.787 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "*", 2024-04-25T11:37:41.787 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.788 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.788 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.788 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T11:37:41.788 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.788 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.788 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.788 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.788 INFO:teuthology.orchestra.run.smithi069.stdout: "max_count_per_host": { 2024-04-25T11:37:41.788 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_count_per_host", 2024-04-25T11:37:41.788 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.788 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.788 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.788 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-25T11:37:41.788 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.788 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.789 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.789 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "max number of daemons per service per host", 2024-04-25T11:37:41.789 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.789 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.789 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.789 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.789 INFO:teuthology.orchestra.run.smithi069.stdout: "max_osd_draining_count": { 2024-04-25T11:37:41.789 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_osd_draining_count", 2024-04-25T11:37:41.789 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.789 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.789 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.789 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-25T11:37:41.789 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.789 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.790 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.790 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T11:37:41.790 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.790 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.790 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.790 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.790 INFO:teuthology.orchestra.run.smithi069.stdout: "migration_current": { 2024-04-25T11:37:41.790 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "migration_current", 2024-04-25T11:37:41.790 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.790 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.790 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.790 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.790 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.790 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.791 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.791 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "internal - do not modify", 2024-04-25T11:37:41.791 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.791 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.791 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.791 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.791 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": { 2024-04-25T11:37:41.791 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mode", 2024-04-25T11:37:41.791 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.791 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.791 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.791 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "root", 2024-04-25T11:37:41.791 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.791 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.792 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.792 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm-package", 2024-04-25T11:37:41.792 INFO:teuthology.orchestra.run.smithi069.stdout: "root" 2024-04-25T11:37:41.792 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.792 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T11:37:41.792 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.792 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.792 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.792 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.792 INFO:teuthology.orchestra.run.smithi069.stdout: "oob_default_addr": { 2024-04-25T11:37:41.792 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "oob_default_addr", 2024-04-25T11:37:41.792 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.792 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.792 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.793 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "169.254.1.1", 2024-04-25T11:37:41.793 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.793 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.793 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.793 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T11:37:41.793 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.793 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.793 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.793 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.793 INFO:teuthology.orchestra.run.smithi069.stdout: "prometheus_alerts_path": { 2024-04-25T11:37:41.793 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prometheus_alerts_path", 2024-04-25T11:37:41.793 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.793 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.793 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.794 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T11:37:41.794 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.794 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.794 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.794 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T11:37:41.794 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.794 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.794 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.794 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.794 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_insecure": { 2024-04-25T11:37:41.794 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_insecure", 2024-04-25T11:37:41.794 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.794 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.794 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.795 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.795 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.795 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.795 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.795 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T11:37:41.795 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.795 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.795 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.795 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.795 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_password": { 2024-04-25T11:37:41.795 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_password", 2024-04-25T11:37:41.795 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.795 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.795 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.796 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.796 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.796 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.796 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.796 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T11:37:41.796 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.796 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.796 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.796 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.796 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_url": { 2024-04-25T11:37:41.796 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_url", 2024-04-25T11:37:41.796 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.796 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.796 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.797 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.797 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.797 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.797 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.797 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T11:37:41.797 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.797 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.797 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.797 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.797 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_username": { 2024-04-25T11:37:41.797 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_username", 2024-04-25T11:37:41.797 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.797 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.797 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.797 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.798 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.798 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.798 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.798 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T11:37:41.798 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.798 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.798 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.798 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.798 INFO:teuthology.orchestra.run.smithi069.stdout: "secure_monitoring_stack": { 2024-04-25T11:37:41.798 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "secure_monitoring_stack", 2024-04-25T11:37:41.798 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.798 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.798 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.798 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.799 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.799 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.799 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.799 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T11:37:41.799 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.799 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.799 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.799 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.799 INFO:teuthology.orchestra.run.smithi069.stdout: "service_discovery_port": { 2024-04-25T11:37:41.799 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "service_discovery_port", 2024-04-25T11:37:41.799 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.799 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.799 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.799 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8765", 2024-04-25T11:37:41.800 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.800 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.800 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.800 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "cephadm service discovery port", 2024-04-25T11:37:41.800 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.800 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.800 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.800 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.800 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.800 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.800 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.800 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.800 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.800 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.800 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.801 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.801 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.801 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.801 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.801 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.801 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.801 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.801 INFO:teuthology.orchestra.run.smithi069.stdout: "ssh_config_file": { 2024-04-25T11:37:41.801 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssh_config_file", 2024-04-25T11:37:41.801 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.801 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.801 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.801 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.801 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.801 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.802 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.802 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T11:37:41.802 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.802 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.802 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.802 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.802 INFO:teuthology.orchestra.run.smithi069.stdout: "use_agent": { 2024-04-25T11:37:41.802 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "use_agent", 2024-04-25T11:37:41.802 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.802 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.802 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.802 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.802 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.802 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.803 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.803 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T11:37:41.803 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.803 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.803 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.803 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.803 INFO:teuthology.orchestra.run.smithi069.stdout: "use_repo_digest": { 2024-04-25T11:37:41.803 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "use_repo_digest", 2024-04-25T11:37:41.803 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.803 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.803 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.803 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.803 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.803 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.803 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.804 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T11:37:41.804 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.804 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.804 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.804 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.804 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_failed_host_check": { 2024-04-25T11:37:41.804 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_failed_host_check", 2024-04-25T11:37:41.804 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.804 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.804 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.804 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.804 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.804 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.804 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.804 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T11:37:41.804 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.805 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.805 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.805 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.805 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_stray_daemons": { 2024-04-25T11:37:41.805 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_stray_daemons", 2024-04-25T11:37:41.805 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.805 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.805 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.805 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.805 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.805 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.806 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.806 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T11:37:41.806 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.806 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.806 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.806 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.806 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_stray_hosts": { 2024-04-25T11:37:41.806 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_stray_hosts", 2024-04-25T11:37:41.806 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.806 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.806 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.806 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.806 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.806 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.806 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.807 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T11:37:41.807 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.807 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.807 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.807 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.807 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.807 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.807 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.807 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crash", 2024-04-25T11:37:41.807 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.807 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.807 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.807 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.807 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.807 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.807 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.808 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.808 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.808 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.808 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.808 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.808 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.808 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.808 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.808 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.808 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.808 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.808 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.808 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.808 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.808 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.809 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.809 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.809 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.809 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.809 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.809 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.809 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.809 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.809 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.809 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.809 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.809 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.809 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.809 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.809 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.810 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.810 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.810 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.810 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.810 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.810 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.810 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.810 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.810 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.810 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.810 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.810 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.810 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.810 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.811 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.811 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.811 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.811 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.811 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.811 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.811 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.811 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.811 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.811 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.811 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.811 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.811 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.811 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.811 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.812 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.812 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.812 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.812 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.812 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.812 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.812 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.812 INFO:teuthology.orchestra.run.smithi069.stdout: "retain_interval": { 2024-04-25T11:37:41.812 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "retain_interval", 2024-04-25T11:37:41.812 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:41.812 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.812 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.812 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "31536000", 2024-04-25T11:37:41.812 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.813 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.813 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.813 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T11:37:41.813 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.813 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.813 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.813 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.813 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.813 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.813 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.813 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.813 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.813 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.813 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.813 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.813 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.813 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.814 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.814 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.814 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.814 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.814 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_recent_interval": { 2024-04-25T11:37:41.814 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_recent_interval", 2024-04-25T11:37:41.814 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:41.814 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.814 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.814 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1209600", 2024-04-25T11:37:41.814 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.814 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.814 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.814 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T11:37:41.815 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.815 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.815 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.815 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.815 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.815 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.815 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.815 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "dashboard", 2024-04-25T11:37:41.815 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.815 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.815 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.815 INFO:teuthology.orchestra.run.smithi069.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T11:37:41.815 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T11:37:41.815 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.816 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.816 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.816 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-25T11:37:41.816 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.816 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.816 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.816 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.816 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.816 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.816 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.816 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.816 INFO:teuthology.orchestra.run.smithi069.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T11:37:41.816 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T11:37:41.816 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.816 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.817 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.817 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.817 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.817 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.817 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.817 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.817 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.817 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.817 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.817 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.817 INFO:teuthology.orchestra.run.smithi069.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T11:37:41.817 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T11:37:41.817 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.817 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.817 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.818 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.818 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.818 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.818 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.818 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.818 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.818 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.818 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.818 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.818 INFO:teuthology.orchestra.run.smithi069.stdout: "AUDIT_API_ENABLED": { 2024-04-25T11:37:41.818 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T11:37:41.818 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.818 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.818 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.819 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.819 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.819 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.819 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.819 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.819 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.819 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.819 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.819 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.819 INFO:teuthology.orchestra.run.smithi069.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T11:37:41.819 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T11:37:41.819 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.819 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.820 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.820 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.820 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.820 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.820 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.820 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.820 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.820 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.820 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.820 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.820 INFO:teuthology.orchestra.run.smithi069.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T11:37:41.820 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T11:37:41.820 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.821 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.821 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.821 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.821 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.821 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.821 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.821 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.821 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.821 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.821 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.821 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.821 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T11:37:41.821 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T11:37:41.822 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.822 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.822 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.822 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.822 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.822 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.822 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.822 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.822 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.822 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.822 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.822 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.822 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T11:37:41.822 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T11:37:41.823 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.823 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.823 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.823 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.823 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.823 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.823 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.823 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.823 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.823 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.823 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.823 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.823 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T11:37:41.823 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T11:37:41.824 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.824 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.824 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.824 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.824 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.824 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.824 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.824 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.824 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.824 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.824 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.824 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.824 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T11:37:41.824 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T11:37:41.824 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.825 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.825 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.825 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.825 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.825 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.825 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.825 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.825 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.825 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.825 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.825 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.825 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T11:37:41.825 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T11:37:41.825 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.826 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.826 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.826 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.826 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.826 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.826 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.826 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.826 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.826 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.826 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.826 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.826 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T11:37:41.826 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T11:37:41.826 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.827 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.827 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.827 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.827 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.827 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.827 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.827 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.827 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.827 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.827 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.827 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.827 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T11:37:41.827 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T11:37:41.827 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.828 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.828 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.828 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.828 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.828 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.828 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.828 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.828 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.828 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.828 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.828 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.828 INFO:teuthology.orchestra.run.smithi069.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T11:37:41.828 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T11:37:41.828 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.829 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.829 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.829 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.829 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.829 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.829 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.829 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.829 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.829 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.829 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.829 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.829 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T11:37:41.829 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T11:37:41.829 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.830 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.830 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.830 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2024-04-25T11:37:41.830 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.830 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.830 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.830 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.830 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.830 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.830 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.830 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.830 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T11:37:41.830 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T11:37:41.830 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.831 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.831 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.831 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.831 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.831 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.831 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.831 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.831 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.831 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.831 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.831 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.831 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_URL": { 2024-04-25T11:37:41.831 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_URL", 2024-04-25T11:37:41.831 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.831 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.831 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.832 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.832 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.832 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.832 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.832 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.832 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.832 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.832 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.832 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.832 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T11:37:41.832 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T11:37:41.832 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.832 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.832 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.833 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2024-04-25T11:37:41.833 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.833 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.833 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.833 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.833 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.833 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.833 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.833 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.833 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T11:37:41.833 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T11:37:41.833 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.833 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.834 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.834 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.834 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.834 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.834 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.834 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.834 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.834 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.834 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.834 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.834 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T11:37:41.834 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T11:37:41.834 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.834 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.835 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.835 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.835 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.835 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.835 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.835 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.835 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.835 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.835 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.835 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.835 INFO:teuthology.orchestra.run.smithi069.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T11:37:41.835 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T11:37:41.835 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.835 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.836 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.836 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.836 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.836 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.836 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.836 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.836 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.836 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.836 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.836 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.836 INFO:teuthology.orchestra.run.smithi069.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T11:37:41.836 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T11:37:41.836 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.837 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.837 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.837 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.837 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.837 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.837 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.837 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.837 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.837 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.837 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.837 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.837 INFO:teuthology.orchestra.run.smithi069.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T11:37:41.837 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T11:37:41.838 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.838 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.838 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.838 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "[]", 2024-04-25T11:37:41.838 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.838 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.838 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.838 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.838 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.838 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.838 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.838 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.838 INFO:teuthology.orchestra.run.smithi069.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T11:37:41.839 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T11:37:41.839 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.839 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.839 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.839 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "{}", 2024-04-25T11:37:41.839 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.839 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.839 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.839 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.839 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.839 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.839 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.839 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.839 INFO:teuthology.orchestra.run.smithi069.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T11:37:41.840 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T11:37:41.840 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.840 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.840 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.840 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.840 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.840 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.840 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.840 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.840 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.840 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.840 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.840 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.841 INFO:teuthology.orchestra.run.smithi069.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T11:37:41.841 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T11:37:41.841 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.841 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.841 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.841 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.841 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.841 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.841 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.841 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.841 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.841 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.841 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.841 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.842 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T11:37:41.842 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T11:37:41.842 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.842 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.842 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.842 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.842 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.842 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.842 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.842 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.842 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.842 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.842 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.843 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.843 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T11:37:41.843 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T11:37:41.843 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.843 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.843 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.843 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.843 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.843 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.843 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.843 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.843 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.843 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.843 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.844 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.844 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T11:37:41.844 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T11:37:41.844 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.844 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.844 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.844 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.844 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.844 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.844 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.844 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.844 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.844 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.844 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.845 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.845 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T11:37:41.845 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T11:37:41.845 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.845 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.845 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.845 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.845 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.845 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.845 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.845 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.845 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.845 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.845 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.846 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.846 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T11:37:41.846 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T11:37:41.846 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.846 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.846 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.846 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.846 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.846 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.846 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.846 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.846 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.846 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.847 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.847 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.847 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T11:37:41.847 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T11:37:41.847 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.847 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.847 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.847 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.847 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.847 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.847 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.847 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.847 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.848 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.848 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.848 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.848 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T11:37:41.848 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T11:37:41.848 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.848 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.848 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.848 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.848 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.848 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.848 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.848 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.849 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.849 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.849 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.849 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.849 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T11:37:41.849 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T11:37:41.849 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.849 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.849 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.849 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.849 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.849 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.849 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.849 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.850 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.850 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.850 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.850 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.850 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T11:37:41.850 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T11:37:41.850 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.850 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.850 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.850 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T11:37:41.850 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.850 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.850 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.850 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.850 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.851 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.851 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.851 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.851 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T11:37:41.851 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T11:37:41.851 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.851 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.851 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.851 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-25T11:37:41.851 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.851 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.851 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.851 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.851 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.852 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.852 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.852 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.852 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T11:37:41.852 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T11:37:41.852 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.852 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.852 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.852 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8", 2024-04-25T11:37:41.852 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.852 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.852 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.852 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.852 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.852 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.853 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.853 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.853 INFO:teuthology.orchestra.run.smithi069.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T11:37:41.853 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T11:37:41.853 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.853 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.853 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.853 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "45", 2024-04-25T11:37:41.853 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.853 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.853 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.853 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.853 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.853 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.854 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.854 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.854 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T11:37:41.854 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T11:37:41.854 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.854 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.854 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.854 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.854 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.854 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.854 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.854 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.854 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.854 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.855 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.855 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.855 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T11:37:41.855 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T11:37:41.855 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.855 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.855 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.855 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2024-04-25T11:37:41.855 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.855 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.855 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.855 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.855 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.855 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.856 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.856 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.856 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T11:37:41.856 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T11:37:41.856 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.856 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.856 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.856 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.856 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.856 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.856 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.856 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.856 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.856 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.856 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.857 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.857 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T11:37:41.857 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T11:37:41.857 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.857 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.857 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.857 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.857 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.857 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.857 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.857 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.857 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.857 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.857 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.857 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.858 INFO:teuthology.orchestra.run.smithi069.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T11:37:41.858 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T11:37:41.858 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.858 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.858 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.858 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.858 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.858 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.858 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.858 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.858 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.858 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.858 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.858 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.859 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T11:37:41.859 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T11:37:41.859 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.859 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.859 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.859 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.859 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.859 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.859 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.859 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.859 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.859 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.859 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.859 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.860 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T11:37:41.860 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T11:37:41.860 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.860 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.860 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.860 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-25T11:37:41.860 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.860 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.860 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.860 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.860 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.860 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.860 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.860 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.860 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T11:37:41.861 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T11:37:41.861 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.861 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.861 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.861 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-04-25T11:37:41.861 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.861 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.861 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.861 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.861 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.861 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.861 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.861 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.861 INFO:teuthology.orchestra.run.smithi069.stdout: "cross_origin_url": { 2024-04-25T11:37:41.861 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cross_origin_url", 2024-04-25T11:37:41.862 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.862 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.862 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.862 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.862 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.862 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.862 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.862 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.862 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.862 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.862 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.862 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.862 INFO:teuthology.orchestra.run.smithi069.stdout: "crt_file": { 2024-04-25T11:37:41.863 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crt_file", 2024-04-25T11:37:41.863 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.863 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.863 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.863 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.863 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.863 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.863 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.863 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.863 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.863 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.863 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.863 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.863 INFO:teuthology.orchestra.run.smithi069.stdout: "debug": { 2024-04-25T11:37:41.863 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "debug", 2024-04-25T11:37:41.864 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.864 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.864 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.864 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.864 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.864 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.864 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.864 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable/disable debug options", 2024-04-25T11:37:41.864 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.864 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.864 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.864 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.864 INFO:teuthology.orchestra.run.smithi069.stdout: "jwt_token_ttl": { 2024-04-25T11:37:41.864 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "jwt_token_ttl", 2024-04-25T11:37:41.865 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.865 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.865 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.865 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "28800", 2024-04-25T11:37:41.865 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.865 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.865 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.865 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.865 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.865 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.865 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.865 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.865 INFO:teuthology.orchestra.run.smithi069.stdout: "key_file": { 2024-04-25T11:37:41.865 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "key_file", 2024-04-25T11:37:41.866 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.866 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.866 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.866 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.866 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.866 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.866 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.866 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.866 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.866 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.866 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.866 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.866 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.866 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.867 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.867 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.867 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.867 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.867 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.867 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.867 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.867 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.867 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.867 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.867 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.867 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.867 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.867 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.867 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.868 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.868 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.868 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.868 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.868 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.868 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.868 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.868 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.868 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.868 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.868 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.868 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.868 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.868 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.868 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.869 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.869 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.869 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.869 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.869 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.869 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.869 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.869 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.869 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.869 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.869 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.869 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.870 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.870 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.870 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.870 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.870 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.870 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.870 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.870 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.870 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.870 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.870 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.870 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.870 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.871 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.871 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.871 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.871 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.871 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.871 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.871 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.871 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.871 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.871 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.871 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.871 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.871 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.871 INFO:teuthology.orchestra.run.smithi069.stdout: "motd": { 2024-04-25T11:37:41.871 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "motd", 2024-04-25T11:37:41.872 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.872 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.872 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.872 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.872 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.872 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.872 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.872 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "The message of the day", 2024-04-25T11:37:41.872 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.872 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.872 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.872 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.872 INFO:teuthology.orchestra.run.smithi069.stdout: "redirect_resolve_ip_addr": { 2024-04-25T11:37:41.873 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T11:37:41.873 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.873 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.873 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.873 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.873 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.873 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.873 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.873 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.873 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.873 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.873 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.873 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.873 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2024-04-25T11:37:41.873 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2024-04-25T11:37:41.874 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.874 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.874 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.874 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "::", 2024-04-25T11:37:41.874 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.874 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.874 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.874 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.874 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.874 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.874 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.874 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.874 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2024-04-25T11:37:41.875 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2024-04-25T11:37:41.875 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.875 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.875 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.875 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8080", 2024-04-25T11:37:41.875 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.875 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.875 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.875 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.875 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.875 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.875 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.875 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.875 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.876 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.876 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.876 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.876 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.876 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.876 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.876 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.876 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.876 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.876 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.876 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.876 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.876 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.877 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl": { 2024-04-25T11:37:41.877 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl", 2024-04-25T11:37:41.877 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.877 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.877 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.877 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.877 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.877 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.877 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.877 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.877 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.877 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.877 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.877 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.877 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl_server_port": { 2024-04-25T11:37:41.878 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl_server_port", 2024-04-25T11:37:41.878 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.878 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.878 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.878 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8443", 2024-04-25T11:37:41.878 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.878 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.878 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.878 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.878 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.878 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.878 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.878 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.879 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_behaviour": { 2024-04-25T11:37:41.879 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_behaviour", 2024-04-25T11:37:41.879 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.879 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.879 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.879 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "redirect", 2024-04-25T11:37:41.879 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.879 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.879 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.879 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.879 INFO:teuthology.orchestra.run.smithi069.stdout: "redirect" 2024-04-25T11:37:41.879 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.879 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.880 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.880 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.880 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.880 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.880 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_error_status_code": { 2024-04-25T11:37:41.880 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_error_status_code", 2024-04-25T11:37:41.880 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.880 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.880 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.880 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "500", 2024-04-25T11:37:41.880 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "400", 2024-04-25T11:37:41.880 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "599", 2024-04-25T11:37:41.880 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.880 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.880 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.881 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.881 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.881 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.881 INFO:teuthology.orchestra.run.smithi069.stdout: "url_prefix": { 2024-04-25T11:37:41.881 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "url_prefix", 2024-04-25T11:37:41.881 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.881 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.881 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.881 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.881 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.881 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.881 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.881 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.881 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.881 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.882 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.882 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.882 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.882 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.882 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.882 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "devicehealth", 2024-04-25T11:37:41.882 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.882 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.882 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.882 INFO:teuthology.orchestra.run.smithi069.stdout: "enable_monitoring": { 2024-04-25T11:37:41.882 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enable_monitoring", 2024-04-25T11:37:41.882 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.882 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.882 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.883 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.883 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.883 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.883 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.883 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "monitor device health metrics", 2024-04-25T11:37:41.883 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.883 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.883 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.883 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.883 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.883 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.883 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.883 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.883 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.883 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.884 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.884 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.884 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.884 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.884 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.884 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.884 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.884 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.884 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.884 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.884 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.884 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.884 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.884 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.885 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.885 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.885 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.885 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.885 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.885 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.885 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.885 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.885 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.885 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.885 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.885 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.885 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.885 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.885 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.886 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.886 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.886 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.886 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.886 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.886 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.886 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.886 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.886 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.886 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.886 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.886 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.886 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.887 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.887 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.887 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.887 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.887 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.887 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.887 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.887 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.887 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.887 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.887 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.887 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.887 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.887 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.888 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.888 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.888 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.888 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.888 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.888 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.888 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.888 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.888 INFO:teuthology.orchestra.run.smithi069.stdout: "mark_out_threshold": { 2024-04-25T11:37:41.888 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mark_out_threshold", 2024-04-25T11:37:41.888 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:41.888 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.888 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.888 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "2419200", 2024-04-25T11:37:41.888 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.889 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.889 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.889 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T11:37:41.889 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.889 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.889 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.889 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.889 INFO:teuthology.orchestra.run.smithi069.stdout: "pool_name": { 2024-04-25T11:37:41.889 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pool_name", 2024-04-25T11:37:41.889 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.889 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.889 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.889 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "device_health_metrics", 2024-04-25T11:37:41.890 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.890 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.890 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.890 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T11:37:41.890 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.890 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.890 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.890 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.890 INFO:teuthology.orchestra.run.smithi069.stdout: "retention_period": { 2024-04-25T11:37:41.890 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "retention_period", 2024-04-25T11:37:41.890 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:41.890 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.890 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.891 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15552000", 2024-04-25T11:37:41.891 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.891 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.891 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.891 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long to retain device health metrics", 2024-04-25T11:37:41.891 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.891 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.891 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.891 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.891 INFO:teuthology.orchestra.run.smithi069.stdout: "scrape_frequency": { 2024-04-25T11:37:41.891 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "scrape_frequency", 2024-04-25T11:37:41.891 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:41.891 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.891 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.891 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "86400", 2024-04-25T11:37:41.892 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.892 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.892 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.892 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T11:37:41.892 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.892 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.892 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.892 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.892 INFO:teuthology.orchestra.run.smithi069.stdout: "self_heal": { 2024-04-25T11:37:41.892 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "self_heal", 2024-04-25T11:37:41.892 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.892 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.892 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.893 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:41.893 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.893 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.893 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.893 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T11:37:41.893 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.893 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.893 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.893 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.893 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-04-25T11:37:41.893 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-04-25T11:37:41.893 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:41.893 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.893 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.894 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-04-25T11:37:41.894 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.894 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.894 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.894 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T11:37:41.894 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.894 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.894 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.894 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.894 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.894 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.894 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.894 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.894 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.895 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.895 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.895 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.895 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.895 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.895 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.895 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.895 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.895 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.895 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_threshold": { 2024-04-25T11:37:41.895 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_threshold", 2024-04-25T11:37:41.895 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:41.895 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.895 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.895 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "7257600", 2024-04-25T11:37:41.895 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.896 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.896 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.896 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T11:37:41.896 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.896 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.896 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.896 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.896 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.896 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.896 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.896 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "diskprediction_local", 2024-04-25T11:37:41.896 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.896 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.896 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.896 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.896 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.896 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.897 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.897 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.897 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.897 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.897 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.897 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.897 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.897 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.897 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.897 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.897 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.897 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.897 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.897 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.897 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.898 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.898 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.898 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.898 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.898 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.898 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.898 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.898 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.898 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.898 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.898 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.898 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.898 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.898 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.899 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.899 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.899 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.899 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.899 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.899 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.899 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.899 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.899 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.899 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.899 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.899 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.899 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.899 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.900 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.900 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.900 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.900 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.900 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.900 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.900 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.900 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.900 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.900 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.900 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.900 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.900 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.900 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.900 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.901 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.901 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.901 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.901 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.901 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.901 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.901 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.901 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.901 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.901 INFO:teuthology.orchestra.run.smithi069.stdout: "predict_interval": { 2024-04-25T11:37:41.901 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "predict_interval", 2024-04-25T11:37:41.901 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.901 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.901 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.901 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "86400", 2024-04-25T11:37:41.901 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.901 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.902 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.902 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.902 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.902 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.902 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.902 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.902 INFO:teuthology.orchestra.run.smithi069.stdout: "predictor_model": { 2024-04-25T11:37:41.902 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "predictor_model", 2024-04-25T11:37:41.902 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.902 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.902 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.902 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "prophetstor", 2024-04-25T11:37:41.902 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.903 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.903 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.903 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.903 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.903 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.903 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.903 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.903 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-04-25T11:37:41.903 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-04-25T11:37:41.903 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.903 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.903 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.903 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-04-25T11:37:41.903 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.904 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.904 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.904 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.904 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.904 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.904 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.904 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.904 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.904 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.904 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.904 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.904 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.904 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.904 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.904 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.904 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.905 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.905 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.905 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.905 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.905 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.905 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.905 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.905 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.905 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "influx", 2024-04-25T11:37:41.905 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": false, 2024-04-25T11:37:41.905 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "influxdb python module not found", 2024-04-25T11:37:41.905 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.905 INFO:teuthology.orchestra.run.smithi069.stdout: "batch_size": { 2024-04-25T11:37:41.905 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "batch_size", 2024-04-25T11:37:41.905 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.906 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.906 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.906 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5000", 2024-04-25T11:37:41.906 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.906 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.906 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.906 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T11:37:41.906 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.906 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.906 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.906 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.906 INFO:teuthology.orchestra.run.smithi069.stdout: "database": { 2024-04-25T11:37:41.906 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "database", 2024-04-25T11:37:41.907 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.907 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.907 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.907 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "ceph", 2024-04-25T11:37:41.907 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.907 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.907 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.907 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T11:37:41.907 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.907 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.907 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.908 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.908 INFO:teuthology.orchestra.run.smithi069.stdout: "hostname": { 2024-04-25T11:37:41.908 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "hostname", 2024-04-25T11:37:41.908 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.908 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.908 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.908 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.908 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.909 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.909 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.909 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB server hostname", 2024-04-25T11:37:41.909 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.909 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.909 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.909 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.910 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-04-25T11:37:41.910 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-04-25T11:37:41.910 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:41.910 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.910 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.910 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "30", 2024-04-25T11:37:41.910 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "5", 2024-04-25T11:37:41.911 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.911 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.911 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T11:37:41.911 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.911 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.911 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.911 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.911 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.911 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.911 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.911 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.912 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.912 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.912 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.912 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.912 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.912 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.912 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.912 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.912 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.912 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.912 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.912 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.912 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.913 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.913 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.913 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.913 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.913 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.913 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.913 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.913 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.913 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.913 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.913 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.913 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.913 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.913 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.914 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.914 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.914 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.914 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.914 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.914 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.914 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.914 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.914 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.914 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.914 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.914 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.914 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.914 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.915 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.915 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.915 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.915 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.915 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.915 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.915 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.915 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.915 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.915 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.915 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.915 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.915 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.915 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.915 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.916 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.916 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.916 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.916 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.916 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.916 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.916 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.916 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.916 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.916 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.916 INFO:teuthology.orchestra.run.smithi069.stdout: "password": { 2024-04-25T11:37:41.916 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "password", 2024-04-25T11:37:41.916 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.916 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.917 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.917 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.917 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.917 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.917 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.917 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "password of InfluxDB server user", 2024-04-25T11:37:41.917 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.917 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.917 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.917 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.917 INFO:teuthology.orchestra.run.smithi069.stdout: "port": { 2024-04-25T11:37:41.917 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "port", 2024-04-25T11:37:41.917 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.917 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.917 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.918 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8086", 2024-04-25T11:37:41.918 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.918 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.918 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.918 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB server port", 2024-04-25T11:37:41.918 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.918 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.918 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.918 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.918 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.918 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.918 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.918 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.918 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.919 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.919 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.919 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.919 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.919 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.919 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.919 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.919 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.919 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.919 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl": { 2024-04-25T11:37:41.919 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl", 2024-04-25T11:37:41.919 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.919 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.920 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.920 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "false", 2024-04-25T11:37:41.920 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.920 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.920 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.920 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T11:37:41.920 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.920 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.920 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.920 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.920 INFO:teuthology.orchestra.run.smithi069.stdout: "threads": { 2024-04-25T11:37:41.920 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "threads", 2024-04-25T11:37:41.920 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.920 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.921 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.921 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-04-25T11:37:41.921 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-04-25T11:37:41.921 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "32", 2024-04-25T11:37:41.921 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.921 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T11:37:41.921 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.921 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.921 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.921 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.921 INFO:teuthology.orchestra.run.smithi069.stdout: "username": { 2024-04-25T11:37:41.921 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "username", 2024-04-25T11:37:41.921 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.921 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.921 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.921 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.922 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.922 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.922 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.922 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "username of InfluxDB server user", 2024-04-25T11:37:41.922 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.922 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.922 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.922 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.922 INFO:teuthology.orchestra.run.smithi069.stdout: "verify_ssl": { 2024-04-25T11:37:41.922 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "verify_ssl", 2024-04-25T11:37:41.922 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.922 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.922 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.923 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "true", 2024-04-25T11:37:41.923 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.923 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.923 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.923 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T11:37:41.923 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.923 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.923 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.923 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.923 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.923 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.923 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.923 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "insights", 2024-04-25T11:37:41.923 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.924 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.924 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.924 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.924 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.924 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.924 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.924 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.924 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.924 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.924 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.924 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.924 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.924 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.924 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.924 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.925 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.925 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.925 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.925 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.925 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.925 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.925 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.925 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.925 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.925 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.925 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.925 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.925 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.925 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.925 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.925 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.926 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.926 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.926 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.926 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.926 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.926 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.926 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.926 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.926 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.926 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.926 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.926 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.926 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.926 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.927 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.927 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.927 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.927 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.927 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.927 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.927 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.927 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.927 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.927 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.927 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.927 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.927 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.927 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.928 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.928 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.928 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.928 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.928 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.928 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.928 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.928 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.928 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.928 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.928 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.928 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.928 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.928 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.928 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.929 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.929 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.929 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.929 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.929 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.929 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.929 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.929 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.929 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.929 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.929 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.929 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.929 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.929 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.930 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.930 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "iostat", 2024-04-25T11:37:41.930 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.930 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.930 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.930 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.930 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.930 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.930 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.930 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.930 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.930 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.930 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.931 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.931 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.931 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.931 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.931 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.931 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.931 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.931 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.931 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.931 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.931 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.931 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.931 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.931 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.931 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.931 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.932 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.932 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.932 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.932 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.932 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.932 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.932 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.932 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.932 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.932 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.932 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.932 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.932 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.932 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.932 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.932 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.933 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.933 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.933 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.933 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.933 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.933 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.933 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.933 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.933 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.933 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.933 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.933 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.933 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.933 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.934 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.934 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.934 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.934 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.934 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.934 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.934 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.934 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.934 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.934 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.934 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.934 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.934 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.934 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.935 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.935 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.935 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.935 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.935 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.935 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.935 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.935 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.935 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.935 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.935 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.935 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.935 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.935 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.935 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.935 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.936 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.936 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.936 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.936 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "k8sevents", 2024-04-25T11:37:41.936 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.936 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.936 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.936 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_event_retention_days": { 2024-04-25T11:37:41.936 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ceph_event_retention_days", 2024-04-25T11:37:41.936 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.936 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.936 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.936 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "7", 2024-04-25T11:37:41.937 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.937 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.937 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.937 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T11:37:41.937 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.937 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.937 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.937 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.937 INFO:teuthology.orchestra.run.smithi069.stdout: "config_check_secs": { 2024-04-25T11:37:41.937 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_check_secs", 2024-04-25T11:37:41.937 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.937 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.937 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.937 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-25T11:37:41.938 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "10", 2024-04-25T11:37:41.938 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.938 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.938 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T11:37:41.938 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.938 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.938 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.938 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.938 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.938 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.938 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.938 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.938 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.939 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.939 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.939 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.939 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.939 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.939 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.939 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.939 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.939 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.939 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.939 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.939 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.939 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.939 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.940 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.940 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.940 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.940 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.940 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.940 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.940 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.940 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.940 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.940 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.940 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.940 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.940 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.940 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.940 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.941 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.941 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.941 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.941 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.941 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.941 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.941 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.941 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.941 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.941 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.941 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.941 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.941 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.941 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.941 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.942 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.942 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.942 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.942 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.942 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.942 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.942 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.942 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.942 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.942 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.942 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.942 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.942 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.943 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.943 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.943 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.943 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.943 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.943 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.943 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.943 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.943 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.943 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.943 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.943 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.943 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.943 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.943 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.944 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.944 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.944 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.944 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.944 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.944 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.944 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.944 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.944 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.944 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.944 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "localpool", 2024-04-25T11:37:41.944 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.944 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.944 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.944 INFO:teuthology.orchestra.run.smithi069.stdout: "failure_domain": { 2024-04-25T11:37:41.945 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "failure_domain", 2024-04-25T11:37:41.945 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.945 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.945 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.945 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "host", 2024-04-25T11:37:41.945 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.945 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.945 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.945 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "failure domain for any created local pool", 2024-04-25T11:37:41.945 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T11:37:41.945 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.945 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.945 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.945 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.946 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.946 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.946 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.946 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.946 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.946 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.946 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.946 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.946 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.946 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.946 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.946 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.946 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.946 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.946 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.947 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.947 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.947 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.947 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.947 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.947 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.947 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.947 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.947 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.947 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.947 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.947 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.947 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.947 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.948 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.948 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.948 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.948 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.948 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.948 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.948 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.948 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.948 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.948 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.948 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.948 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.948 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.948 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.949 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.949 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.949 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.949 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.949 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.949 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.949 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.949 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.949 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.949 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.949 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.949 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.949 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.949 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.949 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.950 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.950 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.950 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.950 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.950 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.950 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.950 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.950 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.950 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.950 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.950 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.950 INFO:teuthology.orchestra.run.smithi069.stdout: "min_size": { 2024-04-25T11:37:41.950 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "min_size", 2024-04-25T11:37:41.951 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.951 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.951 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.951 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.951 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.951 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.951 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.951 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default min_size for any created local pool", 2024-04-25T11:37:41.951 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T11:37:41.951 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.951 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.951 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.951 INFO:teuthology.orchestra.run.smithi069.stdout: "num_rep": { 2024-04-25T11:37:41.951 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "num_rep", 2024-04-25T11:37:41.952 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.952 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.952 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.952 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3", 2024-04-25T11:37:41.952 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.952 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.952 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.952 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default replica count for any created local pool", 2024-04-25T11:37:41.952 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.952 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.952 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.952 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.952 INFO:teuthology.orchestra.run.smithi069.stdout: "pg_num": { 2024-04-25T11:37:41.952 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pg_num", 2024-04-25T11:37:41.952 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.953 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.953 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.953 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "128", 2024-04-25T11:37:41.953 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.953 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.953 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.953 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T11:37:41.953 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.953 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.953 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.953 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.953 INFO:teuthology.orchestra.run.smithi069.stdout: "prefix": { 2024-04-25T11:37:41.953 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prefix", 2024-04-25T11:37:41.953 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.954 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.954 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.954 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.954 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.954 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.954 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.954 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "name prefix for any created local pool", 2024-04-25T11:37:41.954 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.954 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.954 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.954 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.954 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.954 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.955 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.955 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.955 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.955 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.955 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.955 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.955 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.955 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.955 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.955 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.955 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.955 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.955 INFO:teuthology.orchestra.run.smithi069.stdout: "subtree": { 2024-04-25T11:37:41.955 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "subtree", 2024-04-25T11:37:41.956 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.956 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.956 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.956 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "rack", 2024-04-25T11:37:41.956 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.956 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.956 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.956 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T11:37:41.956 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T11:37:41.956 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.956 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.956 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.956 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.956 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.956 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.957 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mds_autoscaler", 2024-04-25T11:37:41.957 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.957 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.957 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.957 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.957 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.957 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.957 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.957 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.957 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.957 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.957 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.957 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.957 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.957 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.958 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.958 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.958 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.958 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.958 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.958 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.958 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.958 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.958 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.958 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.958 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.958 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.958 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.958 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.959 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.959 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.959 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.959 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.959 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.959 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.959 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.959 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.959 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.959 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.959 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.959 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.959 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.959 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.960 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.960 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.960 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.960 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.960 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.960 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.960 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.960 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.960 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.960 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.960 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.960 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.960 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.960 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.960 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.960 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.961 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.961 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.961 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.961 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.961 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.961 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.961 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.961 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.961 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.961 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.961 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.961 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.961 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.961 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.961 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.962 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.962 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.962 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.962 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.962 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.962 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.962 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.962 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.962 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.962 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.962 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.962 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.962 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.962 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.962 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.963 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.963 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.963 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mirroring", 2024-04-25T11:37:41.963 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.963 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.963 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.963 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.963 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.963 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.963 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.963 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.963 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.963 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.963 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.963 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.963 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.964 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.964 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.964 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.964 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.964 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.964 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.964 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.964 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.964 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.964 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.964 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.964 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.964 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.964 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.964 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.964 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.965 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.965 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.965 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.965 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.965 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.965 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.965 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.965 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.965 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.965 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.965 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.965 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.965 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.965 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.965 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.966 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.966 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.966 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.966 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.966 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.966 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.966 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.966 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.966 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.966 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.966 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.966 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.966 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.966 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.967 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.967 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.967 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.967 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.967 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.967 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.967 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.967 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.967 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.967 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.967 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.967 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.967 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.967 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.968 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.968 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.968 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.968 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.968 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.968 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.968 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.968 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.968 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.968 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.968 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.968 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.968 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.968 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.968 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.969 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.969 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.969 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.969 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "nfs", 2024-04-25T11:37:41.969 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.969 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.969 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.969 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.969 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.969 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.969 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.969 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.969 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.969 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.969 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.970 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.970 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.970 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.970 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.970 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.970 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.970 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.970 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.970 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.970 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.970 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.970 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.970 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.971 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.971 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.971 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.971 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.971 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.971 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.971 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.971 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.971 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.971 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.971 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.971 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.971 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.971 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.971 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.972 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.972 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.972 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.972 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.972 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.972 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.972 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.972 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.972 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.972 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.972 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.972 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.972 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.972 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.973 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.973 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.973 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.973 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.973 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.973 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.973 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.973 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.973 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.973 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.973 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.973 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.973 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.973 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.974 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.974 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.974 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.974 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.974 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.974 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.974 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.974 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.974 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.974 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.974 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.974 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.974 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.974 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.975 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.975 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.975 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.975 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.975 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.975 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.975 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.975 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.975 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.975 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "orchestrator", 2024-04-25T11:37:41.975 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.975 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.975 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.975 INFO:teuthology.orchestra.run.smithi069.stdout: "fail_fs": { 2024-04-25T11:37:41.976 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "fail_fs", 2024-04-25T11:37:41.976 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.976 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.976 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:41.976 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.976 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.976 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.976 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.976 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T11:37:41.976 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.976 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.976 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.976 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.976 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.976 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.977 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.977 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.977 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.977 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.977 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.977 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.977 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.977 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.977 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.977 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.977 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.977 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.977 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.977 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.978 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.978 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.978 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.978 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.978 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.978 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.978 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.978 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.978 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.978 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.978 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.978 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.978 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.978 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.979 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.979 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.979 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.979 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.979 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.979 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.979 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.979 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.979 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.979 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.979 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.979 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.979 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.979 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.980 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.980 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.980 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.980 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.980 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.980 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.980 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.980 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.980 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.980 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.980 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.980 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.980 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.980 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.980 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.981 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.981 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.981 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.981 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.981 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.981 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.981 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.981 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.981 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.981 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.981 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.981 INFO:teuthology.orchestra.run.smithi069.stdout: "orchestrator": { 2024-04-25T11:37:41.981 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "orchestrator", 2024-04-25T11:37:41.981 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.982 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.982 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.982 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.982 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.982 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.982 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.982 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm", 2024-04-25T11:37:41.982 INFO:teuthology.orchestra.run.smithi069.stdout: "rook", 2024-04-25T11:37:41.982 INFO:teuthology.orchestra.run.smithi069.stdout: "test_orchestrator" 2024-04-25T11:37:41.982 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.982 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Orchestrator backend", 2024-04-25T11:37:41.982 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.982 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.982 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.983 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.983 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.983 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.983 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.983 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.983 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.983 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.983 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.983 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.983 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.983 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.983 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.983 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.983 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.983 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.984 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.984 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.984 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.984 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "osd_perf_query", 2024-04-25T11:37:41.984 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.984 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.984 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.984 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.984 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.984 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.984 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.984 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.984 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.984 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.984 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.985 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.985 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.985 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.985 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.985 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.985 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.985 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.985 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.985 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.985 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.985 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.985 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.985 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.986 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.986 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.986 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.986 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.986 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.986 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.986 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.986 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.986 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.986 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.986 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.986 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.986 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.986 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.987 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.987 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.987 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.987 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.987 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.987 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.987 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.987 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.987 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.987 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.987 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.987 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.987 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.987 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.987 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.988 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.988 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.988 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.988 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.988 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.988 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.988 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.988 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.988 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.988 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.988 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.988 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.988 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.988 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.989 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.989 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.989 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.989 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.989 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.989 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.989 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.989 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.989 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.989 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.989 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.989 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.989 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.990 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.990 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.990 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.990 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.990 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.990 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.990 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.990 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.990 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.990 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.990 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "osd_support", 2024-04-25T11:37:41.990 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.990 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.990 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.991 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.991 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.991 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.991 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.991 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.991 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.991 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.991 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.991 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.991 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.991 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.991 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.992 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.992 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.992 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.992 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.992 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.992 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.992 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.992 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.992 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.992 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.992 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.992 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.992 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.992 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.992 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.993 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.993 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.993 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.993 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.993 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.993 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.993 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.993 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.993 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.993 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:41.993 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.993 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.993 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.994 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:41.994 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.994 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.994 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.994 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.994 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.994 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.994 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.994 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.994 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.994 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.994 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.994 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.994 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.994 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.995 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.995 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:41.995 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:41.995 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.995 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.995 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.995 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.995 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.995 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.995 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.995 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.995 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.995 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.995 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.995 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.995 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:41.996 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:41.996 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:41.996 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:41.996 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.996 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:41.996 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.996 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.996 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.996 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.996 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.996 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.996 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.996 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.996 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:41.997 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.997 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:41.997 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pg_autoscaler", 2024-04-25T11:37:41.997 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:41.997 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:41.997 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:41.997 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:41.997 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:41.997 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:41.997 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.997 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.997 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:41.997 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.997 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.997 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:41.998 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:41.998 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:41.998 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:41.998 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:41.998 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:41.998 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:41.998 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:41.998 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.998 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.998 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.998 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.998 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.998 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:41.998 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:41.999 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:41.999 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:41.999 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:41.999 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:41.999 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:41.999 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:41.999 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:41.999 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:41.999 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:41.999 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:41.999 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:41.999 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:41.999 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:41.999 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:42.000 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.000 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.000 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.000 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:42.000 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.000 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.000 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:42.000 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:42.000 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:42.000 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:42.000 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:42.000 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:42.000 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:42.000 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:42.000 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.000 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.001 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.001 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.001 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.001 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:42.001 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:42.001 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.001 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.001 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.001 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:42.001 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.001 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.001 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.001 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.001 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.001 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.002 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.002 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.002 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-04-25T11:37:42.002 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-04-25T11:37:42.002 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:42.002 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.002 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.002 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-04-25T11:37:42.002 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.002 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.002 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.002 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.002 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.003 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.003 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.003 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.003 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:42.003 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:42.003 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:42.003 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:42.003 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.003 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:42.003 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.003 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.003 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.003 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.003 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.003 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.004 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.004 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.004 INFO:teuthology.orchestra.run.smithi069.stdout: "threshold": { 2024-04-25T11:37:42.004 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "threshold", 2024-04-25T11:37:42.004 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-04-25T11:37:42.004 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.004 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.004 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3.0", 2024-04-25T11:37:42.004 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1.0", 2024-04-25T11:37:42.004 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.004 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.004 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "scaling threshold", 2024-04-25T11:37:42.004 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T11:37:42.004 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.004 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.005 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:42.005 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:42.005 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.005 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:42.005 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "progress", 2024-04-25T11:37:42.005 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:42.005 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:42.005 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:42.005 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_pg_recovery_event": { 2024-04-25T11:37:42.005 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_pg_recovery_event", 2024-04-25T11:37:42.005 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.005 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.005 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.005 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:42.005 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.006 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.006 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.006 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T11:37:42.006 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.006 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.006 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.006 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.006 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": { 2024-04-25T11:37:42.006 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enabled", 2024-04-25T11:37:42.006 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.006 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.006 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.006 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:42.006 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.007 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.007 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.007 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.007 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.007 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.007 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.007 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.007 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:42.007 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:42.007 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.007 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.007 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.007 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:42.007 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.008 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.008 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:42.008 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:42.008 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:42.008 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:42.008 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:42.008 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:42.008 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:42.008 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:42.008 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.008 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.008 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.008 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.008 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.009 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:42.009 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:42.009 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.009 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.009 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.009 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:42.009 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.009 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.009 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.009 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.009 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.009 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.009 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.009 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.010 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:42.010 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:42.010 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.010 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.010 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.010 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:42.010 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.010 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.010 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:42.010 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:42.010 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:42.010 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:42.010 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:42.011 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:42.011 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:42.011 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:42.011 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.011 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.011 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.011 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.011 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.011 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:42.011 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:42.011 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.011 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.011 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.011 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:42.011 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.011 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.012 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.012 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.012 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.012 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.012 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.012 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.012 INFO:teuthology.orchestra.run.smithi069.stdout: "max_completed_events": { 2024-04-25T11:37:42.012 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_completed_events", 2024-04-25T11:37:42.012 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:42.012 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.012 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.012 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "50", 2024-04-25T11:37:42.012 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.012 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.012 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.013 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "number of past completed events to remember", 2024-04-25T11:37:42.013 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.013 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.013 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.013 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.013 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-04-25T11:37:42.013 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-04-25T11:37:42.013 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:42.013 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.013 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.013 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-04-25T11:37:42.013 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.013 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.013 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.014 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long the module is going to sleep", 2024-04-25T11:37:42.014 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.014 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.014 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.014 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.014 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:42.014 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:42.014 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:42.014 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:42.014 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.014 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:42.014 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.014 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.014 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.015 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.015 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.015 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.015 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.015 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:42.015 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:42.015 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.015 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:42.015 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prometheus", 2024-04-25T11:37:42.015 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:42.015 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:42.015 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:42.015 INFO:teuthology.orchestra.run.smithi069.stdout: "cache": { 2024-04-25T11:37:42.015 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cache", 2024-04-25T11:37:42.016 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.016 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.016 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.016 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:42.016 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.016 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.016 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.016 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.016 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.016 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.016 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.016 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.016 INFO:teuthology.orchestra.run.smithi069.stdout: "exclude_perf_counters": { 2024-04-25T11:37:42.016 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "exclude_perf_counters", 2024-04-25T11:37:42.016 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.017 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.017 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.017 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:42.017 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.017 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.017 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.017 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T11:37:42.017 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T11:37:42.017 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.017 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.017 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.017 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:42.017 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:42.017 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.017 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.018 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.018 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:42.018 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.018 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.018 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:42.018 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:42.018 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:42.018 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:42.018 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:42.018 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:42.018 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:42.018 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:42.018 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.018 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.019 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.019 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.019 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.019 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:42.019 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:42.019 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.019 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.019 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.019 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:42.019 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.019 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.019 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.019 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.019 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.019 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.020 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.020 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.020 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:42.020 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:42.020 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.020 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.020 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.020 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:42.020 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.020 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.020 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:42.020 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:42.020 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:42.020 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:42.021 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:42.021 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:42.021 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:42.021 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:42.021 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.021 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.021 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.021 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.021 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.021 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:42.021 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:42.021 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.021 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.021 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.021 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:42.022 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.022 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.022 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.022 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.022 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.022 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.022 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.022 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.022 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_stats_pools": { 2024-04-25T11:37:42.022 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_stats_pools", 2024-04-25T11:37:42.022 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.022 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.022 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.022 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:42.022 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.023 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.023 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.023 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.023 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.023 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.023 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.023 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.023 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T11:37:42.023 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T11:37:42.023 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:42.023 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.023 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.023 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "300", 2024-04-25T11:37:42.023 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.024 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.024 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.024 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.024 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.024 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.024 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.024 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.024 INFO:teuthology.orchestra.run.smithi069.stdout: "scrape_interval": { 2024-04-25T11:37:42.024 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "scrape_interval", 2024-04-25T11:37:42.024 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-04-25T11:37:42.024 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.024 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.024 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15.0", 2024-04-25T11:37:42.024 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.024 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.025 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.025 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.025 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.025 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.025 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.025 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.025 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2024-04-25T11:37:42.025 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2024-04-25T11:37:42.025 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.025 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.025 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.025 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "::", 2024-04-25T11:37:42.025 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.025 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.025 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.026 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T11:37:42.026 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.026 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.026 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.026 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.026 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2024-04-25T11:37:42.026 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2024-04-25T11:37:42.026 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:42.026 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.026 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.026 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "9283", 2024-04-25T11:37:42.026 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.026 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.026 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.026 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T11:37:42.027 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.027 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.027 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.027 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.027 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:42.027 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:42.027 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:42.027 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:42.027 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.027 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:42.027 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.027 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.027 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.027 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.027 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.028 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.028 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.028 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.028 INFO:teuthology.orchestra.run.smithi069.stdout: "stale_cache_strategy": { 2024-04-25T11:37:42.028 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "stale_cache_strategy", 2024-04-25T11:37:42.028 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.028 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.028 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.028 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "log", 2024-04-25T11:37:42.028 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.028 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.028 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.028 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.028 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.029 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.029 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.029 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.029 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_behaviour": { 2024-04-25T11:37:42.029 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_behaviour", 2024-04-25T11:37:42.029 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.029 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.029 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.029 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "default", 2024-04-25T11:37:42.029 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.029 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.029 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:42.029 INFO:teuthology.orchestra.run.smithi069.stdout: "default", 2024-04-25T11:37:42.029 INFO:teuthology.orchestra.run.smithi069.stdout: "error" 2024-04-25T11:37:42.030 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:42.030 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.030 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.030 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.030 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.030 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.030 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_error_status_code": { 2024-04-25T11:37:42.030 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_error_status_code", 2024-04-25T11:37:42.030 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:42.030 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.030 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.030 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "500", 2024-04-25T11:37:42.030 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "400", 2024-04-25T11:37:42.031 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "599", 2024-04-25T11:37:42.031 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.031 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.031 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.031 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.031 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.031 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:42.031 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:42.031 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.031 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:42.031 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_support", 2024-04-25T11:37:42.031 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:42.031 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:42.031 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:42.032 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:42.032 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:42.032 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.032 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.032 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.032 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:42.032 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.032 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.032 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:42.032 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:42.032 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:42.032 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:42.032 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:42.032 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:42.032 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:42.033 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:42.033 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.033 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.033 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.033 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.033 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.033 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:42.033 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:42.033 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.033 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.033 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.033 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:42.033 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.033 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.033 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.034 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.034 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.034 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.034 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.034 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.034 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:42.034 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:42.034 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.034 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.034 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.034 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:42.034 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.034 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.034 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:42.034 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:42.035 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:42.035 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:42.035 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:42.035 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:42.035 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:42.035 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:42.035 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.035 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.035 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.035 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.035 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.035 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:42.035 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:42.035 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.035 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.036 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.036 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:42.036 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.036 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.036 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.036 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.036 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.036 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.036 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.036 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.036 INFO:teuthology.orchestra.run.smithi069.stdout: "max_concurrent_snap_create": { 2024-04-25T11:37:42.036 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_concurrent_snap_create", 2024-04-25T11:37:42.036 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:42.036 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.036 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.037 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-04-25T11:37:42.037 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.037 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.037 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.037 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.037 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.037 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.037 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.037 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.037 INFO:teuthology.orchestra.run.smithi069.stdout: "mirror_snapshot_schedule": { 2024-04-25T11:37:42.037 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T11:37:42.037 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.037 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.037 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.037 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:42.038 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.038 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.038 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.038 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.038 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.038 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.038 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.038 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.038 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:42.038 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:42.038 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:42.038 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:42.038 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.039 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:42.039 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.039 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.039 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.039 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.039 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.039 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.039 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.039 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.039 INFO:teuthology.orchestra.run.smithi069.stdout: "trash_purge_schedule": { 2024-04-25T11:37:42.039 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "trash_purge_schedule", 2024-04-25T11:37:42.039 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.039 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.039 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.039 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:42.040 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.040 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.040 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.040 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.040 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.040 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.040 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.040 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:42.040 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:42.040 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.040 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:42.040 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "restful", 2024-04-25T11:37:42.040 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:42.040 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:42.041 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:42.041 INFO:teuthology.orchestra.run.smithi069.stdout: "enable_auth": { 2024-04-25T11:37:42.041 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enable_auth", 2024-04-25T11:37:42.041 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.041 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.041 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.041 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:42.041 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.041 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.041 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.041 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.041 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.041 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.041 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.041 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.042 INFO:teuthology.orchestra.run.smithi069.stdout: "key_file": { 2024-04-25T11:37:42.042 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "key_file", 2024-04-25T11:37:42.042 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.042 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.042 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.042 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:42.042 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.042 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.042 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.042 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.042 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.042 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.042 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.042 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.042 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:42.043 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:42.043 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.043 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.043 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.043 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:42.043 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.043 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.043 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:42.043 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:42.043 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:42.043 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:42.043 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:42.043 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:42.043 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:42.043 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:42.043 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.044 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.044 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.044 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.044 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.044 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:42.044 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:42.044 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.044 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.044 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.044 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:42.044 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.044 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.044 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.045 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.045 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.045 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.045 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.045 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.045 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:42.045 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:42.045 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.045 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.045 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.045 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:42.045 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.045 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.045 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:42.045 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:42.046 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:42.046 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:42.046 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:42.046 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:42.046 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:42.046 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:42.046 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.046 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.046 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.046 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.046 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.046 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:42.047 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:42.047 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.047 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.047 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.047 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:42.047 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.047 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.047 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.047 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.047 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.047 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.047 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.047 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.047 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2024-04-25T11:37:42.047 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2024-04-25T11:37:42.048 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.048 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.048 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.048 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:42.048 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.048 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.048 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.048 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.048 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.048 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.048 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.048 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.048 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2024-04-25T11:37:42.048 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2024-04-25T11:37:42.049 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.049 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.049 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.049 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:42.049 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.049 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.049 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.049 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.049 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.049 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.049 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.049 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.049 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:42.049 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:42.050 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:42.050 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:42.050 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.050 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:42.050 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.050 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.050 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.050 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.050 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.050 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.050 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.050 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:42.050 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:42.050 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.051 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:42.051 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rgw", 2024-04-25T11:37:42.051 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:42.051 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:42.051 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:42.051 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:42.051 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:42.051 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.051 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.051 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.051 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:42.051 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.051 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.052 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:42.052 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:42.052 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:42.052 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:42.052 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:42.052 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:42.052 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:42.052 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:42.052 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.052 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.052 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.052 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.052 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.052 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:42.052 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:42.053 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.053 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.053 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.053 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:42.053 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.053 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.053 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.053 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.053 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.053 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.053 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.053 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.053 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:42.053 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:42.054 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.054 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.054 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.054 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:42.054 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.054 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.054 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:42.054 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:42.054 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:42.054 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:42.054 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:42.054 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:42.054 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:42.055 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:42.055 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.055 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.055 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.055 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.055 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.055 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:42.055 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:42.055 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.055 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.055 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.055 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:42.055 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.055 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.055 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.055 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.056 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.056 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.056 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.056 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.056 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:42.056 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:42.056 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:42.056 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:42.056 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.056 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:42.056 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.056 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.056 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.056 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.056 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.056 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.057 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.057 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:42.057 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:42.057 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.057 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:42.057 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rook", 2024-04-25T11:37:42.057 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:42.057 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:42.057 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:42.057 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:42.057 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:42.057 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.057 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.058 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.058 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:42.058 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.058 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.058 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:42.058 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:42.058 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:42.058 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:42.058 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:42.058 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:42.058 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:42.058 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:42.058 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.058 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.058 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.059 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.059 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.059 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:42.059 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:42.059 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.059 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.059 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.059 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:42.059 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.059 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.059 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.059 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.059 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.059 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.060 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.060 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.060 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:42.060 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:42.060 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.060 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.060 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.060 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:42.060 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.060 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.060 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:42.060 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:42.060 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:42.060 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:42.061 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:42.061 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:42.061 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:42.061 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:42.061 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.061 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.061 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.061 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.061 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.061 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:42.061 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:42.061 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.061 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.061 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.061 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:42.062 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.062 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.062 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.062 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.062 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.062 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.062 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.062 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.062 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:42.062 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:42.062 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:42.062 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:42.062 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.062 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:42.063 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.063 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.063 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.063 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.063 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.063 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.063 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.063 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.063 INFO:teuthology.orchestra.run.smithi069.stdout: "storage_class": { 2024-04-25T11:37:42.063 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "storage_class", 2024-04-25T11:37:42.063 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.063 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.063 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.063 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "local", 2024-04-25T11:37:42.063 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.063 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.064 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.064 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T11:37:42.064 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.064 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.064 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.064 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:42.064 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:42.064 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.064 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:42.064 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "selftest", 2024-04-25T11:37:42.064 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:42.064 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:42.064 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:42.064 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:42.064 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:42.065 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.065 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.065 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.065 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:42.065 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.065 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.065 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:42.065 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:42.065 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:42.065 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:42.065 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:42.065 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:42.065 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:42.065 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:42.065 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.066 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.066 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.066 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.066 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.066 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:42.066 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:42.066 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.066 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.066 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.066 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:42.066 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.066 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.066 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.066 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.066 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.067 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.067 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.067 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.067 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:42.067 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:42.067 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.067 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.067 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.067 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:42.067 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.067 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.067 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:42.067 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:42.067 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:42.067 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:42.067 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:42.068 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:42.068 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:42.068 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:42.068 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.068 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.068 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.068 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.068 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.068 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:42.068 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:42.068 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.068 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.068 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.068 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:42.068 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.069 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.069 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.069 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.069 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.069 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.069 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.069 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.069 INFO:teuthology.orchestra.run.smithi069.stdout: "roption1": { 2024-04-25T11:37:42.069 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "roption1", 2024-04-25T11:37:42.069 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.069 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.069 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.069 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:42.069 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.070 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.070 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.070 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.070 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.070 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.070 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.070 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.070 INFO:teuthology.orchestra.run.smithi069.stdout: "roption2": { 2024-04-25T11:37:42.070 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "roption2", 2024-04-25T11:37:42.070 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.070 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.070 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.070 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "xyz", 2024-04-25T11:37:42.070 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.071 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.071 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.071 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.071 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.071 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.071 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.071 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.071 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption1": { 2024-04-25T11:37:42.071 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption1", 2024-04-25T11:37:42.071 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.071 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.071 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.071 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:42.071 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.071 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.071 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.072 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.072 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.072 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.072 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.072 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.072 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption2": { 2024-04-25T11:37:42.072 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption2", 2024-04-25T11:37:42.072 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:42.072 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.072 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.072 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:42.072 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.072 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.072 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.072 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.073 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.073 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.073 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.073 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.073 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption3": { 2024-04-25T11:37:42.073 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption3", 2024-04-25T11:37:42.073 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-04-25T11:37:42.073 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.073 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.073 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:42.073 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.073 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.073 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.073 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.073 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.074 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.074 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.074 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.074 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption4": { 2024-04-25T11:37:42.074 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption4", 2024-04-25T11:37:42.074 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.074 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.074 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.074 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:42.074 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.074 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.074 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.074 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.074 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.075 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.075 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.075 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.075 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption5": { 2024-04-25T11:37:42.075 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption5", 2024-04-25T11:37:42.075 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.075 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.075 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.075 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:42.075 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.075 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.075 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.075 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.075 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.076 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.076 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.076 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.076 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption6": { 2024-04-25T11:37:42.076 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption6", 2024-04-25T11:37:42.076 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.076 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.076 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.076 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:42.076 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.076 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.076 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.076 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.077 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.077 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.077 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.077 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.077 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption7": { 2024-04-25T11:37:42.077 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption7", 2024-04-25T11:37:42.077 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:42.077 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.077 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.077 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:42.077 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-04-25T11:37:42.077 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "42", 2024-04-25T11:37:42.077 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.077 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.077 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.077 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.078 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.078 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.078 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:42.078 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:42.078 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:42.078 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:42.078 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.078 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:42.078 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.078 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.078 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.078 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.078 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.078 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.078 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.079 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.079 INFO:teuthology.orchestra.run.smithi069.stdout: "testkey": { 2024-04-25T11:37:42.079 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testkey", 2024-04-25T11:37:42.079 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.079 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.079 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.079 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:42.079 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.079 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.079 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.079 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.079 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.079 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.079 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.080 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.080 INFO:teuthology.orchestra.run.smithi069.stdout: "testlkey": { 2024-04-25T11:37:42.080 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testlkey", 2024-04-25T11:37:42.080 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.080 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.080 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.080 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:42.080 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.080 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.080 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.080 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.080 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.080 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.081 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.081 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.081 INFO:teuthology.orchestra.run.smithi069.stdout: "testnewline": { 2024-04-25T11:37:42.081 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testnewline", 2024-04-25T11:37:42.081 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.081 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.081 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.081 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:42.081 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.081 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.081 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.081 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.081 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.081 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.082 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.082 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:42.082 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:42.082 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.082 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:42.082 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snap_schedule", 2024-04-25T11:37:42.082 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:42.082 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:42.082 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:42.082 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_m_granularity": { 2024-04-25T11:37:42.082 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_m_granularity", 2024-04-25T11:37:42.082 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.082 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.083 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.083 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:42.083 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.083 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.083 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.083 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T11:37:42.083 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.083 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.083 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.083 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.083 INFO:teuthology.orchestra.run.smithi069.stdout: "dump_on_update": { 2024-04-25T11:37:42.083 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "dump_on_update", 2024-04-25T11:37:42.083 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.083 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.083 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.083 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:42.083 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.084 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.084 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.084 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "dump database to debug log on update", 2024-04-25T11:37:42.084 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.084 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.084 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.084 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.084 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:42.084 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:42.084 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.084 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.084 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.084 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:42.084 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.084 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.084 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:42.085 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:42.085 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:42.085 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:42.085 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:42.085 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:42.085 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:42.085 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:42.085 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.085 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.085 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.085 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.085 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.085 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:42.085 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:42.086 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.086 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.086 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.086 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:42.086 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.086 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.086 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.086 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.086 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.086 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.086 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.086 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.086 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:42.086 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:42.087 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.087 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.087 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.087 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:42.087 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.087 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.087 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:42.087 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:42.087 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:42.087 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:42.087 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:42.087 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:42.087 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:42.088 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:42.088 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.088 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.088 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.088 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.088 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.088 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:42.088 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:42.088 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.088 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.088 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.088 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:42.088 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.088 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.088 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.088 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.089 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.089 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.089 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.089 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.089 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:42.089 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:42.089 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:42.089 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:42.089 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.089 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:42.089 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.089 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.089 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.089 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.089 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.090 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.090 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.090 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:42.090 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:42.090 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.090 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:42.090 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "stats", 2024-04-25T11:37:42.090 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:42.090 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:42.090 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:42.090 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:42.090 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:42.090 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.090 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.090 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.091 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:42.091 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.091 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.091 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:42.091 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:42.091 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:42.091 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:42.091 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:42.091 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:42.091 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:42.091 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:42.091 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.091 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.091 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.092 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.092 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.092 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:42.092 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:42.092 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.092 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.092 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.092 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:42.092 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.092 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.092 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.092 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.092 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.092 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.093 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.093 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.093 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:42.093 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:42.093 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.093 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.093 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.093 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:42.093 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.093 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.093 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:42.093 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:42.093 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:42.093 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:42.093 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:42.093 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:42.093 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:42.094 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:42.094 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.094 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.094 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.094 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.094 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.094 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:42.094 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:42.094 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.094 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.094 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.094 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:42.094 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.094 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.095 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.095 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.095 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.095 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.095 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.095 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.095 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:42.095 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:42.095 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:42.095 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:42.095 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.095 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:42.095 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.095 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.095 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.096 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.096 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.096 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.096 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.096 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:42.096 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:42.096 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.096 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:42.096 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "status", 2024-04-25T11:37:42.096 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:42.096 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:42.096 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:42.096 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:42.096 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:42.097 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.097 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.097 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.097 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:42.097 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.097 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.097 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:42.097 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:42.097 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:42.097 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:42.097 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:42.097 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:42.097 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:42.097 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:42.097 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.097 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.098 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.098 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.098 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.098 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:42.098 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:42.098 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.098 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.098 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.098 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:42.098 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.098 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.098 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.098 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.098 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.098 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.099 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.099 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.099 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:42.099 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:42.099 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.099 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.099 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.099 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:42.099 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.099 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.099 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:42.099 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:42.099 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:42.100 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:42.100 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:42.100 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:42.100 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:42.100 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:42.100 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.100 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.100 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.100 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.100 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.100 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:42.100 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:42.100 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.100 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.100 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.101 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:42.101 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.101 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.101 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.101 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.101 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.101 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.101 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.101 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.101 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:42.101 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:42.101 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:42.101 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:42.101 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.102 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:42.102 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.102 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.102 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.102 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.102 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.102 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.102 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.102 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:42.102 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:42.102 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.102 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:42.102 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "telegraf", 2024-04-25T11:37:42.102 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:42.102 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:42.103 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:42.103 INFO:teuthology.orchestra.run.smithi069.stdout: "address": { 2024-04-25T11:37:42.103 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "address", 2024-04-25T11:37:42.103 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.103 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.103 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.103 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T11:37:42.103 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.103 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.103 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.103 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.103 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.103 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.103 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.103 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.103 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-04-25T11:37:42.104 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-04-25T11:37:42.104 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:42.104 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.104 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.104 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15", 2024-04-25T11:37:42.104 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.104 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.104 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.104 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.104 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.104 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.104 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.104 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.104 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:42.105 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:42.105 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.105 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.105 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.105 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:42.105 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.105 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.105 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:42.105 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:42.105 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:42.105 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:42.105 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:42.105 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:42.105 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:42.105 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:42.106 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.106 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.106 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.106 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.106 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.106 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:42.106 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:42.106 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.106 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.106 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.106 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:42.106 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.106 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.106 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.107 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.107 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.107 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.107 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.107 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.107 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:42.107 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:42.107 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.107 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.107 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.107 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:42.107 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.107 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.107 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:42.107 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:42.108 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:42.108 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:42.108 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:42.108 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:42.108 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:42.108 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:42.108 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.108 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.108 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.108 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.108 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.108 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:42.108 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:42.108 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.109 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.109 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.109 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:42.109 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.109 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.109 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.109 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.109 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.109 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.109 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.109 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.109 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:42.109 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:42.109 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:42.110 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:42.110 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.110 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:42.110 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.110 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.110 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.110 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.110 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.110 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.110 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.110 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:42.110 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:42.110 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.110 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:42.110 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "telemetry", 2024-04-25T11:37:42.111 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:42.111 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:42.111 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:42.111 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_basic": { 2024-04-25T11:37:42.111 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_basic", 2024-04-25T11:37:42.111 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.111 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.111 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.111 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:42.111 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.111 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.111 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.111 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T11:37:42.111 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.112 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.112 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.112 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.112 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_crash": { 2024-04-25T11:37:42.112 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_crash", 2024-04-25T11:37:42.112 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.112 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.112 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.112 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:42.112 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.112 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.112 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.112 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T11:37:42.112 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.112 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.113 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.113 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.113 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_device": { 2024-04-25T11:37:42.113 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_device", 2024-04-25T11:37:42.113 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.113 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.113 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.113 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:42.113 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.113 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.113 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.113 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T11:37:42.113 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.113 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.114 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.114 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.114 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_ident": { 2024-04-25T11:37:42.114 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_ident", 2024-04-25T11:37:42.114 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.114 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.114 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.114 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:42.114 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.114 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.114 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.114 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T11:37:42.114 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.114 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.114 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.115 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.115 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_perf": { 2024-04-25T11:37:42.115 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_perf", 2024-04-25T11:37:42.115 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.115 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.115 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.115 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:42.115 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.115 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.115 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.115 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T11:37:42.115 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.115 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.115 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.115 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.116 INFO:teuthology.orchestra.run.smithi069.stdout: "contact": { 2024-04-25T11:37:42.116 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "contact", 2024-04-25T11:37:42.116 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.116 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.116 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.116 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:42.116 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.116 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.116 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.116 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.116 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.116 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.116 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.116 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.117 INFO:teuthology.orchestra.run.smithi069.stdout: "description": { 2024-04-25T11:37:42.117 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "description", 2024-04-25T11:37:42.117 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.117 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.117 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.117 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:42.117 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.117 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.117 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.117 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.117 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.117 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.117 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.117 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.117 INFO:teuthology.orchestra.run.smithi069.stdout: "device_url": { 2024-04-25T11:37:42.118 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_url", 2024-04-25T11:37:42.118 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.118 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.118 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.118 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T11:37:42.118 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.118 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.118 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.118 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.118 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.118 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.118 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.119 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.119 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": { 2024-04-25T11:37:42.119 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enabled", 2024-04-25T11:37:42.119 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.119 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.119 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.119 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:42.119 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.119 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.119 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.119 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.119 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.119 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.120 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.120 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.120 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-04-25T11:37:42.120 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-04-25T11:37:42.120 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:42.120 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.120 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.120 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "24", 2024-04-25T11:37:42.120 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "8", 2024-04-25T11:37:42.120 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.120 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.120 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.120 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.120 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.120 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.121 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.121 INFO:teuthology.orchestra.run.smithi069.stdout: "last_opt_revision": { 2024-04-25T11:37:42.121 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "last_opt_revision", 2024-04-25T11:37:42.121 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:42.121 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.121 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.121 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1", 2024-04-25T11:37:42.121 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.121 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.121 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.121 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.121 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.121 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.121 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.122 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.122 INFO:teuthology.orchestra.run.smithi069.stdout: "leaderboard": { 2024-04-25T11:37:42.122 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "leaderboard", 2024-04-25T11:37:42.122 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.122 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.122 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.122 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:42.122 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.122 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.122 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.122 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.122 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.122 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.122 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.122 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.123 INFO:teuthology.orchestra.run.smithi069.stdout: "leaderboard_description": { 2024-04-25T11:37:42.123 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "leaderboard_description", 2024-04-25T11:37:42.123 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.123 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.123 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.123 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:42.123 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.123 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.123 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.123 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.123 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.123 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.123 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.123 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.123 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:42.124 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:42.124 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.124 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.124 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.124 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:42.124 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.124 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.124 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:42.124 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:42.124 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:42.124 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:42.124 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:42.124 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:42.124 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:42.125 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:42.125 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.125 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.125 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.125 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.125 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.125 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:42.125 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:42.125 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.125 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.125 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.125 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:42.125 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.125 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.125 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.126 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.126 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.126 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.126 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.126 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.126 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:42.126 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:42.126 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.126 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.126 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.126 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:42.126 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.126 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.126 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:42.127 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:42.127 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:42.127 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:42.127 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:42.127 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:42.127 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:42.127 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:42.127 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.127 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.127 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.127 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.127 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.127 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:42.127 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:42.127 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.128 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.128 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.128 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:42.128 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.128 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.128 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.128 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.128 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.128 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.128 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.128 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.128 INFO:teuthology.orchestra.run.smithi069.stdout: "organization": { 2024-04-25T11:37:42.128 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "organization", 2024-04-25T11:37:42.128 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.129 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.129 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.129 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:42.129 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.129 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.129 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.129 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.129 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.129 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.129 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.129 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.129 INFO:teuthology.orchestra.run.smithi069.stdout: "proxy": { 2024-04-25T11:37:42.129 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "proxy", 2024-04-25T11:37:42.129 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.129 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.130 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.130 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:42.130 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.130 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.130 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.130 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.130 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.130 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.130 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.130 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.130 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:42.130 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:42.130 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:42.131 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:42.131 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.131 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:42.131 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.131 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.131 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.131 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.131 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.131 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.131 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.131 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.131 INFO:teuthology.orchestra.run.smithi069.stdout: "url": { 2024-04-25T11:37:42.131 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "url", 2024-04-25T11:37:42.131 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.131 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.131 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.132 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T11:37:42.132 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.132 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.132 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.132 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.132 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.132 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.132 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.132 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:42.132 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:42.132 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.132 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:42.132 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "test_orchestrator", 2024-04-25T11:37:42.132 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:42.133 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:42.133 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:42.133 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:42.133 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:42.133 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.133 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.133 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.133 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:42.133 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.133 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.133 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:42.133 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:42.133 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:42.133 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:42.133 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:42.134 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:42.134 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:42.134 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:42.134 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.134 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.134 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.134 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.134 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.134 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:42.134 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:42.134 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.134 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.134 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.134 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:42.135 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.135 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.135 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.135 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.135 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.135 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.135 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.135 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.135 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:42.135 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:42.135 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.135 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.135 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.135 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:42.135 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.136 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.136 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:42.136 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:42.136 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:42.136 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:42.136 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:42.136 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:42.136 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:42.136 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:42.136 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.136 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.136 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.136 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.136 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.137 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:42.137 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:42.137 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.137 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.137 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.137 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:42.137 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.137 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.137 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.137 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.137 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.137 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.137 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.137 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.137 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:42.138 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:42.138 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:42.138 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:42.138 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.138 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:42.138 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.138 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.138 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.138 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.138 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.138 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.138 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.138 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:42.138 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:42.139 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.139 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:42.139 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "volumes", 2024-04-25T11:37:42.139 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:42.139 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:42.139 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:42.139 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:42.139 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:42.139 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.139 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.139 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.139 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:42.139 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.139 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.139 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:42.140 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:42.140 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:42.140 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:42.140 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:42.140 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:42.140 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:42.140 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:42.140 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.140 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.140 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.140 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.140 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.140 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:42.141 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:42.141 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.141 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.141 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.141 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:42.141 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.141 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.141 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.141 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.141 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.141 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.141 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.141 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.141 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:42.141 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:42.142 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.142 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.142 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.142 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:42.142 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.142 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.142 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:42.142 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:42.142 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:42.142 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:42.142 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:42.142 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:42.142 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:42.142 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:42.143 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.143 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.143 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.143 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.143 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.143 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:42.143 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:42.143 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.143 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.143 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.143 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:42.143 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.143 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.143 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.143 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.144 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.144 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.144 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.144 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.144 INFO:teuthology.orchestra.run.smithi069.stdout: "max_concurrent_clones": { 2024-04-25T11:37:42.144 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_concurrent_clones", 2024-04-25T11:37:42.144 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:42.144 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.144 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.144 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "4", 2024-04-25T11:37:42.144 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.144 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.144 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.144 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T11:37:42.144 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.145 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.145 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.145 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.145 INFO:teuthology.orchestra.run.smithi069.stdout: "periodic_async_work": { 2024-04-25T11:37:42.145 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "periodic_async_work", 2024-04-25T11:37:42.145 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.145 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.145 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.145 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:42.145 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.145 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.145 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.145 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Periodically check for async work", 2024-04-25T11:37:42.145 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.145 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.146 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.146 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.146 INFO:teuthology.orchestra.run.smithi069.stdout: "snapshot_clone_delay": { 2024-04-25T11:37:42.146 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snapshot_clone_delay", 2024-04-25T11:37:42.146 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:42.146 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.146 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.146 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:42.146 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.146 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.146 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.146 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T11:37:42.146 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.146 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.146 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.147 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.147 INFO:teuthology.orchestra.run.smithi069.stdout: "snapshot_clone_no_wait": { 2024-04-25T11:37:42.147 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T11:37:42.147 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.147 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.147 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.147 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-04-25T11:37:42.147 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.147 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.147 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.147 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T11:37:42.147 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.147 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.147 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.147 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.148 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:42.148 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:42.148 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:42.148 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:42.148 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.148 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:42.148 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.148 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.148 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.148 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.148 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.148 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.148 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.148 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:42.148 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:42.148 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.149 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:42.149 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix", 2024-04-25T11:37:42.149 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-04-25T11:37:42.149 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-04-25T11:37:42.149 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-04-25T11:37:42.149 INFO:teuthology.orchestra.run.smithi069.stdout: "discovery_interval": { 2024-04-25T11:37:42.149 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "discovery_interval", 2024-04-25T11:37:42.149 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-04-25T11:37:42.149 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.149 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.149 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "100", 2024-04-25T11:37:42.149 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.149 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.149 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.150 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.150 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.150 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.150 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.150 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.150 INFO:teuthology.orchestra.run.smithi069.stdout: "identifier": { 2024-04-25T11:37:42.150 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "identifier", 2024-04-25T11:37:42.150 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.150 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.150 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.150 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:42.150 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.150 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.151 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.151 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.151 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.151 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.151 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.151 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.151 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-04-25T11:37:42.151 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-04-25T11:37:42.151 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-04-25T11:37:42.151 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.151 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.151 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-04-25T11:37:42.151 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.152 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.152 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.152 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.152 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.152 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.152 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.152 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.152 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-04-25T11:37:42.152 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-04-25T11:37:42.152 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.152 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.152 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.152 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:42.152 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.152 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.153 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:42.153 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:42.153 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:42.153 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:42.153 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:42.153 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:42.153 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:42.153 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:42.153 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.153 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.153 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.153 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.153 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.153 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-04-25T11:37:42.153 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-04-25T11:37:42.153 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.154 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.154 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.154 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:42.154 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.154 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.154 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.154 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.154 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.154 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.154 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.154 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.154 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-04-25T11:37:42.155 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-04-25T11:37:42.155 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.155 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.155 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.155 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-04-25T11:37:42.155 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.155 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.155 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-04-25T11:37:42.155 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-04-25T11:37:42.155 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-04-25T11:37:42.155 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-04-25T11:37:42.155 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-04-25T11:37:42.155 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-04-25T11:37:42.155 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-04-25T11:37:42.155 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:42.155 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.156 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.156 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.156 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.156 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.156 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-04-25T11:37:42.156 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-04-25T11:37:42.156 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-04-25T11:37:42.156 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.156 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.156 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-04-25T11:37:42.156 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.156 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.156 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.156 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.156 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.156 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.157 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.157 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.157 INFO:teuthology.orchestra.run.smithi069.stdout: "sqlite3_killpoint": { 2024-04-25T11:37:42.157 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sqlite3_killpoint", 2024-04-25T11:37:42.157 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:42.157 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "dev", 2024-04-25T11:37:42.157 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-04-25T11:37:42.157 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-04-25T11:37:42.157 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.157 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.157 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.157 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.157 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.157 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.157 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.158 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.158 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_host": { 2024-04-25T11:37:42.158 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_host", 2024-04-25T11:37:42.158 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.158 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.158 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.158 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-04-25T11:37:42.158 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.158 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.158 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.158 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.158 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.159 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.159 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.159 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.159 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_port": { 2024-04-25T11:37:42.159 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_port", 2024-04-25T11:37:42.159 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-04-25T11:37:42.159 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.159 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.159 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10051", 2024-04-25T11:37:42.159 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.159 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.159 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.159 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.159 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.160 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.160 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.160 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.160 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_sender": { 2024-04-25T11:37:42.160 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_sender", 2024-04-25T11:37:42.160 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-04-25T11:37:42.160 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-04-25T11:37:42.160 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-04-25T11:37:42.160 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T11:37:42.160 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-04-25T11:37:42.160 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-04-25T11:37:42.160 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-04-25T11:37:42.160 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-04-25T11:37:42.160 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-04-25T11:37:42.160 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-04-25T11:37:42.161 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-04-25T11:37:42.161 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:42.161 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:42.161 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:42.161 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:42.161 INFO:teuthology.orchestra.run.smithi069.stdout: "services": { 2024-04-25T11:37:42.161 INFO:teuthology.orchestra.run.smithi069.stdout: "dashboard": "https://172.21.15.69:8443/" 2024-04-25T11:37:42.161 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.161 INFO:teuthology.orchestra.run.smithi069.stdout: "always_on_modules": { 2024-04-25T11:37:42.161 INFO:teuthology.orchestra.run.smithi069.stdout: "octopus": [ 2024-04-25T11:37:42.161 INFO:teuthology.orchestra.run.smithi069.stdout: "balancer", 2024-04-25T11:37:42.161 INFO:teuthology.orchestra.run.smithi069.stdout: "crash", 2024-04-25T11:37:42.161 INFO:teuthology.orchestra.run.smithi069.stdout: "devicehealth", 2024-04-25T11:37:42.161 INFO:teuthology.orchestra.run.smithi069.stdout: "orchestrator", 2024-04-25T11:37:42.161 INFO:teuthology.orchestra.run.smithi069.stdout: "pg_autoscaler", 2024-04-25T11:37:42.161 INFO:teuthology.orchestra.run.smithi069.stdout: "progress", 2024-04-25T11:37:42.161 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_support", 2024-04-25T11:37:42.162 INFO:teuthology.orchestra.run.smithi069.stdout: "status", 2024-04-25T11:37:42.162 INFO:teuthology.orchestra.run.smithi069.stdout: "telemetry", 2024-04-25T11:37:42.162 INFO:teuthology.orchestra.run.smithi069.stdout: "volumes" 2024-04-25T11:37:42.162 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:42.162 INFO:teuthology.orchestra.run.smithi069.stdout: "pacific": [ 2024-04-25T11:37:42.162 INFO:teuthology.orchestra.run.smithi069.stdout: "balancer", 2024-04-25T11:37:42.162 INFO:teuthology.orchestra.run.smithi069.stdout: "crash", 2024-04-25T11:37:42.162 INFO:teuthology.orchestra.run.smithi069.stdout: "devicehealth", 2024-04-25T11:37:42.162 INFO:teuthology.orchestra.run.smithi069.stdout: "orchestrator", 2024-04-25T11:37:42.162 INFO:teuthology.orchestra.run.smithi069.stdout: "pg_autoscaler", 2024-04-25T11:37:42.162 INFO:teuthology.orchestra.run.smithi069.stdout: "progress", 2024-04-25T11:37:42.162 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_support", 2024-04-25T11:37:42.162 INFO:teuthology.orchestra.run.smithi069.stdout: "status", 2024-04-25T11:37:42.162 INFO:teuthology.orchestra.run.smithi069.stdout: "telemetry", 2024-04-25T11:37:42.163 INFO:teuthology.orchestra.run.smithi069.stdout: "volumes" 2024-04-25T11:37:42.163 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:42.163 INFO:teuthology.orchestra.run.smithi069.stdout: "quincy": [ 2024-04-25T11:37:42.163 INFO:teuthology.orchestra.run.smithi069.stdout: "balancer", 2024-04-25T11:37:42.163 INFO:teuthology.orchestra.run.smithi069.stdout: "crash", 2024-04-25T11:37:42.163 INFO:teuthology.orchestra.run.smithi069.stdout: "devicehealth", 2024-04-25T11:37:42.163 INFO:teuthology.orchestra.run.smithi069.stdout: "orchestrator", 2024-04-25T11:37:42.163 INFO:teuthology.orchestra.run.smithi069.stdout: "pg_autoscaler", 2024-04-25T11:37:42.163 INFO:teuthology.orchestra.run.smithi069.stdout: "progress", 2024-04-25T11:37:42.163 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_support", 2024-04-25T11:37:42.163 INFO:teuthology.orchestra.run.smithi069.stdout: "status", 2024-04-25T11:37:42.163 INFO:teuthology.orchestra.run.smithi069.stdout: "telemetry", 2024-04-25T11:37:42.163 INFO:teuthology.orchestra.run.smithi069.stdout: "volumes" 2024-04-25T11:37:42.163 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:42.164 INFO:teuthology.orchestra.run.smithi069.stdout: "reef": [ 2024-04-25T11:37:42.164 INFO:teuthology.orchestra.run.smithi069.stdout: "balancer", 2024-04-25T11:37:42.164 INFO:teuthology.orchestra.run.smithi069.stdout: "crash", 2024-04-25T11:37:42.164 INFO:teuthology.orchestra.run.smithi069.stdout: "devicehealth", 2024-04-25T11:37:42.164 INFO:teuthology.orchestra.run.smithi069.stdout: "orchestrator", 2024-04-25T11:37:42.164 INFO:teuthology.orchestra.run.smithi069.stdout: "pg_autoscaler", 2024-04-25T11:37:42.164 INFO:teuthology.orchestra.run.smithi069.stdout: "progress", 2024-04-25T11:37:42.164 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_support", 2024-04-25T11:37:42.164 INFO:teuthology.orchestra.run.smithi069.stdout: "status", 2024-04-25T11:37:42.164 INFO:teuthology.orchestra.run.smithi069.stdout: "telemetry", 2024-04-25T11:37:42.164 INFO:teuthology.orchestra.run.smithi069.stdout: "volumes" 2024-04-25T11:37:42.164 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-04-25T11:37:42.164 INFO:teuthology.orchestra.run.smithi069.stdout: "squid": [ 2024-04-25T11:37:42.164 INFO:teuthology.orchestra.run.smithi069.stdout: "balancer", 2024-04-25T11:37:42.165 INFO:teuthology.orchestra.run.smithi069.stdout: "crash", 2024-04-25T11:37:42.165 INFO:teuthology.orchestra.run.smithi069.stdout: "devicehealth", 2024-04-25T11:37:42.165 INFO:teuthology.orchestra.run.smithi069.stdout: "orchestrator", 2024-04-25T11:37:42.165 INFO:teuthology.orchestra.run.smithi069.stdout: "pg_autoscaler", 2024-04-25T11:37:42.165 INFO:teuthology.orchestra.run.smithi069.stdout: "progress", 2024-04-25T11:37:42.165 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_support", 2024-04-25T11:37:42.165 INFO:teuthology.orchestra.run.smithi069.stdout: "status", 2024-04-25T11:37:42.165 INFO:teuthology.orchestra.run.smithi069.stdout: "telemetry", 2024-04-25T11:37:42.165 INFO:teuthology.orchestra.run.smithi069.stdout: "volumes" 2024-04-25T11:37:42.165 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-04-25T11:37:42.165 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.165 INFO:teuthology.orchestra.run.smithi069.stdout: "last_failure_osd_epoch": 3, 2024-04-25T11:37:42.165 INFO:teuthology.orchestra.run.smithi069.stdout: "active_clients": [ 2024-04-25T11:37:42.165 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:42.165 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "devicehealth", 2024-04-25T11:37:42.166 INFO:teuthology.orchestra.run.smithi069.stdout: "addrvec": [ 2024-04-25T11:37:42.166 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:42.166 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v2", 2024-04-25T11:37:42.166 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "172.21.15.69:0", 2024-04-25T11:37:42.166 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 2942075650 2024-04-25T11:37:42.166 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:42.166 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-04-25T11:37:42.166 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.166 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:42.166 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "libcephsqlite", 2024-04-25T11:37:42.166 INFO:teuthology.orchestra.run.smithi069.stdout: "addrvec": [ 2024-04-25T11:37:42.166 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:42.166 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v2", 2024-04-25T11:37:42.166 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "172.21.15.69:0", 2024-04-25T11:37:42.167 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 711059222 2024-04-25T11:37:42.167 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:42.167 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-04-25T11:37:42.167 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.167 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:42.167 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_support", 2024-04-25T11:37:42.167 INFO:teuthology.orchestra.run.smithi069.stdout: "addrvec": [ 2024-04-25T11:37:42.167 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:42.167 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v2", 2024-04-25T11:37:42.167 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "172.21.15.69:0", 2024-04-25T11:37:42.167 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 3467705820 2024-04-25T11:37:42.167 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:42.167 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-04-25T11:37:42.168 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:42.168 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:42.168 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "volumes", 2024-04-25T11:37:42.168 INFO:teuthology.orchestra.run.smithi069.stdout: "addrvec": [ 2024-04-25T11:37:42.168 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-04-25T11:37:42.168 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v2", 2024-04-25T11:37:42.168 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "172.21.15.69:0", 2024-04-25T11:37:42.168 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 1139337151 2024-04-25T11:37:42.168 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:42.168 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-04-25T11:37:42.168 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:42.168 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-04-25T11:37:42.168 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-25T11:37:42.170 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:37:41 smithi152 ceph-mon[32701]: pgmap v708: 129 pgs: 129 active+clean; 7.0 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-25T11:37:42.170 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:37:41 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/3763292593' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T11:37:42.170 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:37:41 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/3659100046' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T11:37:42.170 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:37:41 smithi146 ceph-mon[32821]: pgmap v708: 129 pgs: 129 active+clean; 7.0 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-25T11:37:42.170 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:37:41 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/3763292593' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T11:37:42.170 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:37:41 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/3659100046' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T11:37:42.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.799+0000 7f7052434640 1 -- 172.21.15.69:0/3659100046 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f70340afaf0 msgr2=0x7f70340b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:42.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.799+0000 7f7052434640 1 --2- 172.21.15.69:0/3659100046 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f70340afaf0 0x7f70340b2000 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f703c005fd0 tx=0x7f703c00b390 comp rx=0 tx=0).stop 2024-04-25T11:37:42.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.799+0000 7f7052434640 1 -- 172.21.15.69:0/3659100046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f704c108910 msgr2=0x7f704c1a15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:42.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.799+0000 7f7052434640 1 --2- 172.21.15.69:0/3659100046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f704c108910 0x7f704c1a15c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7044031040 tx=0x7f7044003ba0 comp rx=0 tx=0).stop 2024-04-25T11:37:42.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.800+0000 7f7052434640 1 -- 172.21.15.69:0/3659100046 shutdown_connections 2024-04-25T11:37:42.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.800+0000 7f7052434640 1 --2- 172.21.15.69:0/3659100046 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f70340afaf0 0x7f70340b2000 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:42.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.800+0000 7f7052434640 1 --2- 172.21.15.69:0/3659100046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f704c108910 0x7f704c1a15c0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:42.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.800+0000 7f7052434640 1 --2- 172.21.15.69:0/3659100046 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f704c103c10 0x7f704c19a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:42.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.800+0000 7f7052434640 1 --2- 172.21.15.69:0/3659100046 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f704c1029d0 0x7f704c19a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:42.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.800+0000 7f7052434640 1 -- 172.21.15.69:0/3659100046 >> 172.21.15.69:0/3659100046 conn(0x7f704c0fdba0 msgr2=0x7f704c109e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:42.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.800+0000 7f7052434640 1 -- 172.21.15.69:0/3659100046 shutdown_connections 2024-04-25T11:37:42.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:37:40.800+0000 7f7052434640 1 -- 172.21.15.69:0/3659100046 wait complete. 2024-04-25T11:37:42.178 DEBUG:tasks.check_counter:Getting stats from a 2024-04-25T11:37:42.178 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-25T11:37:42.179 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:37:41 smithi069 ceph-mon[29188]: pgmap v708: 129 pgs: 129 active+clean; 7.0 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 16 op/s 2024-04-25T11:37:42.179 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:37:41 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/3763292593' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T11:37:42.179 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:37:41 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/3659100046' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T11:37:42.430 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:37:43.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:37:43 smithi146 ceph-mon[32821]: pgmap v709: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-04-25T11:37:43.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:37:43 smithi069 ceph-mon[29188]: pgmap v709: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-04-25T11:37:43.724 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:37:43 smithi152 ceph-mon[32701]: pgmap v709: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-04-25T11:37:44.306 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-25T11:37:44.306 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T11:37:44.306 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 29, 2024-04-25T11:37:44.306 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 35, 2024-04-25T11:37:44.306 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 6208, 2024-04-25T11:37:44.306 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 57822, 2024-04-25T11:37:44.306 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 13, 2024-04-25T11:37:44.306 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 6, 2024-04-25T11:37:44.306 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.006373200, 2024-04-25T11:37:44.306 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.000857285, 2024-04-25T11:37:44.307 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.003844138, 2024-04-25T11:37:44.307 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000363940, 2024-04-25T11:37:44.307 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:37:44.307 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 29, 2024-04-25T11:37:44.307 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.004540400, 2024-04-25T11:37:44.307 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000156565 2024-04-25T11:37:44.307 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.307 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:37:44.307 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 22, 2024-04-25T11:37:44.307 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000006326, 2024-04-25T11:37:44.307 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000287 2024-04-25T11:37:44.307 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.307 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T11:37:44.307 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T11:37:44.307 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.307 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T11:37:44.308 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 452, 2024-04-25T11:37:44.308 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 693, 2024-04-25T11:37:44.308 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 108230, 2024-04-25T11:37:44.308 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 610573, 2024-04-25T11:37:44.308 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 7, 2024-04-25T11:37:44.308 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 4, 2024-04-25T11:37:44.308 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.079314107, 2024-04-25T11:37:44.308 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.038233376, 2024-04-25T11:37:44.308 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.035802288, 2024-04-25T11:37:44.308 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.005400714, 2024-04-25T11:37:44.308 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:37:44.308 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 696, 2024-04-25T11:37:44.308 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.047586354, 2024-04-25T11:37:44.308 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000068371 2024-04-25T11:37:44.308 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.309 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:37:44.309 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 935, 2024-04-25T11:37:44.309 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000291855, 2024-04-25T11:37:44.309 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000312 2024-04-25T11:37:44.309 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.309 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 3120, 2024-04-25T11:37:44.309 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 382176 2024-04-25T11:37:44.309 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.309 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T11:37:44.309 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 302, 2024-04-25T11:37:44.309 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 294, 2024-04-25T11:37:44.309 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 855497, 2024-04-25T11:37:44.309 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 159478, 2024-04-25T11:37:44.309 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 6, 2024-04-25T11:37:44.309 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 5, 2024-04-25T11:37:44.310 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.044835023, 2024-04-25T11:37:44.310 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.017049264, 2024-04-25T11:37:44.310 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.025954556, 2024-04-25T11:37:44.310 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.005507670, 2024-04-25T11:37:44.310 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:37:44.310 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 293, 2024-04-25T11:37:44.310 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.038630662, 2024-04-25T11:37:44.310 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000131845 2024-04-25T11:37:44.310 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.310 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:37:44.310 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 15, 2024-04-25T11:37:44.310 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000001751, 2024-04-25T11:37:44.310 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000116 2024-04-25T11:37:44.311 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.311 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 850832, 2024-04-25T11:37:44.311 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 151168 2024-04-25T11:37:44.311 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.311 INFO:teuthology.orchestra.run.smithi069.stdout: "cct": { 2024-04-25T11:37:44.311 INFO:teuthology.orchestra.run.smithi069.stdout: "total_workers": 1, 2024-04-25T11:37:44.311 INFO:teuthology.orchestra.run.smithi069.stdout: "unhealthy_workers": 0 2024-04-25T11:37:44.311 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.311 INFO:teuthology.orchestra.run.smithi069.stdout: "finisher-MDSRank": { 2024-04-25T11:37:44.311 INFO:teuthology.orchestra.run.smithi069.stdout: "queue_len": 0, 2024-04-25T11:37:44.311 INFO:teuthology.orchestra.run.smithi069.stdout: "complete_latency": { 2024-04-25T11:37:44.311 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 55, 2024-04-25T11:37:44.311 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.001573549, 2024-04-25T11:37:44.311 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000028609 2024-04-25T11:37:44.312 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:44.312 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.312 INFO:teuthology.orchestra.run.smithi069.stdout: "finisher-PurgeQueue": { 2024-04-25T11:37:44.312 INFO:teuthology.orchestra.run.smithi069.stdout: "queue_len": 0, 2024-04-25T11:37:44.312 INFO:teuthology.orchestra.run.smithi069.stdout: "complete_latency": { 2024-04-25T11:37:44.312 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 2, 2024-04-25T11:37:44.312 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000005905, 2024-04-25T11:37:44.312 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000002952 2024-04-25T11:37:44.312 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:44.312 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.312 INFO:teuthology.orchestra.run.smithi069.stdout: "mds": { 2024-04-25T11:37:44.312 INFO:teuthology.orchestra.run.smithi069.stdout: "request": 4, 2024-04-25T11:37:44.312 INFO:teuthology.orchestra.run.smithi069.stdout: "reply": 3, 2024-04-25T11:37:44.312 INFO:teuthology.orchestra.run.smithi069.stdout: "reply_latency": { 2024-04-25T11:37:44.312 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 3, 2024-04-25T11:37:44.313 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.001181468, 2024-04-25T11:37:44.313 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000393822 2024-04-25T11:37:44.313 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.313 INFO:teuthology.orchestra.run.smithi069.stdout: "slow_reply": 0, 2024-04-25T11:37:44.313 INFO:teuthology.orchestra.run.smithi069.stdout: "forward": 1, 2024-04-25T11:37:44.313 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_fetch_complete": 0, 2024-04-25T11:37:44.313 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_fetch_keys": 0, 2024-04-25T11:37:44.313 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_commit": 11, 2024-04-25T11:37:44.313 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_split": 0, 2024-04-25T11:37:44.313 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_merge": 0, 2024-04-25T11:37:44.313 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes": 12, 2024-04-25T11:37:44.313 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_top": 0, 2024-04-25T11:37:44.313 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_bottom": 12, 2024-04-25T11:37:44.313 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_pin_tail": 0, 2024-04-25T11:37:44.314 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_pinned": 11, 2024-04-25T11:37:44.314 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_expired": 0, 2024-04-25T11:37:44.314 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_with_caps": 0, 2024-04-25T11:37:44.314 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": 0, 2024-04-25T11:37:44.314 INFO:teuthology.orchestra.run.smithi069.stdout: "subtrees": 2, 2024-04-25T11:37:44.314 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse": 8, 2024-04-25T11:37:44.314 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_hit": 6, 2024-04-25T11:37:44.314 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_forward": 0, 2024-04-25T11:37:44.314 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_discover": 2, 2024-04-25T11:37:44.314 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_dir_fetch": 0, 2024-04-25T11:37:44.314 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_remote_ino": 0, 2024-04-25T11:37:44.314 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_lock": 0, 2024-04-25T11:37:44.314 INFO:teuthology.orchestra.run.smithi069.stdout: "load_cent": 0, 2024-04-25T11:37:44.315 INFO:teuthology.orchestra.run.smithi069.stdout: "q": 0, 2024-04-25T11:37:44.315 INFO:teuthology.orchestra.run.smithi069.stdout: "exported": 0, 2024-04-25T11:37:44.315 INFO:teuthology.orchestra.run.smithi069.stdout: "exported_inodes": 0, 2024-04-25T11:37:44.315 INFO:teuthology.orchestra.run.smithi069.stdout: "imported": 0, 2024-04-25T11:37:44.315 INFO:teuthology.orchestra.run.smithi069.stdout: "imported_inodes": 0, 2024-04-25T11:37:44.315 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_dir_fetch": 0, 2024-04-25T11:37:44.315 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_backtrace_fetch": 0, 2024-04-25T11:37:44.315 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_peer_discover": 0, 2024-04-25T11:37:44.315 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rfiles": 5, 2024-04-25T11:37:44.315 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rbytes": 298, 2024-04-25T11:37:44.315 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rsnaps": 0, 2024-04-25T11:37:44.315 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T11:37:44.315 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_set_tag": 0, 2024-04-25T11:37:44.315 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T11:37:44.315 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_inotable_repaired": 0, 2024-04-25T11:37:44.316 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dir_inodes": 0, 2024-04-25T11:37:44.316 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T11:37:44.316 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T11:37:44.316 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_file_inodes": 0, 2024-04-25T11:37:44.316 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_inode_file_caps": 0, 2024-04-25T11:37:44.316 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T11:37:44.316 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_grant": 0, 2024-04-25T11:37:44.316 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T11:37:44.316 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T11:37:44.316 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T11:37:44.316 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_caps": 0, 2024-04-25T11:37:44.316 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_caps_dirty": 0, 2024-04-25T11:37:44.316 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_cap_release": 0, 2024-04-25T11:37:44.316 INFO:teuthology.orchestra.run.smithi069.stdout: "process_request_cap_release": 0 2024-04-25T11:37:44.316 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.316 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_cache": { 2024-04-25T11:37:44.317 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_update": 0, 2024-04-25T11:37:44.317 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_update_receipt": 0, 2024-04-25T11:37:44.317 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_try_discover": 0, 2024-04-25T11:37:44.317 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_send_discover": 3, 2024-04-25T11:37:44.317 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_handle_discover": 0, 2024-04-25T11:37:44.317 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays": 0, 2024-04-25T11:37:44.317 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays_delayed": 0, 2024-04-25T11:37:44.317 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays_enqueuing": 0, 2024-04-25T11:37:44.317 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_created": 0, 2024-04-25T11:37:44.317 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_enqueued": 0, 2024-04-25T11:37:44.317 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_reintegrated": 0, 2024-04-25T11:37:44.317 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_migrated": 0, 2024-04-25T11:37:44.317 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_processing": 0, 2024-04-25T11:37:44.317 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_enqueued": 0, 2024-04-25T11:37:44.317 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_prioritized": 0, 2024-04-25T11:37:44.318 INFO:teuthology.orchestra.run.smithi069.stdout: "recovery_started": 0, 2024-04-25T11:37:44.318 INFO:teuthology.orchestra.run.smithi069.stdout: "recovery_completed": 0, 2024-04-25T11:37:44.318 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_quiesce_path": 0, 2024-04-25T11:37:44.318 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_quiesce_inode": 0, 2024-04-25T11:37:44.318 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T11:37:44.318 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_exportdir": 0, 2024-04-25T11:37:44.318 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_flush": 0, 2024-04-25T11:37:44.318 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_fragmentdir": 0, 2024-04-25T11:37:44.318 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_fragstats": 0, 2024-04-25T11:37:44.318 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_inodestats": 0 2024-04-25T11:37:44.318 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.318 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_log": { 2024-04-25T11:37:44.318 INFO:teuthology.orchestra.run.smithi069.stdout: "evlrg": 0, 2024-04-25T11:37:44.319 INFO:teuthology.orchestra.run.smithi069.stdout: "evadd": 5, 2024-04-25T11:37:44.319 INFO:teuthology.orchestra.run.smithi069.stdout: "evex": 0, 2024-04-25T11:37:44.319 INFO:teuthology.orchestra.run.smithi069.stdout: "evtrm": 0, 2024-04-25T11:37:44.319 INFO:teuthology.orchestra.run.smithi069.stdout: "ev": 5, 2024-04-25T11:37:44.319 INFO:teuthology.orchestra.run.smithi069.stdout: "evexg": 0, 2024-04-25T11:37:44.319 INFO:teuthology.orchestra.run.smithi069.stdout: "evexd": 0, 2024-04-25T11:37:44.319 INFO:teuthology.orchestra.run.smithi069.stdout: "segadd": 2, 2024-04-25T11:37:44.319 INFO:teuthology.orchestra.run.smithi069.stdout: "segex": 0, 2024-04-25T11:37:44.319 INFO:teuthology.orchestra.run.smithi069.stdout: "segtrm": 0, 2024-04-25T11:37:44.319 INFO:teuthology.orchestra.run.smithi069.stdout: "seg": 2, 2024-04-25T11:37:44.319 INFO:teuthology.orchestra.run.smithi069.stdout: "segmjr": 2, 2024-04-25T11:37:44.319 INFO:teuthology.orchestra.run.smithi069.stdout: "segexg": 0, 2024-04-25T11:37:44.319 INFO:teuthology.orchestra.run.smithi069.stdout: "segexd": 0, 2024-04-25T11:37:44.319 INFO:teuthology.orchestra.run.smithi069.stdout: "expos": 4194304, 2024-04-25T11:37:44.320 INFO:teuthology.orchestra.run.smithi069.stdout: "wrpos": 4195917, 2024-04-25T11:37:44.320 INFO:teuthology.orchestra.run.smithi069.stdout: "rdpos": 0, 2024-04-25T11:37:44.320 INFO:teuthology.orchestra.run.smithi069.stdout: "jlat": { 2024-04-25T11:37:44.320 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 4, 2024-04-25T11:37:44.320 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.007198905, 2024-04-25T11:37:44.320 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.001799726 2024-04-25T11:37:44.320 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.320 INFO:teuthology.orchestra.run.smithi069.stdout: "replayed": 0 2024-04-25T11:37:44.320 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.320 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_mem": { 2024-04-25T11:37:44.320 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 15, 2024-04-25T11:37:44.320 INFO:teuthology.orchestra.run.smithi069.stdout: "ino+": 15, 2024-04-25T11:37:44.320 INFO:teuthology.orchestra.run.smithi069.stdout: "ino-": 0, 2024-04-25T11:37:44.320 INFO:teuthology.orchestra.run.smithi069.stdout: "dir": 13, 2024-04-25T11:37:44.321 INFO:teuthology.orchestra.run.smithi069.stdout: "dir+": 14, 2024-04-25T11:37:44.321 INFO:teuthology.orchestra.run.smithi069.stdout: "dir-": 1, 2024-04-25T11:37:44.321 INFO:teuthology.orchestra.run.smithi069.stdout: "dn": 12, 2024-04-25T11:37:44.321 INFO:teuthology.orchestra.run.smithi069.stdout: "dn+": 12, 2024-04-25T11:37:44.321 INFO:teuthology.orchestra.run.smithi069.stdout: "dn-": 0, 2024-04-25T11:37:44.321 INFO:teuthology.orchestra.run.smithi069.stdout: "cap": 0, 2024-04-25T11:37:44.321 INFO:teuthology.orchestra.run.smithi069.stdout: "cap+": 3, 2024-04-25T11:37:44.321 INFO:teuthology.orchestra.run.smithi069.stdout: "cap-": 3, 2024-04-25T11:37:44.321 INFO:teuthology.orchestra.run.smithi069.stdout: "rss": 121060, 2024-04-25T11:37:44.321 INFO:teuthology.orchestra.run.smithi069.stdout: "heap": 223632 2024-04-25T11:37:44.321 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.321 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_server": { 2024-04-25T11:37:44.321 INFO:teuthology.orchestra.run.smithi069.stdout: "dispatch_client_request": 6, 2024-04-25T11:37:44.321 INFO:teuthology.orchestra.run.smithi069.stdout: "dispatch_server_request": 0, 2024-04-25T11:37:44.321 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_request": 4, 2024-04-25T11:37:44.322 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_session": 13, 2024-04-25T11:37:44.322 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_peer_request": 0, 2024-04-25T11:37:44.322 INFO:teuthology.orchestra.run.smithi069.stdout: "req_create_latency": { 2024-04-25T11:37:44.322 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:44.322 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:44.322 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:44.322 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.322 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getattr_latency": { 2024-04-25T11:37:44.322 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 1, 2024-04-25T11:37:44.322 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000232529, 2024-04-25T11:37:44.322 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000232529 2024-04-25T11:37:44.322 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.322 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getfilelock_latency": { 2024-04-25T11:37:44.322 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:44.323 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:44.323 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:44.323 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.323 INFO:teuthology.orchestra.run.smithi069.stdout: "req_link_latency": { 2024-04-25T11:37:44.323 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:44.323 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:44.323 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:44.323 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.323 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookup_latency": { 2024-04-25T11:37:44.323 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 2, 2024-04-25T11:37:44.323 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000948939, 2024-04-25T11:37:44.323 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000474469 2024-04-25T11:37:44.323 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.323 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookuphash_latency": { 2024-04-25T11:37:44.324 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:44.324 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:44.324 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:44.324 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.324 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupino_latency": { 2024-04-25T11:37:44.324 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:44.324 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:44.324 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:44.324 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.324 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupname_latency": { 2024-04-25T11:37:44.324 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:44.324 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:44.324 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:44.324 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.324 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupparent_latency": { 2024-04-25T11:37:44.324 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:44.325 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:44.325 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:44.325 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.325 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupsnap_latency": { 2024-04-25T11:37:44.325 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:44.325 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:44.325 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:44.325 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.325 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lssnap_latency": { 2024-04-25T11:37:44.325 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:44.325 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:44.325 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:44.325 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.325 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mkdir_latency": { 2024-04-25T11:37:44.325 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:44.326 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:44.326 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:44.326 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.326 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mknod_latency": { 2024-04-25T11:37:44.326 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:44.327 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:44.327 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:44.327 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.327 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mksnap_latency": { 2024-04-25T11:37:44.327 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:44.327 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:44.327 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:44.327 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.327 INFO:teuthology.orchestra.run.smithi069.stdout: "req_open_latency": { 2024-04-25T11:37:44.327 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:44.327 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:44.327 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:44.327 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.327 INFO:teuthology.orchestra.run.smithi069.stdout: "req_readdir_latency": { 2024-04-25T11:37:44.328 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:44.328 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:44.328 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:44.328 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.328 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rename_latency": { 2024-04-25T11:37:44.328 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:44.328 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:44.328 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:44.328 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.328 INFO:teuthology.orchestra.run.smithi069.stdout: "req_renamesnap_latency": { 2024-04-25T11:37:44.328 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:44.328 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:44.328 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:44.329 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.329 INFO:teuthology.orchestra.run.smithi069.stdout: "req_snapdiff_latency": { 2024-04-25T11:37:44.329 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:44.329 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:44.329 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:44.329 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.329 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmdir_latency": { 2024-04-25T11:37:44.329 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:44.329 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:44.329 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:44.329 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.329 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmsnap_latency": { 2024-04-25T11:37:44.329 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:44.330 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:44.330 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:44.330 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.330 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmxattr_latency": { 2024-04-25T11:37:44.330 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:44.330 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:44.330 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:44.330 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.330 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setattr_latency": { 2024-04-25T11:37:44.330 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:44.330 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:44.330 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:44.331 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.331 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setdirlayout_latency": { 2024-04-25T11:37:44.331 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:44.331 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:44.331 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:44.331 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.331 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setfilelock_latency": { 2024-04-25T11:37:44.331 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:44.331 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:44.331 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:44.331 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.331 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setlayout_latency": { 2024-04-25T11:37:44.331 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:44.331 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:44.331 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:44.332 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.332 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setxattr_latency": { 2024-04-25T11:37:44.332 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:44.332 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:44.332 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:44.332 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.332 INFO:teuthology.orchestra.run.smithi069.stdout: "req_symlink_latency": { 2024-04-25T11:37:44.332 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:44.332 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:44.332 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:44.332 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.332 INFO:teuthology.orchestra.run.smithi069.stdout: "req_unlink_latency": { 2024-04-25T11:37:44.332 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:44.332 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:44.332 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:44.333 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.333 INFO:teuthology.orchestra.run.smithi069.stdout: "cap_revoke_eviction": 0, 2024-04-25T11:37:44.333 INFO:teuthology.orchestra.run.smithi069.stdout: "cap_acquisition_throttle": 0, 2024-04-25T11:37:44.333 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getvxattr_latency": { 2024-04-25T11:37:44.333 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:44.333 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:44.333 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:44.333 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:44.333 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.333 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_sessions": { 2024-04-25T11:37:44.333 INFO:teuthology.orchestra.run.smithi069.stdout: "session_count": 1, 2024-04-25T11:37:44.333 INFO:teuthology.orchestra.run.smithi069.stdout: "session_add": 2, 2024-04-25T11:37:44.333 INFO:teuthology.orchestra.run.smithi069.stdout: "session_remove": 1, 2024-04-25T11:37:44.333 INFO:teuthology.orchestra.run.smithi069.stdout: "sessions_open": 1, 2024-04-25T11:37:44.334 INFO:teuthology.orchestra.run.smithi069.stdout: "sessions_stale": 0, 2024-04-25T11:37:44.334 INFO:teuthology.orchestra.run.smithi069.stdout: "total_load": 2, 2024-04-25T11:37:44.334 INFO:teuthology.orchestra.run.smithi069.stdout: "average_load": 1, 2024-04-25T11:37:44.334 INFO:teuthology.orchestra.run.smithi069.stdout: "avg_session_uptime": 696, 2024-04-25T11:37:44.334 INFO:teuthology.orchestra.run.smithi069.stdout: "mdthresh_evicted": 0 2024-04-25T11:37:44.334 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.334 INFO:teuthology.orchestra.run.smithi069.stdout: "mempool": { 2024-04-25T11:37:44.334 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_bytes": 0, 2024-04-25T11:37:44.334 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_items": 0, 2024-04-25T11:37:44.334 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T11:37:44.334 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_items": 0, 2024-04-25T11:37:44.334 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T11:37:44.334 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_items": 0, 2024-04-25T11:37:44.334 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T11:37:44.335 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T11:37:44.335 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T11:37:44.335 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T11:37:44.335 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T11:37:44.335 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_items": 0, 2024-04-25T11:37:44.335 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T11:37:44.335 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T11:37:44.335 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_bytes": 0, 2024-04-25T11:37:44.335 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_items": 0, 2024-04-25T11:37:44.335 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_bytes": 0, 2024-04-25T11:37:44.335 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_items": 0, 2024-04-25T11:37:44.335 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T11:37:44.335 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T11:37:44.335 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T11:37:44.336 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T11:37:44.336 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T11:37:44.336 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_items": 0, 2024-04-25T11:37:44.336 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_bytes": 0, 2024-04-25T11:37:44.336 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_items": 0, 2024-04-25T11:37:44.336 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T11:37:44.336 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T11:37:44.336 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_bytes": 0, 2024-04-25T11:37:44.336 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_items": 0, 2024-04-25T11:37:44.336 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_bytes": 0, 2024-04-25T11:37:44.336 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_items": 0, 2024-04-25T11:37:44.336 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T11:37:44.336 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_items": 0, 2024-04-25T11:37:44.336 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T11:37:44.337 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_items": 0, 2024-04-25T11:37:44.337 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_bytes": 239095, 2024-04-25T11:37:44.337 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_items": 83, 2024-04-25T11:37:44.337 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_bytes": 88, 2024-04-25T11:37:44.337 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_items": 1, 2024-04-25T11:37:44.337 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_bytes": 0, 2024-04-25T11:37:44.337 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_items": 0, 2024-04-25T11:37:44.337 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_bytes": 0, 2024-04-25T11:37:44.337 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_items": 0, 2024-04-25T11:37:44.337 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_bytes": 0, 2024-04-25T11:37:44.337 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_items": 0, 2024-04-25T11:37:44.337 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_bytes": 9088, 2024-04-25T11:37:44.337 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_items": 175, 2024-04-25T11:37:44.337 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T11:37:44.337 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_items": 0, 2024-04-25T11:37:44.338 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_bytes": 0, 2024-04-25T11:37:44.338 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_items": 0, 2024-04-25T11:37:44.338 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_bytes": 59336, 2024-04-25T11:37:44.338 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_items": 234, 2024-04-25T11:37:44.338 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_bytes": 0, 2024-04-25T11:37:44.338 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_items": 0, 2024-04-25T11:37:44.338 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_bytes": 0, 2024-04-25T11:37:44.338 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_items": 0 2024-04-25T11:37:44.338 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.338 INFO:teuthology.orchestra.run.smithi069.stdout: "objecter": { 2024-04-25T11:37:44.338 INFO:teuthology.orchestra.run.smithi069.stdout: "op_active": 0, 2024-04-25T11:37:44.338 INFO:teuthology.orchestra.run.smithi069.stdout: "op_laggy": 0, 2024-04-25T11:37:44.338 INFO:teuthology.orchestra.run.smithi069.stdout: "op_send": 42, 2024-04-25T11:37:44.338 INFO:teuthology.orchestra.run.smithi069.stdout: "op_send_bytes": 12140, 2024-04-25T11:37:44.339 INFO:teuthology.orchestra.run.smithi069.stdout: "op_resend": 33, 2024-04-25T11:37:44.339 INFO:teuthology.orchestra.run.smithi069.stdout: "op_reply": 42, 2024-04-25T11:37:44.339 INFO:teuthology.orchestra.run.smithi069.stdout: "op_latency": { 2024-04-25T11:37:44.339 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 42, 2024-04-25T11:37:44.339 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.247994690, 2024-04-25T11:37:44.339 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.005904635 2024-04-25T11:37:44.339 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.339 INFO:teuthology.orchestra.run.smithi069.stdout: "op_inflight": 0, 2024-04-25T11:37:44.339 INFO:teuthology.orchestra.run.smithi069.stdout: "oplen_avg": { 2024-04-25T11:37:44.339 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 42, 2024-04-25T11:37:44.339 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 66 2024-04-25T11:37:44.339 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.339 INFO:teuthology.orchestra.run.smithi069.stdout: "op": 42, 2024-04-25T11:37:44.340 INFO:teuthology.orchestra.run.smithi069.stdout: "op_r": 0, 2024-04-25T11:37:44.340 INFO:teuthology.orchestra.run.smithi069.stdout: "op_w": 42, 2024-04-25T11:37:44.340 INFO:teuthology.orchestra.run.smithi069.stdout: "op_rmw": 0, 2024-04-25T11:37:44.340 INFO:teuthology.orchestra.run.smithi069.stdout: "op_pg": 0, 2024-04-25T11:37:44.340 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_stat": 0, 2024-04-25T11:37:44.340 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_create": 10, 2024-04-25T11:37:44.340 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_read": 0, 2024-04-25T11:37:44.340 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_write": 4, 2024-04-25T11:37:44.340 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_writefull": 7, 2024-04-25T11:37:44.340 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_writesame": 0, 2024-04-25T11:37:44.340 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_append": 0, 2024-04-25T11:37:44.340 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_zero": 0, 2024-04-25T11:37:44.340 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_truncate": 0, 2024-04-25T11:37:44.340 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_delete": 6, 2024-04-25T11:37:44.340 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_mapext": 0, 2024-04-25T11:37:44.341 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_sparse_read": 0, 2024-04-25T11:37:44.341 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_clonerange": 0, 2024-04-25T11:37:44.341 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_getxattr": 0, 2024-04-25T11:37:44.341 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_setxattr": 20, 2024-04-25T11:37:44.341 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_cmpxattr": 0, 2024-04-25T11:37:44.341 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_rmxattr": 0, 2024-04-25T11:37:44.341 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_resetxattrs": 0, 2024-04-25T11:37:44.341 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_call": 0, 2024-04-25T11:37:44.341 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_watch": 0, 2024-04-25T11:37:44.341 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_notify": 0, 2024-04-25T11:37:44.341 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T11:37:44.341 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_pgls": 0, 2024-04-25T11:37:44.341 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_pgls_filter": 0, 2024-04-25T11:37:44.341 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_other": 0, 2024-04-25T11:37:44.342 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_active": 0, 2024-04-25T11:37:44.342 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_send": 0, 2024-04-25T11:37:44.342 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_resend": 0, 2024-04-25T11:37:44.342 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_ping": 0, 2024-04-25T11:37:44.342 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_active": 0, 2024-04-25T11:37:44.342 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_send": 0, 2024-04-25T11:37:44.342 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_resend": 0, 2024-04-25T11:37:44.342 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_active": 0, 2024-04-25T11:37:44.342 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_send": 0, 2024-04-25T11:37:44.342 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_resend": 0, 2024-04-25T11:37:44.342 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_active": 0, 2024-04-25T11:37:44.342 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_send": 0, 2024-04-25T11:37:44.342 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_resend": 0, 2024-04-25T11:37:44.342 INFO:teuthology.orchestra.run.smithi069.stdout: "command_active": 0, 2024-04-25T11:37:44.343 INFO:teuthology.orchestra.run.smithi069.stdout: "command_send": 0, 2024-04-25T11:37:44.343 INFO:teuthology.orchestra.run.smithi069.stdout: "command_resend": 0, 2024-04-25T11:37:44.343 INFO:teuthology.orchestra.run.smithi069.stdout: "map_epoch": 84, 2024-04-25T11:37:44.343 INFO:teuthology.orchestra.run.smithi069.stdout: "map_full": 0, 2024-04-25T11:37:44.343 INFO:teuthology.orchestra.run.smithi069.stdout: "map_inc": 4, 2024-04-25T11:37:44.343 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_sessions": 9, 2024-04-25T11:37:44.343 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_session_open": 9, 2024-04-25T11:37:44.343 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_session_close": 0, 2024-04-25T11:37:44.343 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_laggy": 0, 2024-04-25T11:37:44.343 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_wr": 17, 2024-04-25T11:37:44.343 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_rd": 0, 2024-04-25T11:37:44.343 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_del": 2 2024-04-25T11:37:44.343 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.343 INFO:teuthology.orchestra.run.smithi069.stdout: "oft": { 2024-04-25T11:37:44.344 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_objs": 1, 2024-04-25T11:37:44.344 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_kv_pairs": 0, 2024-04-25T11:37:44.344 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_updates": 0, 2024-04-25T11:37:44.344 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_removes": 0 2024-04-25T11:37:44.344 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.344 INFO:teuthology.orchestra.run.smithi069.stdout: "osd-slow-ops": { 2024-04-25T11:37:44.344 INFO:teuthology.orchestra.run.smithi069.stdout: "slow_ops_count": 0 2024-04-25T11:37:44.344 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.344 INFO:teuthology.orchestra.run.smithi069.stdout: "purge_queue": { 2024-04-25T11:37:44.344 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_ops": 0, 2024-04-25T11:37:44.344 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T11:37:44.344 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing": 0, 2024-04-25T11:37:44.344 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_high_water": 0, 2024-04-25T11:37:44.344 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executed_ops": 0, 2024-04-25T11:37:44.344 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executed": 0, 2024-04-25T11:37:44.345 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_item_in_journal": 0 2024-04-25T11:37:44.345 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.345 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T11:37:44.345 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-04-25T11:37:44.345 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2024-04-25T11:37:44.345 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-04-25T11:37:44.345 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 779, 2024-04-25T11:37:44.345 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 882469, 2024-04-25T11:37:44.345 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-04-25T11:37:44.345 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 779, 2024-04-25T11:37:44.345 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-04-25T11:37:44.345 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-04-25T11:37:44.345 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 779, 2024-04-25T11:37:44.345 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 882469, 2024-04-25T11:37:44.346 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-04-25T11:37:44.346 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:44.346 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:44.346 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:44.346 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:44.346 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.346 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-objecter_bytes": { 2024-04-25T11:37:44.346 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-04-25T11:37:44.346 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2024-04-25T11:37:44.346 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-04-25T11:37:44.346 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2024-04-25T11:37:44.346 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2024-04-25T11:37:44.346 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-04-25T11:37:44.346 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2024-04-25T11:37:44.346 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 42, 2024-04-25T11:37:44.347 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 12140, 2024-04-25T11:37:44.347 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 42, 2024-04-25T11:37:44.347 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 12140, 2024-04-25T11:37:44.347 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-04-25T11:37:44.347 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:44.347 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:44.347 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:44.347 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:44.347 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.347 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-objecter_ops": { 2024-04-25T11:37:44.347 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-04-25T11:37:44.347 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 1024, 2024-04-25T11:37:44.347 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-04-25T11:37:44.347 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2024-04-25T11:37:44.348 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2024-04-25T11:37:44.348 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-04-25T11:37:44.348 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2024-04-25T11:37:44.348 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 42, 2024-04-25T11:37:44.348 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 42, 2024-04-25T11:37:44.348 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 42, 2024-04-25T11:37:44.348 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 42, 2024-04-25T11:37:44.348 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-04-25T11:37:44.348 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:44.348 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:44.348 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:44.348 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:44.348 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.349 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-write_buf_throttle": { 2024-04-25T11:37:44.349 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-04-25T11:37:44.349 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 3758096384, 2024-04-25T11:37:44.349 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-04-25T11:37:44.349 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2024-04-25T11:37:44.349 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2024-04-25T11:37:44.349 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-04-25T11:37:44.349 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2024-04-25T11:37:44.349 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-04-25T11:37:44.349 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-04-25T11:37:44.349 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 0, 2024-04-25T11:37:44.349 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 0, 2024-04-25T11:37:44.349 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-04-25T11:37:44.349 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:44.350 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:44.350 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:44.350 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:44.350 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:44.350 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-write_buf_throttle-0x55d8d1410680": { 2024-04-25T11:37:44.350 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-04-25T11:37:44.350 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 3758096384, 2024-04-25T11:37:44.350 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-04-25T11:37:44.350 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 5, 2024-04-25T11:37:44.350 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 1613, 2024-04-25T11:37:44.350 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-04-25T11:37:44.350 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 5, 2024-04-25T11:37:44.350 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-04-25T11:37:44.350 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-04-25T11:37:44.351 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 4, 2024-04-25T11:37:44.351 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 1613, 2024-04-25T11:37:44.351 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-04-25T11:37:44.351 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:44.351 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:44.351 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:44.351 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:44.351 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:44.351 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-25T11:37:45.111 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-04-25T11:37:45.111 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-04-25T11:37:45.112 DEBUG:tasks.check_counter:Getting stats from d 2024-04-25T11:37:45.112 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-25T11:37:45.360 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:37:45.501 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:37:45 smithi069 ceph-mon[29188]: pgmap v710: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.1 KiB/s wr, 16 op/s 2024-04-25T11:37:45.527 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:37:45 smithi152 ceph-mon[32701]: pgmap v710: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.1 KiB/s wr, 16 op/s 2024-04-25T11:37:45.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:37:45 smithi146 ceph-mon[32821]: pgmap v710: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.1 KiB/s wr, 16 op/s 2024-04-25T11:37:47.170 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-25T11:37:47.170 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T11:37:47.170 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 0, 2024-04-25T11:37:47.170 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 2, 2024-04-25T11:37:47.170 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 0, 2024-04-25T11:37:47.170 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 0, 2024-04-25T11:37:47.170 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 4, 2024-04-25T11:37:47.171 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 0, 2024-04-25T11:37:47.171 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.001094281, 2024-04-25T11:37:47.171 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T11:37:47.171 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.000230187, 2024-04-25T11:37:47.171 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T11:37:47.171 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:37:47.171 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:47.171 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:47.171 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:47.171 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:47.172 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:37:47.172 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:47.172 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:47.172 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:47.172 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:47.172 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T11:37:47.172 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T11:37:47.172 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:47.172 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T11:37:47.172 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 5, 2024-04-25T11:37:47.172 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 181, 2024-04-25T11:37:47.173 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 3120, 2024-04-25T11:37:47.173 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 110368, 2024-04-25T11:37:47.173 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 3, 2024-04-25T11:37:47.173 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 2, 2024-04-25T11:37:47.173 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.015315167, 2024-04-25T11:37:47.173 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.011732289, 2024-04-25T11:37:47.173 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.001658747, 2024-04-25T11:37:47.173 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T11:37:47.173 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:37:47.173 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 180, 2024-04-25T11:37:47.173 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.011380254, 2024-04-25T11:37:47.174 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000063223 2024-04-25T11:37:47.174 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:47.174 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:37:47.174 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:47.174 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:47.174 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:47.174 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:47.174 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 3120, 2024-04-25T11:37:47.174 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 110368 2024-04-25T11:37:47.174 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:47.174 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T11:37:47.174 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 239, 2024-04-25T11:37:47.174 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 234, 2024-04-25T11:37:47.175 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 807520, 2024-04-25T11:37:47.175 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 144064, 2024-04-25T11:37:47.175 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 2, 2024-04-25T11:37:47.175 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 1, 2024-04-25T11:37:47.175 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.042828467, 2024-04-25T11:37:47.175 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.017205984, 2024-04-25T11:37:47.175 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.022891856, 2024-04-25T11:37:47.175 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.005233995, 2024-04-25T11:37:47.175 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:37:47.175 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 233, 2024-04-25T11:37:47.175 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.011811432, 2024-04-25T11:37:47.176 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000050692 2024-04-25T11:37:47.176 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:47.176 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:37:47.176 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:47.176 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:47.176 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:47.176 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:47.176 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 807520, 2024-04-25T11:37:47.176 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 144064 2024-04-25T11:37:47.176 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:47.176 INFO:teuthology.orchestra.run.smithi069.stdout: "cct": { 2024-04-25T11:37:47.177 INFO:teuthology.orchestra.run.smithi069.stdout: "total_workers": 0, 2024-04-25T11:37:47.177 INFO:teuthology.orchestra.run.smithi069.stdout: "unhealthy_workers": 0 2024-04-25T11:37:47.177 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:47.177 INFO:teuthology.orchestra.run.smithi069.stdout: "mempool": { 2024-04-25T11:37:47.177 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_bytes": 0, 2024-04-25T11:37:47.177 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_items": 0, 2024-04-25T11:37:47.177 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T11:37:47.177 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_items": 0, 2024-04-25T11:37:47.177 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T11:37:47.177 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_items": 0, 2024-04-25T11:37:47.177 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T11:37:47.177 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T11:37:47.177 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T11:37:47.178 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T11:37:47.178 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T11:37:47.178 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_items": 0, 2024-04-25T11:37:47.178 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T11:37:47.178 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T11:37:47.178 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_bytes": 0, 2024-04-25T11:37:47.178 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_items": 0, 2024-04-25T11:37:47.178 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_bytes": 0, 2024-04-25T11:37:47.178 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_items": 0, 2024-04-25T11:37:47.178 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T11:37:47.178 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T11:37:47.178 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T11:37:47.179 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T11:37:47.179 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T11:37:47.179 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_items": 0, 2024-04-25T11:37:47.179 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_bytes": 0, 2024-04-25T11:37:47.179 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_items": 0, 2024-04-25T11:37:47.179 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T11:37:47.179 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T11:37:47.179 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_bytes": 0, 2024-04-25T11:37:47.179 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_items": 0, 2024-04-25T11:37:47.179 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_bytes": 0, 2024-04-25T11:37:47.179 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_items": 0, 2024-04-25T11:37:47.179 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T11:37:47.179 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_items": 0, 2024-04-25T11:37:47.179 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T11:37:47.180 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_items": 0, 2024-04-25T11:37:47.180 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_bytes": 149047, 2024-04-25T11:37:47.180 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_items": 31, 2024-04-25T11:37:47.180 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_bytes": 0, 2024-04-25T11:37:47.180 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_items": 0, 2024-04-25T11:37:47.180 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_bytes": 0, 2024-04-25T11:37:47.180 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_items": 0, 2024-04-25T11:37:47.180 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_bytes": 0, 2024-04-25T11:37:47.180 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_items": 0, 2024-04-25T11:37:47.180 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_bytes": 0, 2024-04-25T11:37:47.180 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_items": 0, 2024-04-25T11:37:47.180 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_bytes": 0, 2024-04-25T11:37:47.180 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_items": 0, 2024-04-25T11:37:47.180 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T11:37:47.181 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_items": 0, 2024-04-25T11:37:47.181 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_bytes": 0, 2024-04-25T11:37:47.181 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_items": 0, 2024-04-25T11:37:47.181 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_bytes": 928, 2024-04-25T11:37:47.181 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_items": 2, 2024-04-25T11:37:47.181 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_bytes": 0, 2024-04-25T11:37:47.181 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_items": 0, 2024-04-25T11:37:47.181 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_bytes": 0, 2024-04-25T11:37:47.181 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_items": 0 2024-04-25T11:37:47.181 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:47.181 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T11:37:47.181 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-04-25T11:37:47.181 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2024-04-25T11:37:47.182 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-04-25T11:37:47.182 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 240, 2024-04-25T11:37:47.182 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 773750, 2024-04-25T11:37:47.182 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-04-25T11:37:47.182 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 240, 2024-04-25T11:37:47.182 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-04-25T11:37:47.182 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-04-25T11:37:47.182 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 240, 2024-04-25T11:37:47.182 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 773750, 2024-04-25T11:37:47.182 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-04-25T11:37:47.182 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:47.182 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:47.182 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:47.183 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:47.183 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:47.183 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-25T11:37:47.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:37:47 smithi069 ceph-mon[29188]: pgmap v711: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 KiB/s wr, 14 op/s 2024-04-25T11:37:47.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:37:47 smithi146 ceph-mon[32821]: pgmap v711: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 KiB/s wr, 14 op/s 2024-04-25T11:37:47.724 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:37:47 smithi152 ceph-mon[32701]: pgmap v711: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 KiB/s wr, 14 op/s 2024-04-25T11:37:47.865 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-25T11:37:47.865 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-25T11:37:47.865 DEBUG:tasks.check_counter:Getting stats from g 2024-04-25T11:37:47.865 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-25T11:37:48.104 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:37:49.591 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:37:49 smithi069 ceph-mon[29188]: pgmap v712: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.5 KiB/s wr, 14 op/s 2024-04-25T11:37:49.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:37:49 smithi146 ceph-mon[32821]: pgmap v712: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.5 KiB/s wr, 14 op/s 2024-04-25T11:37:49.724 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:37:49 smithi152 ceph-mon[32701]: pgmap v712: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.5 KiB/s wr, 14 op/s 2024-04-25T11:37:49.882 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-25T11:37:49.882 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T11:37:49.882 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 1066, 2024-04-25T11:37:49.882 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 1060, 2024-04-25T11:37:49.882 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 1107344, 2024-04-25T11:37:49.882 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 404440, 2024-04-25T11:37:49.882 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 5, 2024-04-25T11:37:49.882 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 1, 2024-04-25T11:37:49.882 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.140351443, 2024-04-25T11:37:49.882 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.059558950, 2024-04-25T11:37:49.883 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.043624623, 2024-04-25T11:37:49.883 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.030706732, 2024-04-25T11:37:49.883 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:37:49.883 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 1059, 2024-04-25T11:37:49.883 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.055230684, 2024-04-25T11:37:49.883 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000052153 2024-04-25T11:37:49.883 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.883 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:37:49.883 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:49.883 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:49.883 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:49.883 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.883 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 831824, 2024-04-25T11:37:49.883 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 143680 2024-04-25T11:37:49.883 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.883 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T11:37:49.883 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 1534, 2024-04-25T11:37:49.884 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 1707, 2024-04-25T11:37:49.884 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 394051, 2024-04-25T11:37:49.884 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 872039, 2024-04-25T11:37:49.884 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 5, 2024-04-25T11:37:49.884 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 4, 2024-04-25T11:37:49.884 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.118765174, 2024-04-25T11:37:49.884 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.039821509, 2024-04-25T11:37:49.884 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.029811683, 2024-04-25T11:37:49.884 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.040139031, 2024-04-25T11:37:49.884 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:37:49.884 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 1706, 2024-04-25T11:37:49.884 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.058393845, 2024-04-25T11:37:49.885 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000034228 2024-04-25T11:37:49.885 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.885 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:37:49.885 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:49.885 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:49.885 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:49.885 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.885 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 3120, 2024-04-25T11:37:49.885 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 385760 2024-04-25T11:37:49.885 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.885 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T11:37:49.885 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 4, 2024-04-25T11:37:49.885 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 6, 2024-04-25T11:37:49.885 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 1629, 2024-04-25T11:37:49.886 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 1284, 2024-04-25T11:37:49.886 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 5, 2024-04-25T11:37:49.886 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 4, 2024-04-25T11:37:49.886 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.001199959, 2024-04-25T11:37:49.886 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.000059988, 2024-04-25T11:37:49.886 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.000978867, 2024-04-25T11:37:49.886 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000072725, 2024-04-25T11:37:49.886 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:37:49.886 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 4, 2024-04-25T11:37:49.886 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.001920745, 2024-04-25T11:37:49.886 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000480186 2024-04-25T11:37:49.886 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.886 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:37:49.886 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:49.887 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:49.887 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:49.887 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.887 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T11:37:49.887 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T11:37:49.887 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.887 INFO:teuthology.orchestra.run.smithi069.stdout: "cct": { 2024-04-25T11:37:49.887 INFO:teuthology.orchestra.run.smithi069.stdout: "total_workers": 1, 2024-04-25T11:37:49.887 INFO:teuthology.orchestra.run.smithi069.stdout: "unhealthy_workers": 0 2024-04-25T11:37:49.887 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.887 INFO:teuthology.orchestra.run.smithi069.stdout: "finisher-MDSRank": { 2024-04-25T11:37:49.887 INFO:teuthology.orchestra.run.smithi069.stdout: "queue_len": 0, 2024-04-25T11:37:49.887 INFO:teuthology.orchestra.run.smithi069.stdout: "complete_latency": { 2024-04-25T11:37:49.887 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 3978, 2024-04-25T11:37:49.888 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.110044719, 2024-04-25T11:37:49.888 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000027663 2024-04-25T11:37:49.888 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:49.888 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.888 INFO:teuthology.orchestra.run.smithi069.stdout: "finisher-PurgeQueue": { 2024-04-25T11:37:49.888 INFO:teuthology.orchestra.run.smithi069.stdout: "queue_len": 0, 2024-04-25T11:37:49.888 INFO:teuthology.orchestra.run.smithi069.stdout: "complete_latency": { 2024-04-25T11:37:49.888 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:49.888 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:49.888 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:49.888 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:49.889 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.889 INFO:teuthology.orchestra.run.smithi069.stdout: "mds": { 2024-04-25T11:37:49.889 INFO:teuthology.orchestra.run.smithi069.stdout: "request": 0, 2024-04-25T11:37:49.889 INFO:teuthology.orchestra.run.smithi069.stdout: "reply": 0, 2024-04-25T11:37:49.889 INFO:teuthology.orchestra.run.smithi069.stdout: "reply_latency": { 2024-04-25T11:37:49.889 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:49.889 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:49.889 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:49.889 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.889 INFO:teuthology.orchestra.run.smithi069.stdout: "slow_reply": 0, 2024-04-25T11:37:49.889 INFO:teuthology.orchestra.run.smithi069.stdout: "forward": 0, 2024-04-25T11:37:49.890 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_fetch_complete": 0, 2024-04-25T11:37:49.890 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_fetch_keys": 0, 2024-04-25T11:37:49.890 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_commit": 0, 2024-04-25T11:37:49.890 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_split": 0, 2024-04-25T11:37:49.890 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_merge": 0, 2024-04-25T11:37:49.890 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes": 0, 2024-04-25T11:37:49.890 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_top": 0, 2024-04-25T11:37:49.890 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_bottom": 0, 2024-04-25T11:37:49.890 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_pin_tail": 0, 2024-04-25T11:37:49.890 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_pinned": 0, 2024-04-25T11:37:49.890 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_expired": 0, 2024-04-25T11:37:49.890 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_with_caps": 0, 2024-04-25T11:37:49.890 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": 0, 2024-04-25T11:37:49.890 INFO:teuthology.orchestra.run.smithi069.stdout: "subtrees": 1, 2024-04-25T11:37:49.891 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse": 0, 2024-04-25T11:37:49.891 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_hit": 0, 2024-04-25T11:37:49.891 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_forward": 0, 2024-04-25T11:37:49.891 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_discover": 0, 2024-04-25T11:37:49.891 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_dir_fetch": 0, 2024-04-25T11:37:49.891 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_remote_ino": 0, 2024-04-25T11:37:49.891 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_lock": 0, 2024-04-25T11:37:49.891 INFO:teuthology.orchestra.run.smithi069.stdout: "load_cent": 0, 2024-04-25T11:37:49.891 INFO:teuthology.orchestra.run.smithi069.stdout: "q": 0, 2024-04-25T11:37:49.891 INFO:teuthology.orchestra.run.smithi069.stdout: "exported": 0, 2024-04-25T11:37:49.891 INFO:teuthology.orchestra.run.smithi069.stdout: "exported_inodes": 0, 2024-04-25T11:37:49.891 INFO:teuthology.orchestra.run.smithi069.stdout: "imported": 0, 2024-04-25T11:37:49.891 INFO:teuthology.orchestra.run.smithi069.stdout: "imported_inodes": 0, 2024-04-25T11:37:49.891 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_dir_fetch": 0, 2024-04-25T11:37:49.892 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_backtrace_fetch": 0, 2024-04-25T11:37:49.892 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_peer_discover": 0, 2024-04-25T11:37:49.892 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rfiles": 0, 2024-04-25T11:37:49.892 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rbytes": 0, 2024-04-25T11:37:49.892 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rsnaps": 0, 2024-04-25T11:37:49.892 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T11:37:49.892 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_set_tag": 0, 2024-04-25T11:37:49.892 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T11:37:49.892 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_inotable_repaired": 0, 2024-04-25T11:37:49.892 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dir_inodes": 0, 2024-04-25T11:37:49.892 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T11:37:49.892 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T11:37:49.892 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_file_inodes": 0, 2024-04-25T11:37:49.892 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_inode_file_caps": 0, 2024-04-25T11:37:49.892 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T11:37:49.893 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_grant": 0, 2024-04-25T11:37:49.893 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T11:37:49.893 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T11:37:49.893 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T11:37:49.893 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_caps": 0, 2024-04-25T11:37:49.893 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_caps_dirty": 0, 2024-04-25T11:37:49.893 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_cap_release": 0, 2024-04-25T11:37:49.893 INFO:teuthology.orchestra.run.smithi069.stdout: "process_request_cap_release": 0 2024-04-25T11:37:49.893 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.893 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_cache": { 2024-04-25T11:37:49.893 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_update": 0, 2024-04-25T11:37:49.893 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_update_receipt": 0, 2024-04-25T11:37:49.893 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_try_discover": 0, 2024-04-25T11:37:49.893 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_send_discover": 0, 2024-04-25T11:37:49.893 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_handle_discover": 0, 2024-04-25T11:37:49.894 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays": 0, 2024-04-25T11:37:49.894 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays_delayed": 0, 2024-04-25T11:37:49.894 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays_enqueuing": 0, 2024-04-25T11:37:49.894 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_created": 0, 2024-04-25T11:37:49.894 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_enqueued": 0, 2024-04-25T11:37:49.894 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_reintegrated": 0, 2024-04-25T11:37:49.894 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_migrated": 0, 2024-04-25T11:37:49.894 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_processing": 0, 2024-04-25T11:37:49.894 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_enqueued": 0, 2024-04-25T11:37:49.894 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_prioritized": 0, 2024-04-25T11:37:49.894 INFO:teuthology.orchestra.run.smithi069.stdout: "recovery_started": 0, 2024-04-25T11:37:49.894 INFO:teuthology.orchestra.run.smithi069.stdout: "recovery_completed": 0, 2024-04-25T11:37:49.894 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_quiesce_path": 0, 2024-04-25T11:37:49.895 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_quiesce_inode": 0, 2024-04-25T11:37:49.895 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T11:37:49.895 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_exportdir": 0, 2024-04-25T11:37:49.895 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_flush": 0, 2024-04-25T11:37:49.895 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_fragmentdir": 0, 2024-04-25T11:37:49.895 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_fragstats": 0, 2024-04-25T11:37:49.895 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_inodestats": 0 2024-04-25T11:37:49.895 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.895 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_log": { 2024-04-25T11:37:49.895 INFO:teuthology.orchestra.run.smithi069.stdout: "evlrg": 0, 2024-04-25T11:37:49.895 INFO:teuthology.orchestra.run.smithi069.stdout: "evadd": 0, 2024-04-25T11:37:49.895 INFO:teuthology.orchestra.run.smithi069.stdout: "evex": 0, 2024-04-25T11:37:49.895 INFO:teuthology.orchestra.run.smithi069.stdout: "evtrm": 0, 2024-04-25T11:37:49.895 INFO:teuthology.orchestra.run.smithi069.stdout: "ev": 5, 2024-04-25T11:37:49.895 INFO:teuthology.orchestra.run.smithi069.stdout: "evexg": 0, 2024-04-25T11:37:49.896 INFO:teuthology.orchestra.run.smithi069.stdout: "evexd": 0, 2024-04-25T11:37:49.896 INFO:teuthology.orchestra.run.smithi069.stdout: "segadd": 0, 2024-04-25T11:37:49.896 INFO:teuthology.orchestra.run.smithi069.stdout: "segex": 0, 2024-04-25T11:37:49.896 INFO:teuthology.orchestra.run.smithi069.stdout: "segtrm": 0, 2024-04-25T11:37:49.896 INFO:teuthology.orchestra.run.smithi069.stdout: "seg": 2, 2024-04-25T11:37:49.896 INFO:teuthology.orchestra.run.smithi069.stdout: "segmjr": 2, 2024-04-25T11:37:49.896 INFO:teuthology.orchestra.run.smithi069.stdout: "segexg": 0, 2024-04-25T11:37:49.896 INFO:teuthology.orchestra.run.smithi069.stdout: "segexd": 0, 2024-04-25T11:37:49.896 INFO:teuthology.orchestra.run.smithi069.stdout: "expos": 4194304, 2024-04-25T11:37:49.896 INFO:teuthology.orchestra.run.smithi069.stdout: "wrpos": 4195917, 2024-04-25T11:37:49.896 INFO:teuthology.orchestra.run.smithi069.stdout: "rdpos": 4195776, 2024-04-25T11:37:49.896 INFO:teuthology.orchestra.run.smithi069.stdout: "jlat": { 2024-04-25T11:37:49.896 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:49.896 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:49.896 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:49.896 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.897 INFO:teuthology.orchestra.run.smithi069.stdout: "replayed": 5 2024-04-25T11:37:49.897 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.897 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_mem": { 2024-04-25T11:37:49.897 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 3, 2024-04-25T11:37:49.897 INFO:teuthology.orchestra.run.smithi069.stdout: "ino+": 3, 2024-04-25T11:37:49.897 INFO:teuthology.orchestra.run.smithi069.stdout: "ino-": 0, 2024-04-25T11:37:49.897 INFO:teuthology.orchestra.run.smithi069.stdout: "dir": 1, 2024-04-25T11:37:49.897 INFO:teuthology.orchestra.run.smithi069.stdout: "dir+": 1, 2024-04-25T11:37:49.897 INFO:teuthology.orchestra.run.smithi069.stdout: "dir-": 0, 2024-04-25T11:37:49.897 INFO:teuthology.orchestra.run.smithi069.stdout: "dn": 0, 2024-04-25T11:37:49.897 INFO:teuthology.orchestra.run.smithi069.stdout: "dn+": 0, 2024-04-25T11:37:49.897 INFO:teuthology.orchestra.run.smithi069.stdout: "dn-": 0, 2024-04-25T11:37:49.897 INFO:teuthology.orchestra.run.smithi069.stdout: "cap": 0, 2024-04-25T11:37:49.897 INFO:teuthology.orchestra.run.smithi069.stdout: "cap+": 0, 2024-04-25T11:37:49.897 INFO:teuthology.orchestra.run.smithi069.stdout: "cap-": 0, 2024-04-25T11:37:49.898 INFO:teuthology.orchestra.run.smithi069.stdout: "rss": 120436, 2024-04-25T11:37:49.898 INFO:teuthology.orchestra.run.smithi069.stdout: "heap": 215440 2024-04-25T11:37:49.898 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.898 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_server": { 2024-04-25T11:37:49.898 INFO:teuthology.orchestra.run.smithi069.stdout: "dispatch_client_request": 0, 2024-04-25T11:37:49.898 INFO:teuthology.orchestra.run.smithi069.stdout: "dispatch_server_request": 0, 2024-04-25T11:37:49.898 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_request": 0, 2024-04-25T11:37:49.898 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_session": 0, 2024-04-25T11:37:49.898 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_peer_request": 0, 2024-04-25T11:37:49.898 INFO:teuthology.orchestra.run.smithi069.stdout: "req_create_latency": { 2024-04-25T11:37:49.898 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:49.898 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:49.898 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:49.898 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.898 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getattr_latency": { 2024-04-25T11:37:49.899 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:49.899 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:49.899 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:49.899 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.899 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getfilelock_latency": { 2024-04-25T11:37:49.899 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:49.899 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:49.899 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:49.899 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.899 INFO:teuthology.orchestra.run.smithi069.stdout: "req_link_latency": { 2024-04-25T11:37:49.899 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:49.899 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:49.899 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:49.899 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.900 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookup_latency": { 2024-04-25T11:37:49.900 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:49.900 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:49.900 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:49.900 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.900 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookuphash_latency": { 2024-04-25T11:37:49.900 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:49.900 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:49.900 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:49.900 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.900 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupino_latency": { 2024-04-25T11:37:49.900 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:49.900 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:49.900 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:49.900 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.900 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupname_latency": { 2024-04-25T11:37:49.901 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:49.901 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:49.901 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:49.901 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.901 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupparent_latency": { 2024-04-25T11:37:49.901 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:49.901 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:49.901 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:49.901 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.901 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupsnap_latency": { 2024-04-25T11:37:49.901 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:49.901 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:49.901 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:49.901 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.901 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lssnap_latency": { 2024-04-25T11:37:49.901 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:49.901 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:49.902 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:49.902 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.902 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mkdir_latency": { 2024-04-25T11:37:49.902 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:49.902 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:49.902 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:49.903 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.903 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mknod_latency": { 2024-04-25T11:37:49.903 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:49.903 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:49.903 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:49.903 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.903 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mksnap_latency": { 2024-04-25T11:37:49.903 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:49.903 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:49.903 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:49.903 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.903 INFO:teuthology.orchestra.run.smithi069.stdout: "req_open_latency": { 2024-04-25T11:37:49.903 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:49.903 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:49.903 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:49.904 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.904 INFO:teuthology.orchestra.run.smithi069.stdout: "req_readdir_latency": { 2024-04-25T11:37:49.904 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:49.904 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:49.904 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:49.904 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.904 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rename_latency": { 2024-04-25T11:37:49.904 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:49.904 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:49.904 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:49.904 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.904 INFO:teuthology.orchestra.run.smithi069.stdout: "req_renamesnap_latency": { 2024-04-25T11:37:49.904 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:49.904 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:49.904 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:49.904 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.905 INFO:teuthology.orchestra.run.smithi069.stdout: "req_snapdiff_latency": { 2024-04-25T11:37:49.905 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:49.905 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:49.905 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:49.905 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.905 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmdir_latency": { 2024-04-25T11:37:49.905 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:49.905 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:49.905 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:49.905 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.905 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmsnap_latency": { 2024-04-25T11:37:49.905 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:49.905 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:49.905 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:49.905 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.905 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmxattr_latency": { 2024-04-25T11:37:49.906 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:49.906 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:49.906 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:49.906 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.906 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setattr_latency": { 2024-04-25T11:37:49.906 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:49.906 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:49.906 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:49.906 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.906 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setdirlayout_latency": { 2024-04-25T11:37:49.906 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:49.906 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:49.906 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:49.906 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.906 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setfilelock_latency": { 2024-04-25T11:37:49.906 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:49.907 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:49.907 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:49.907 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.907 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setlayout_latency": { 2024-04-25T11:37:49.907 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:49.907 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:49.907 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:49.907 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.907 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setxattr_latency": { 2024-04-25T11:37:49.907 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:49.907 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:49.907 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:49.907 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.907 INFO:teuthology.orchestra.run.smithi069.stdout: "req_symlink_latency": { 2024-04-25T11:37:49.908 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:49.908 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:49.908 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:49.908 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.908 INFO:teuthology.orchestra.run.smithi069.stdout: "req_unlink_latency": { 2024-04-25T11:37:49.908 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:49.908 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:49.908 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:49.908 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.908 INFO:teuthology.orchestra.run.smithi069.stdout: "cap_revoke_eviction": 0, 2024-04-25T11:37:49.908 INFO:teuthology.orchestra.run.smithi069.stdout: "cap_acquisition_throttle": 0, 2024-04-25T11:37:49.908 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getvxattr_latency": { 2024-04-25T11:37:49.908 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:49.908 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:49.908 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:49.908 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:49.908 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.909 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_sessions": { 2024-04-25T11:37:49.909 INFO:teuthology.orchestra.run.smithi069.stdout: "session_count": 1, 2024-04-25T11:37:49.909 INFO:teuthology.orchestra.run.smithi069.stdout: "session_add": 2, 2024-04-25T11:37:49.909 INFO:teuthology.orchestra.run.smithi069.stdout: "session_remove": 1, 2024-04-25T11:37:49.909 INFO:teuthology.orchestra.run.smithi069.stdout: "sessions_open": 2, 2024-04-25T11:37:49.909 INFO:teuthology.orchestra.run.smithi069.stdout: "sessions_stale": 0, 2024-04-25T11:37:49.909 INFO:teuthology.orchestra.run.smithi069.stdout: "total_load": 0, 2024-04-25T11:37:49.909 INFO:teuthology.orchestra.run.smithi069.stdout: "average_load": 0, 2024-04-25T11:37:49.909 INFO:teuthology.orchestra.run.smithi069.stdout: "avg_session_uptime": 2998, 2024-04-25T11:37:49.909 INFO:teuthology.orchestra.run.smithi069.stdout: "mdthresh_evicted": 0 2024-04-25T11:37:49.909 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.909 INFO:teuthology.orchestra.run.smithi069.stdout: "mempool": { 2024-04-25T11:37:49.909 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_bytes": 0, 2024-04-25T11:37:49.910 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_items": 0, 2024-04-25T11:37:49.910 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T11:37:49.910 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_items": 0, 2024-04-25T11:37:49.910 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T11:37:49.910 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_items": 0, 2024-04-25T11:37:49.910 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T11:37:49.910 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T11:37:49.910 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T11:37:49.910 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T11:37:49.910 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T11:37:49.910 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_items": 0, 2024-04-25T11:37:49.910 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T11:37:49.910 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T11:37:49.910 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_bytes": 0, 2024-04-25T11:37:49.911 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_items": 0, 2024-04-25T11:37:49.911 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_bytes": 0, 2024-04-25T11:37:49.911 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_items": 0, 2024-04-25T11:37:49.911 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T11:37:49.911 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T11:37:49.911 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T11:37:49.911 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T11:37:49.911 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T11:37:49.911 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_items": 0, 2024-04-25T11:37:49.911 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_bytes": 0, 2024-04-25T11:37:49.911 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_items": 0, 2024-04-25T11:37:49.911 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T11:37:49.911 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T11:37:49.911 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_bytes": 0, 2024-04-25T11:37:49.911 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_items": 0, 2024-04-25T11:37:49.911 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_bytes": 0, 2024-04-25T11:37:49.912 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_items": 0, 2024-04-25T11:37:49.912 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T11:37:49.912 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_items": 0, 2024-04-25T11:37:49.912 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T11:37:49.912 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_items": 0, 2024-04-25T11:37:49.912 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_bytes": 197480, 2024-04-25T11:37:49.912 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_items": 55, 2024-04-25T11:37:49.912 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_bytes": 0, 2024-04-25T11:37:49.912 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_items": 0, 2024-04-25T11:37:49.912 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_bytes": 0, 2024-04-25T11:37:49.912 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_items": 0, 2024-04-25T11:37:49.912 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_bytes": 0, 2024-04-25T11:37:49.912 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_items": 0, 2024-04-25T11:37:49.912 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_bytes": 0, 2024-04-25T11:37:49.912 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_items": 0, 2024-04-25T11:37:49.913 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_bytes": 9088, 2024-04-25T11:37:49.913 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_items": 175, 2024-04-25T11:37:49.913 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T11:37:49.913 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_items": 0, 2024-04-25T11:37:49.913 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_bytes": 0, 2024-04-25T11:37:49.913 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_items": 0, 2024-04-25T11:37:49.913 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_bytes": 8512, 2024-04-25T11:37:49.913 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_items": 13, 2024-04-25T11:37:49.913 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_bytes": 0, 2024-04-25T11:37:49.913 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_items": 0, 2024-04-25T11:37:49.913 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_bytes": 0, 2024-04-25T11:37:49.913 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_items": 0 2024-04-25T11:37:49.913 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.913 INFO:teuthology.orchestra.run.smithi069.stdout: "objecter": { 2024-04-25T11:37:49.914 INFO:teuthology.orchestra.run.smithi069.stdout: "op_active": 0, 2024-04-25T11:37:49.914 INFO:teuthology.orchestra.run.smithi069.stdout: "op_laggy": 0, 2024-04-25T11:37:49.914 INFO:teuthology.orchestra.run.smithi069.stdout: "op_send": 2353, 2024-04-25T11:37:49.914 INFO:teuthology.orchestra.run.smithi069.stdout: "op_send_bytes": 21, 2024-04-25T11:37:49.914 INFO:teuthology.orchestra.run.smithi069.stdout: "op_resend": 3, 2024-04-25T11:37:49.914 INFO:teuthology.orchestra.run.smithi069.stdout: "op_reply": 2353, 2024-04-25T11:37:49.914 INFO:teuthology.orchestra.run.smithi069.stdout: "op_latency": { 2024-04-25T11:37:49.914 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 2353, 2024-04-25T11:37:49.914 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 1.309971690, 2024-04-25T11:37:49.914 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000556724 2024-04-25T11:37:49.914 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.914 INFO:teuthology.orchestra.run.smithi069.stdout: "op_inflight": 0, 2024-04-25T11:37:49.914 INFO:teuthology.orchestra.run.smithi069.stdout: "oplen_avg": { 2024-04-25T11:37:49.914 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 2353, 2024-04-25T11:37:49.914 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 2354 2024-04-25T11:37:49.915 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.915 INFO:teuthology.orchestra.run.smithi069.stdout: "op": 2353, 2024-04-25T11:37:49.915 INFO:teuthology.orchestra.run.smithi069.stdout: "op_r": 2353, 2024-04-25T11:37:49.915 INFO:teuthology.orchestra.run.smithi069.stdout: "op_w": 0, 2024-04-25T11:37:49.915 INFO:teuthology.orchestra.run.smithi069.stdout: "op_rmw": 0, 2024-04-25T11:37:49.915 INFO:teuthology.orchestra.run.smithi069.stdout: "op_pg": 0, 2024-04-25T11:37:49.915 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_stat": 1524, 2024-04-25T11:37:49.915 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_create": 0, 2024-04-25T11:37:49.915 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_read": 827, 2024-04-25T11:37:49.915 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_write": 0, 2024-04-25T11:37:49.915 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_writefull": 0, 2024-04-25T11:37:49.915 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_writesame": 0, 2024-04-25T11:37:49.915 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_append": 0, 2024-04-25T11:37:49.915 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_zero": 0, 2024-04-25T11:37:49.915 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_truncate": 0, 2024-04-25T11:37:49.915 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_delete": 0, 2024-04-25T11:37:49.916 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_mapext": 0, 2024-04-25T11:37:49.916 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_sparse_read": 0, 2024-04-25T11:37:49.916 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_clonerange": 0, 2024-04-25T11:37:49.916 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_getxattr": 1, 2024-04-25T11:37:49.916 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_setxattr": 0, 2024-04-25T11:37:49.916 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_cmpxattr": 0, 2024-04-25T11:37:49.916 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_rmxattr": 0, 2024-04-25T11:37:49.916 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_resetxattrs": 0, 2024-04-25T11:37:49.916 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_call": 0, 2024-04-25T11:37:49.916 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_watch": 0, 2024-04-25T11:37:49.916 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_notify": 0, 2024-04-25T11:37:49.916 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T11:37:49.916 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_pgls": 0, 2024-04-25T11:37:49.916 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_pgls_filter": 0, 2024-04-25T11:37:49.916 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_other": 0, 2024-04-25T11:37:49.916 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_active": 0, 2024-04-25T11:37:49.917 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_send": 0, 2024-04-25T11:37:49.917 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_resend": 0, 2024-04-25T11:37:49.917 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_ping": 0, 2024-04-25T11:37:49.917 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_active": 0, 2024-04-25T11:37:49.917 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_send": 0, 2024-04-25T11:37:49.917 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_resend": 0, 2024-04-25T11:37:49.917 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_active": 0, 2024-04-25T11:37:49.917 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_send": 0, 2024-04-25T11:37:49.917 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_resend": 0, 2024-04-25T11:37:49.917 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_active": 0, 2024-04-25T11:37:49.917 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_send": 0, 2024-04-25T11:37:49.917 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_resend": 0, 2024-04-25T11:37:49.917 INFO:teuthology.orchestra.run.smithi069.stdout: "command_active": 0, 2024-04-25T11:37:49.917 INFO:teuthology.orchestra.run.smithi069.stdout: "command_send": 0, 2024-04-25T11:37:49.917 INFO:teuthology.orchestra.run.smithi069.stdout: "command_resend": 0, 2024-04-25T11:37:49.917 INFO:teuthology.orchestra.run.smithi069.stdout: "map_epoch": 84, 2024-04-25T11:37:49.917 INFO:teuthology.orchestra.run.smithi069.stdout: "map_full": 0, 2024-04-25T11:37:49.918 INFO:teuthology.orchestra.run.smithi069.stdout: "map_inc": 4, 2024-04-25T11:37:49.918 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_sessions": 6, 2024-04-25T11:37:49.918 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_session_open": 6, 2024-04-25T11:37:49.918 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_session_close": 0, 2024-04-25T11:37:49.918 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_laggy": 0, 2024-04-25T11:37:49.918 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_wr": 0, 2024-04-25T11:37:49.918 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_rd": 2, 2024-04-25T11:37:49.918 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_del": 0 2024-04-25T11:37:49.918 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.918 INFO:teuthology.orchestra.run.smithi069.stdout: "oft": { 2024-04-25T11:37:49.918 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_objs": 0, 2024-04-25T11:37:49.918 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_kv_pairs": 0, 2024-04-25T11:37:49.918 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_updates": 0, 2024-04-25T11:37:49.918 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_removes": 0 2024-04-25T11:37:49.918 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.919 INFO:teuthology.orchestra.run.smithi069.stdout: "osd-slow-ops": { 2024-04-25T11:37:49.919 INFO:teuthology.orchestra.run.smithi069.stdout: "slow_ops_count": 0 2024-04-25T11:37:49.919 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.919 INFO:teuthology.orchestra.run.smithi069.stdout: "purge_queue": { 2024-04-25T11:37:49.919 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_ops": 0, 2024-04-25T11:37:49.919 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T11:37:49.919 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing": 0, 2024-04-25T11:37:49.919 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_high_water": 0, 2024-04-25T11:37:49.919 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executed_ops": 0, 2024-04-25T11:37:49.919 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executed": 0, 2024-04-25T11:37:49.919 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_item_in_journal": 0 2024-04-25T11:37:49.919 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.919 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T11:37:49.919 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-04-25T11:37:49.920 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2024-04-25T11:37:49.920 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-04-25T11:37:49.920 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 2600, 2024-04-25T11:37:49.920 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 1253437, 2024-04-25T11:37:49.920 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-04-25T11:37:49.920 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 2600, 2024-04-25T11:37:49.920 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-04-25T11:37:49.920 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-04-25T11:37:49.920 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 2600, 2024-04-25T11:37:49.920 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 1253437, 2024-04-25T11:37:49.920 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-04-25T11:37:49.920 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:49.921 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:49.921 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:49.921 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:49.921 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.921 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-objecter_bytes": { 2024-04-25T11:37:49.921 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-04-25T11:37:49.921 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2024-04-25T11:37:49.921 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-04-25T11:37:49.921 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2024-04-25T11:37:49.921 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2024-04-25T11:37:49.921 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-04-25T11:37:49.921 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2024-04-25T11:37:49.921 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 2353, 2024-04-25T11:37:49.922 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 1618, 2024-04-25T11:37:49.922 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 2353, 2024-04-25T11:37:49.922 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 1618, 2024-04-25T11:37:49.922 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-04-25T11:37:49.922 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:49.922 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:49.922 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:49.922 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:49.922 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.922 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-objecter_ops": { 2024-04-25T11:37:49.922 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-04-25T11:37:49.922 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 1024, 2024-04-25T11:37:49.923 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-04-25T11:37:49.923 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2024-04-25T11:37:49.923 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2024-04-25T11:37:49.923 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-04-25T11:37:49.923 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2024-04-25T11:37:49.923 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 2353, 2024-04-25T11:37:49.923 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 2353, 2024-04-25T11:37:49.923 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 2353, 2024-04-25T11:37:49.923 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 2353, 2024-04-25T11:37:49.923 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-04-25T11:37:49.923 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:49.923 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:49.923 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:49.923 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:49.923 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.924 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-write_buf_throttle": { 2024-04-25T11:37:49.924 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-04-25T11:37:49.924 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 3758096384, 2024-04-25T11:37:49.924 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-04-25T11:37:49.924 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2024-04-25T11:37:49.924 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2024-04-25T11:37:49.924 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-04-25T11:37:49.924 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2024-04-25T11:37:49.924 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-04-25T11:37:49.924 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-04-25T11:37:49.924 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 0, 2024-04-25T11:37:49.924 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 0, 2024-04-25T11:37:49.924 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-04-25T11:37:49.924 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:49.924 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:49.924 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:49.925 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:49.925 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:49.925 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-write_buf_throttle-0x557282846c30": { 2024-04-25T11:37:49.925 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-04-25T11:37:49.925 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 3758096384, 2024-04-25T11:37:49.925 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-04-25T11:37:49.925 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2024-04-25T11:37:49.925 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2024-04-25T11:37:49.925 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-04-25T11:37:49.925 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2024-04-25T11:37:49.925 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-04-25T11:37:49.925 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-04-25T11:37:49.925 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 0, 2024-04-25T11:37:49.925 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 0, 2024-04-25T11:37:49.926 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-04-25T11:37:49.926 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:49.926 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:49.926 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:49.926 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:49.926 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:49.926 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-25T11:37:50.660 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-04-25T11:37:50.660 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-04-25T11:37:50.660 DEBUG:tasks.check_counter:Getting stats from j 2024-04-25T11:37:50.660 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-25T11:37:50.900 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:37:51.302 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:37:51 smithi069 ceph-mon[29188]: pgmap v713: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.5 KiB/s wr, 13 op/s 2024-04-25T11:37:51.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:37:51 smithi146 ceph-mon[32821]: pgmap v713: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.5 KiB/s wr, 13 op/s 2024-04-25T11:37:51.724 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:37:51 smithi152 ceph-mon[32701]: pgmap v713: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.5 KiB/s wr, 13 op/s 2024-04-25T11:37:52.759 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-25T11:37:52.759 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T11:37:52.759 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 242, 2024-04-25T11:37:52.759 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 237, 2024-04-25T11:37:52.759 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 808336, 2024-04-25T11:37:52.760 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 145984, 2024-04-25T11:37:52.760 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 4, 2024-04-25T11:37:52.760 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 0, 2024-04-25T11:37:52.760 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.043340619, 2024-04-25T11:37:52.760 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.016985179, 2024-04-25T11:37:52.760 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.022994780, 2024-04-25T11:37:52.760 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.005209679, 2024-04-25T11:37:52.760 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:37:52.760 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 236, 2024-04-25T11:37:52.760 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.013276886, 2024-04-25T11:37:52.760 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000056257 2024-04-25T11:37:52.760 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:52.760 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:37:52.760 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:52.761 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:52.761 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:52.761 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:52.761 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 808336, 2024-04-25T11:37:52.761 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 145984 2024-04-25T11:37:52.761 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:52.761 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T11:37:52.761 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 1, 2024-04-25T11:37:52.761 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 180, 2024-04-25T11:37:52.761 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 160, 2024-04-25T11:37:52.761 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 110096, 2024-04-25T11:37:52.761 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 3, 2024-04-25T11:37:52.761 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 2, 2024-04-25T11:37:52.761 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.017559238, 2024-04-25T11:37:52.762 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.013447375, 2024-04-25T11:37:52.762 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.001917973, 2024-04-25T11:37:52.762 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T11:37:52.762 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:37:52.762 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 178, 2024-04-25T11:37:52.762 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.011136364, 2024-04-25T11:37:52.762 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000062563 2024-04-25T11:37:52.762 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:52.762 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:37:52.762 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:52.762 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:52.762 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:52.762 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:52.762 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T11:37:52.763 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 110096 2024-04-25T11:37:52.763 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:52.763 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T11:37:52.763 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 4, 2024-04-25T11:37:52.763 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 4, 2024-04-25T11:37:52.763 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 2960, 2024-04-25T11:37:52.763 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 448, 2024-04-25T11:37:52.763 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 2, 2024-04-25T11:37:52.763 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 1, 2024-04-25T11:37:52.763 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.000694208, 2024-04-25T11:37:52.763 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.000032353, 2024-04-25T11:37:52.763 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.000652188, 2024-04-25T11:37:52.763 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T11:37:52.763 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:37:52.764 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 3, 2024-04-25T11:37:52.764 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000782890, 2024-04-25T11:37:52.764 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000260963 2024-04-25T11:37:52.764 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:52.764 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:37:52.764 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:52.764 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:52.764 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:52.764 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:52.764 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 2960, 2024-04-25T11:37:52.764 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T11:37:52.764 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:52.764 INFO:teuthology.orchestra.run.smithi069.stdout: "cct": { 2024-04-25T11:37:52.764 INFO:teuthology.orchestra.run.smithi069.stdout: "total_workers": 0, 2024-04-25T11:37:52.764 INFO:teuthology.orchestra.run.smithi069.stdout: "unhealthy_workers": 0 2024-04-25T11:37:52.765 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:52.765 INFO:teuthology.orchestra.run.smithi069.stdout: "mempool": { 2024-04-25T11:37:52.765 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_bytes": 0, 2024-04-25T11:37:52.765 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_items": 0, 2024-04-25T11:37:52.765 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T11:37:52.765 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_items": 0, 2024-04-25T11:37:52.765 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T11:37:52.765 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_items": 0, 2024-04-25T11:37:52.765 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T11:37:52.765 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T11:37:52.765 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T11:37:52.765 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T11:37:52.765 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T11:37:52.765 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_items": 0, 2024-04-25T11:37:52.766 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T11:37:52.766 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T11:37:52.766 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_bytes": 0, 2024-04-25T11:37:52.766 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_items": 0, 2024-04-25T11:37:52.766 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_bytes": 0, 2024-04-25T11:37:52.766 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_items": 0, 2024-04-25T11:37:52.766 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T11:37:52.766 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T11:37:52.766 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T11:37:52.766 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T11:37:52.766 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T11:37:52.766 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_items": 0, 2024-04-25T11:37:52.766 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_bytes": 0, 2024-04-25T11:37:52.766 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_items": 0, 2024-04-25T11:37:52.767 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T11:37:52.767 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T11:37:52.767 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_bytes": 0, 2024-04-25T11:37:52.767 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_items": 0, 2024-04-25T11:37:52.767 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_bytes": 0, 2024-04-25T11:37:52.767 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_items": 0, 2024-04-25T11:37:52.767 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T11:37:52.767 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_items": 0, 2024-04-25T11:37:52.767 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T11:37:52.767 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_items": 0, 2024-04-25T11:37:52.767 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_bytes": 149998, 2024-04-25T11:37:52.767 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_items": 30, 2024-04-25T11:37:52.767 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_bytes": 0, 2024-04-25T11:37:52.767 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_items": 0, 2024-04-25T11:37:52.768 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_bytes": 0, 2024-04-25T11:37:52.768 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_items": 0, 2024-04-25T11:37:52.768 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_bytes": 0, 2024-04-25T11:37:52.768 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_items": 0, 2024-04-25T11:37:52.768 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_bytes": 0, 2024-04-25T11:37:52.768 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_items": 0, 2024-04-25T11:37:52.768 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_bytes": 0, 2024-04-25T11:37:52.768 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_items": 0, 2024-04-25T11:37:52.768 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T11:37:52.768 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_items": 0, 2024-04-25T11:37:52.768 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_bytes": 0, 2024-04-25T11:37:52.768 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_items": 0, 2024-04-25T11:37:52.768 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_bytes": 928, 2024-04-25T11:37:52.768 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_items": 2, 2024-04-25T11:37:52.768 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_bytes": 0, 2024-04-25T11:37:52.768 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_items": 0, 2024-04-25T11:37:52.769 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_bytes": 0, 2024-04-25T11:37:52.769 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_items": 0 2024-04-25T11:37:52.769 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T11:37:52.769 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T11:37:52.769 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-04-25T11:37:52.769 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2024-04-25T11:37:52.769 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-04-25T11:37:52.769 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 243, 2024-04-25T11:37:52.769 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 774140, 2024-04-25T11:37:52.769 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-04-25T11:37:52.769 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 243, 2024-04-25T11:37:52.769 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-04-25T11:37:52.769 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-04-25T11:37:52.769 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 243, 2024-04-25T11:37:52.769 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 774140, 2024-04-25T11:37:52.770 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-04-25T11:37:52.770 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T11:37:52.770 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T11:37:52.770 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T11:37:52.770 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:52.770 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T11:37:52.770 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-25T11:37:53.453 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-25T11:37:53.453 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-25T11:37:53.453 DEBUG:tasks.check_counter:Getting stats from b 2024-04-25T11:37:53.453 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-25T11:37:53.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:37:53 smithi146 ceph-mon[32821]: pgmap v714: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.3 KiB/s wr, 12 op/s 2024-04-25T11:37:53.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:37:53 smithi069 ceph-mon[29188]: pgmap v714: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.3 KiB/s wr, 12 op/s 2024-04-25T11:37:53.707 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T11:37:53.724 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:37:53 smithi152 ceph-mon[32701]: pgmap v714: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.3 KiB/s wr, 12 op/s 2024-04-25T11:37:55.437 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:37:55 smithi069 ceph-mon[29188]: pgmap v715: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-04-25T11:37:55.536 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:37:55 smithi146 ceph-mon[32821]: pgmap v715: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-04-25T11:37:55.537 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-04-25T11:37:55.537 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T11:37:55.537 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 3466, 2024-04-25T11:37:55.537 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 3375, 2024-04-25T11:37:55.537 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 953148, 2024-04-25T11:37:55.537 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 7873707, 2024-04-25T11:37:55.537 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 11, 2024-04-25T11:37:55.537 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 3, 2024-04-25T11:37:55.537 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.329193627, 2024-04-25T11:37:55.537 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.121037349, 2024-04-25T11:37:55.537 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.088760069, 2024-04-25T11:37:55.537 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.096713834, 2024-04-25T11:37:55.537 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:37:55.538 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 3376, 2024-04-25T11:37:55.538 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.499039476, 2024-04-25T11:37:55.538 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000147819 2024-04-25T11:37:55.538 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.538 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:37:55.538 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 976, 2024-04-25T11:37:55.538 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000291289, 2024-04-25T11:37:55.538 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000298 2024-04-25T11:37:55.538 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.538 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T11:37:55.538 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T11:37:55.538 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.538 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T11:37:55.539 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 1319, 2024-04-25T11:37:55.539 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 1463, 2024-04-25T11:37:55.539 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 534687, 2024-04-25T11:37:55.539 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 729407, 2024-04-25T11:37:55.539 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 9, 2024-04-25T11:37:55.539 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 7, 2024-04-25T11:37:55.539 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.168144637, 2024-04-25T11:37:55.539 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.060170633, 2024-04-25T11:37:55.539 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.058708442, 2024-04-25T11:37:55.539 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.043253288, 2024-04-25T11:37:55.539 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:37:55.539 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 1462, 2024-04-25T11:37:55.539 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 1.045024543, 2024-04-25T11:37:55.539 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000714791 2024-04-25T11:37:55.540 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.540 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:37:55.540 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 2108, 2024-04-25T11:37:55.540 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000563904, 2024-04-25T11:37:55.540 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000267 2024-04-25T11:37:55.540 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.540 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 3120, 2024-04-25T11:37:55.540 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 398208 2024-04-25T11:37:55.540 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.540 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T11:37:55.540 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 3102, 2024-04-25T11:37:55.540 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 5067, 2024-04-25T11:37:55.540 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 1916268, 2024-04-25T11:37:55.540 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 1873924, 2024-04-25T11:37:55.541 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 18, 2024-04-25T11:37:55.541 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 10, 2024-04-25T11:37:55.541 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.442938251, 2024-04-25T11:37:55.541 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.149138344, 2024-04-25T11:37:55.541 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.215931771, 2024-04-25T11:37:55.541 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.087843800, 2024-04-25T11:37:55.541 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:37:55.541 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 5063, 2024-04-25T11:37:55.541 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.449325753, 2024-04-25T11:37:55.541 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000088746 2024-04-25T11:37:55.541 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.541 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:37:55.541 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 6728, 2024-04-25T11:37:55.541 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.001832802, 2024-04-25T11:37:55.542 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000272 2024-04-25T11:37:55.542 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.542 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 974816, 2024-04-25T11:37:55.542 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 181936 2024-04-25T11:37:55.542 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.542 INFO:teuthology.orchestra.run.smithi146.stdout: "cct": { 2024-04-25T11:37:55.542 INFO:teuthology.orchestra.run.smithi146.stdout: "total_workers": 1, 2024-04-25T11:37:55.542 INFO:teuthology.orchestra.run.smithi146.stdout: "unhealthy_workers": 0 2024-04-25T11:37:55.542 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.542 INFO:teuthology.orchestra.run.smithi146.stdout: "finisher-MDSRank": { 2024-04-25T11:37:55.542 INFO:teuthology.orchestra.run.smithi146.stdout: "queue_len": 0, 2024-04-25T11:37:55.542 INFO:teuthology.orchestra.run.smithi146.stdout: "complete_latency": { 2024-04-25T11:37:55.542 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 4706, 2024-04-25T11:37:55.542 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 1.608244866, 2024-04-25T11:37:55.543 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000341743 2024-04-25T11:37:55.543 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-04-25T11:37:55.543 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.543 INFO:teuthology.orchestra.run.smithi146.stdout: "finisher-PurgeQueue": { 2024-04-25T11:37:55.543 INFO:teuthology.orchestra.run.smithi146.stdout: "queue_len": 0, 2024-04-25T11:37:55.543 INFO:teuthology.orchestra.run.smithi146.stdout: "complete_latency": { 2024-04-25T11:37:55.543 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 17, 2024-04-25T11:37:55.543 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000493841, 2024-04-25T11:37:55.543 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000029049 2024-04-25T11:37:55.543 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-04-25T11:37:55.543 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.543 INFO:teuthology.orchestra.run.smithi146.stdout: "mds": { 2024-04-25T11:37:55.543 INFO:teuthology.orchestra.run.smithi146.stdout: "request": 659, 2024-04-25T11:37:55.543 INFO:teuthology.orchestra.run.smithi146.stdout: "reply": 659, 2024-04-25T11:37:55.543 INFO:teuthology.orchestra.run.smithi146.stdout: "reply_latency": { 2024-04-25T11:37:55.544 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 659, 2024-04-25T11:37:55.544 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 3.513294982, 2024-04-25T11:37:55.544 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.005331251 2024-04-25T11:37:55.544 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.544 INFO:teuthology.orchestra.run.smithi146.stdout: "slow_reply": 0, 2024-04-25T11:37:55.544 INFO:teuthology.orchestra.run.smithi146.stdout: "forward": 0, 2024-04-25T11:37:55.544 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_fetch_complete": 0, 2024-04-25T11:37:55.544 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_fetch_keys": 0, 2024-04-25T11:37:55.544 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_commit": 12, 2024-04-25T11:37:55.544 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_split": 0, 2024-04-25T11:37:55.544 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_merge": 0, 2024-04-25T11:37:55.544 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes": 24, 2024-04-25T11:37:55.544 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_top": 0, 2024-04-25T11:37:55.544 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_bottom": 24, 2024-04-25T11:37:55.545 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_pin_tail": 0, 2024-04-25T11:37:55.545 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_pinned": 24, 2024-04-25T11:37:55.545 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_expired": 0, 2024-04-25T11:37:55.545 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_with_caps": 2, 2024-04-25T11:37:55.545 INFO:teuthology.orchestra.run.smithi146.stdout: "caps": 2, 2024-04-25T11:37:55.545 INFO:teuthology.orchestra.run.smithi146.stdout: "subtrees": 2, 2024-04-25T11:37:55.545 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse": 688, 2024-04-25T11:37:55.545 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_hit": 680, 2024-04-25T11:37:55.545 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_forward": 0, 2024-04-25T11:37:55.545 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_discover": 0, 2024-04-25T11:37:55.545 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_dir_fetch": 0, 2024-04-25T11:37:55.545 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_remote_ino": 0, 2024-04-25T11:37:55.545 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_lock": 0, 2024-04-25T11:37:55.545 INFO:teuthology.orchestra.run.smithi146.stdout: "load_cent": 0, 2024-04-25T11:37:55.545 INFO:teuthology.orchestra.run.smithi146.stdout: "q": 0, 2024-04-25T11:37:55.546 INFO:teuthology.orchestra.run.smithi146.stdout: "exported": 0, 2024-04-25T11:37:55.546 INFO:teuthology.orchestra.run.smithi146.stdout: "exported_inodes": 0, 2024-04-25T11:37:55.546 INFO:teuthology.orchestra.run.smithi146.stdout: "imported": 0, 2024-04-25T11:37:55.546 INFO:teuthology.orchestra.run.smithi146.stdout: "imported_inodes": 0, 2024-04-25T11:37:55.546 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_dir_fetch": 0, 2024-04-25T11:37:55.546 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_backtrace_fetch": 0, 2024-04-25T11:37:55.546 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_peer_discover": 0, 2024-04-25T11:37:55.546 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rfiles": 4, 2024-04-25T11:37:55.546 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rbytes": 234, 2024-04-25T11:37:55.546 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rsnaps": 0, 2024-04-25T11:37:55.546 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T11:37:55.546 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_set_tag": 0, 2024-04-25T11:37:55.546 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T11:37:55.547 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_inotable_repaired": 0, 2024-04-25T11:37:55.547 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dir_inodes": 0, 2024-04-25T11:37:55.547 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T11:37:55.547 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T11:37:55.547 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_file_inodes": 0, 2024-04-25T11:37:55.547 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_inode_file_caps": 0, 2024-04-25T11:37:55.547 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_revoke": 615, 2024-04-25T11:37:55.547 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_grant": 1212, 2024-04-25T11:37:55.547 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_trunc": 599, 2024-04-25T11:37:55.547 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T11:37:55.547 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T11:37:55.547 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_caps": 622, 2024-04-25T11:37:55.547 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_caps_dirty": 603, 2024-04-25T11:37:55.547 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_cap_release": 2, 2024-04-25T11:37:55.547 INFO:teuthology.orchestra.run.smithi146.stdout: "process_request_cap_release": 26 2024-04-25T11:37:55.548 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.548 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_cache": { 2024-04-25T11:37:55.548 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_update": 0, 2024-04-25T11:37:55.548 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_update_receipt": 0, 2024-04-25T11:37:55.548 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_try_discover": 0, 2024-04-25T11:37:55.548 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_send_discover": 0, 2024-04-25T11:37:55.548 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_handle_discover": 9, 2024-04-25T11:37:55.548 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays": 1, 2024-04-25T11:37:55.548 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays_delayed": 0, 2024-04-25T11:37:55.548 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays_enqueuing": 0, 2024-04-25T11:37:55.548 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_created": 3, 2024-04-25T11:37:55.548 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_enqueued": 2, 2024-04-25T11:37:55.548 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_reintegrated": 0, 2024-04-25T11:37:55.548 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_migrated": 0, 2024-04-25T11:37:55.548 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_processing": 0, 2024-04-25T11:37:55.549 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_enqueued": 0, 2024-04-25T11:37:55.549 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_prioritized": 0, 2024-04-25T11:37:55.549 INFO:teuthology.orchestra.run.smithi146.stdout: "recovery_started": 0, 2024-04-25T11:37:55.549 INFO:teuthology.orchestra.run.smithi146.stdout: "recovery_completed": 0, 2024-04-25T11:37:55.549 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_quiesce_path": 0, 2024-04-25T11:37:55.549 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_quiesce_inode": 0, 2024-04-25T11:37:55.549 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T11:37:55.549 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_exportdir": 0, 2024-04-25T11:37:55.549 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_flush": 0, 2024-04-25T11:37:55.549 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_fragmentdir": 0, 2024-04-25T11:37:55.549 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_fragstats": 0, 2024-04-25T11:37:55.549 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_inodestats": 0 2024-04-25T11:37:55.549 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.549 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_log": { 2024-04-25T11:37:55.549 INFO:teuthology.orchestra.run.smithi146.stdout: "evlrg": 0, 2024-04-25T11:37:55.550 INFO:teuthology.orchestra.run.smithi146.stdout: "evadd": 2057, 2024-04-25T11:37:55.550 INFO:teuthology.orchestra.run.smithi146.stdout: "evex": 0, 2024-04-25T11:37:55.550 INFO:teuthology.orchestra.run.smithi146.stdout: "evtrm": 0, 2024-04-25T11:37:55.550 INFO:teuthology.orchestra.run.smithi146.stdout: "ev": 2057, 2024-04-25T11:37:55.550 INFO:teuthology.orchestra.run.smithi146.stdout: "evexg": 0, 2024-04-25T11:37:55.550 INFO:teuthology.orchestra.run.smithi146.stdout: "evexd": 0, 2024-04-25T11:37:55.550 INFO:teuthology.orchestra.run.smithi146.stdout: "segadd": 4, 2024-04-25T11:37:55.550 INFO:teuthology.orchestra.run.smithi146.stdout: "segex": 0, 2024-04-25T11:37:55.550 INFO:teuthology.orchestra.run.smithi146.stdout: "segtrm": 0, 2024-04-25T11:37:55.550 INFO:teuthology.orchestra.run.smithi146.stdout: "seg": 4, 2024-04-25T11:37:55.550 INFO:teuthology.orchestra.run.smithi146.stdout: "segmjr": 2, 2024-04-25T11:37:55.550 INFO:teuthology.orchestra.run.smithi146.stdout: "segexg": 0, 2024-04-25T11:37:55.550 INFO:teuthology.orchestra.run.smithi146.stdout: "segexd": 0, 2024-04-25T11:37:55.550 INFO:teuthology.orchestra.run.smithi146.stdout: "expos": 4194304, 2024-04-25T11:37:55.551 INFO:teuthology.orchestra.run.smithi146.stdout: "wrpos": 11012442, 2024-04-25T11:37:55.551 INFO:teuthology.orchestra.run.smithi146.stdout: "rdpos": 0, 2024-04-25T11:37:55.551 INFO:teuthology.orchestra.run.smithi146.stdout: "jlat": { 2024-04-25T11:37:55.551 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 2040, 2024-04-25T11:37:55.551 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 4.177321813, 2024-04-25T11:37:55.551 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.002047706 2024-04-25T11:37:55.551 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.551 INFO:teuthology.orchestra.run.smithi146.stdout: "replayed": 0 2024-04-25T11:37:55.551 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.551 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_mem": { 2024-04-25T11:37:55.551 INFO:teuthology.orchestra.run.smithi146.stdout: "ino": 24, 2024-04-25T11:37:55.551 INFO:teuthology.orchestra.run.smithi146.stdout: "ino+": 26, 2024-04-25T11:37:55.551 INFO:teuthology.orchestra.run.smithi146.stdout: "ino-": 2, 2024-04-25T11:37:55.551 INFO:teuthology.orchestra.run.smithi146.stdout: "dir": 19, 2024-04-25T11:37:55.552 INFO:teuthology.orchestra.run.smithi146.stdout: "dir+": 20, 2024-04-25T11:37:55.552 INFO:teuthology.orchestra.run.smithi146.stdout: "dir-": 1, 2024-04-25T11:37:55.552 INFO:teuthology.orchestra.run.smithi146.stdout: "dn": 24, 2024-04-25T11:37:55.552 INFO:teuthology.orchestra.run.smithi146.stdout: "dn+": 28, 2024-04-25T11:37:55.552 INFO:teuthology.orchestra.run.smithi146.stdout: "dn-": 4, 2024-04-25T11:37:55.552 INFO:teuthology.orchestra.run.smithi146.stdout: "cap": 2, 2024-04-25T11:37:55.552 INFO:teuthology.orchestra.run.smithi146.stdout: "cap+": 18, 2024-04-25T11:37:55.552 INFO:teuthology.orchestra.run.smithi146.stdout: "cap-": 16, 2024-04-25T11:37:55.552 INFO:teuthology.orchestra.run.smithi146.stdout: "rss": 126720, 2024-04-25T11:37:55.552 INFO:teuthology.orchestra.run.smithi146.stdout: "heap": 231824 2024-04-25T11:37:55.552 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.552 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_server": { 2024-04-25T11:37:55.552 INFO:teuthology.orchestra.run.smithi146.stdout: "dispatch_client_request": 1879, 2024-04-25T11:37:55.552 INFO:teuthology.orchestra.run.smithi146.stdout: "dispatch_server_request": 1, 2024-04-25T11:37:55.552 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_request": 659, 2024-04-25T11:37:55.553 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_session": 24, 2024-04-25T11:37:55.553 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_peer_request": 2, 2024-04-25T11:37:55.553 INFO:teuthology.orchestra.run.smithi146.stdout: "req_create_latency": { 2024-04-25T11:37:55.553 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 5, 2024-04-25T11:37:55.553 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.005155766, 2024-04-25T11:37:55.553 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.001031153 2024-04-25T11:37:55.553 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.553 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getattr_latency": { 2024-04-25T11:37:55.553 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 29, 2024-04-25T11:37:55.553 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.006924335, 2024-04-25T11:37:55.553 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000238770 2024-04-25T11:37:55.553 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.553 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getfilelock_latency": { 2024-04-25T11:37:55.553 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:55.554 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:55.554 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:55.554 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.554 INFO:teuthology.orchestra.run.smithi146.stdout: "req_link_latency": { 2024-04-25T11:37:55.554 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:55.554 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:55.554 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:55.554 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.554 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookup_latency": { 2024-04-25T11:37:55.554 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:55.554 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:55.554 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:55.554 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.554 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookuphash_latency": { 2024-04-25T11:37:55.555 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:55.555 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:55.555 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:55.555 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.555 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupino_latency": { 2024-04-25T11:37:55.555 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:55.555 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:55.555 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:55.555 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.555 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupname_latency": { 2024-04-25T11:37:55.555 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:55.555 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:55.555 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:55.555 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.555 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupparent_latency": { 2024-04-25T11:37:55.556 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:55.556 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:55.556 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:55.556 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.556 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupsnap_latency": { 2024-04-25T11:37:55.556 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:55.556 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:55.556 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:55.556 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.556 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lssnap_latency": { 2024-04-25T11:37:55.556 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:55.556 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:55.556 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:55.556 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.557 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mkdir_latency": { 2024-04-25T11:37:55.557 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 8, 2024-04-25T11:37:55.557 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.015021795, 2024-04-25T11:37:55.557 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.001877724 2024-04-25T11:37:55.557 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.557 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mknod_latency": { 2024-04-25T11:37:55.557 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:55.558 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:55.558 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:55.558 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.558 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mksnap_latency": { 2024-04-25T11:37:55.558 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:55.558 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:55.558 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:55.558 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.558 INFO:teuthology.orchestra.run.smithi146.stdout: "req_open_latency": { 2024-04-25T11:37:55.558 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 599, 2024-04-25T11:37:55.558 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 3.443493629, 2024-04-25T11:37:55.558 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.005748737 2024-04-25T11:37:55.558 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.558 INFO:teuthology.orchestra.run.smithi146.stdout: "req_readdir_latency": { 2024-04-25T11:37:55.559 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 2, 2024-04-25T11:37:55.559 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.003820801, 2024-04-25T11:37:55.559 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.001910400 2024-04-25T11:37:55.559 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.559 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rename_latency": { 2024-04-25T11:37:55.559 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 2, 2024-04-25T11:37:55.559 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.002931236, 2024-04-25T11:37:55.559 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.001465618 2024-04-25T11:37:55.559 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.559 INFO:teuthology.orchestra.run.smithi146.stdout: "req_renamesnap_latency": { 2024-04-25T11:37:55.559 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:55.559 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:55.559 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:55.559 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.559 INFO:teuthology.orchestra.run.smithi146.stdout: "req_snapdiff_latency": { 2024-04-25T11:37:55.559 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:55.560 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:55.560 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:55.560 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.560 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmdir_latency": { 2024-04-25T11:37:55.560 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 2, 2024-04-25T11:37:55.560 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.009535781, 2024-04-25T11:37:55.560 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.004767890 2024-04-25T11:37:55.560 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.560 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmsnap_latency": { 2024-04-25T11:37:55.560 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:55.560 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:55.560 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:55.560 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.560 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmxattr_latency": { 2024-04-25T11:37:55.561 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:55.561 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:55.561 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:55.561 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.561 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setattr_latency": { 2024-04-25T11:37:55.561 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 2, 2024-04-25T11:37:55.561 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.001640786, 2024-04-25T11:37:55.561 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000820393 2024-04-25T11:37:55.561 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.561 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setdirlayout_latency": { 2024-04-25T11:37:55.561 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:55.561 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:55.561 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:55.561 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.561 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setfilelock_latency": { 2024-04-25T11:37:55.562 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:55.562 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:55.562 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:55.562 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.562 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setlayout_latency": { 2024-04-25T11:37:55.562 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:55.562 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:55.562 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:55.562 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.562 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setxattr_latency": { 2024-04-25T11:37:55.562 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 9, 2024-04-25T11:37:55.562 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.022626750, 2024-04-25T11:37:55.562 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.002514083 2024-04-25T11:37:55.562 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.563 INFO:teuthology.orchestra.run.smithi146.stdout: "req_symlink_latency": { 2024-04-25T11:37:55.563 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:55.563 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:55.563 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:55.563 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.563 INFO:teuthology.orchestra.run.smithi146.stdout: "req_unlink_latency": { 2024-04-25T11:37:55.563 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 1, 2024-04-25T11:37:55.563 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.002144103, 2024-04-25T11:37:55.563 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.002144103 2024-04-25T11:37:55.563 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.563 INFO:teuthology.orchestra.run.smithi146.stdout: "cap_revoke_eviction": 0, 2024-04-25T11:37:55.563 INFO:teuthology.orchestra.run.smithi146.stdout: "cap_acquisition_throttle": 0, 2024-04-25T11:37:55.563 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getvxattr_latency": { 2024-04-25T11:37:55.563 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:55.563 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:55.564 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:55.564 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-04-25T11:37:55.564 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.564 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_sessions": { 2024-04-25T11:37:55.564 INFO:teuthology.orchestra.run.smithi146.stdout: "session_count": 1, 2024-04-25T11:37:55.564 INFO:teuthology.orchestra.run.smithi146.stdout: "session_add": 4, 2024-04-25T11:37:55.564 INFO:teuthology.orchestra.run.smithi146.stdout: "session_remove": 3, 2024-04-25T11:37:55.564 INFO:teuthology.orchestra.run.smithi146.stdout: "sessions_open": 1, 2024-04-25T11:37:55.564 INFO:teuthology.orchestra.run.smithi146.stdout: "sessions_stale": 0, 2024-04-25T11:37:55.564 INFO:teuthology.orchestra.run.smithi146.stdout: "total_load": 95, 2024-04-25T11:37:55.564 INFO:teuthology.orchestra.run.smithi146.stdout: "average_load": 95, 2024-04-25T11:37:55.564 INFO:teuthology.orchestra.run.smithi146.stdout: "avg_session_uptime": 709, 2024-04-25T11:37:55.564 INFO:teuthology.orchestra.run.smithi146.stdout: "mdthresh_evicted": 0 2024-04-25T11:37:55.564 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.565 INFO:teuthology.orchestra.run.smithi146.stdout: "mempool": { 2024-04-25T11:37:55.565 INFO:teuthology.orchestra.run.smithi146.stdout: "bloom_filter_bytes": 0, 2024-04-25T11:37:55.565 INFO:teuthology.orchestra.run.smithi146.stdout: "bloom_filter_items": 0, 2024-04-25T11:37:55.565 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T11:37:55.565 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_alloc_items": 0, 2024-04-25T11:37:55.565 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T11:37:55.565 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_data_items": 0, 2024-04-25T11:37:55.565 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T11:37:55.565 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T11:37:55.565 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T11:37:55.565 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T11:37:55.565 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T11:37:55.565 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_other_items": 0, 2024-04-25T11:37:55.565 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T11:37:55.565 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T11:37:55.566 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_extent_bytes": 0, 2024-04-25T11:37:55.566 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_extent_items": 0, 2024-04-25T11:37:55.566 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_blob_bytes": 0, 2024-04-25T11:37:55.566 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_blob_items": 0, 2024-04-25T11:37:55.566 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T11:37:55.566 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T11:37:55.566 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T11:37:55.566 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T11:37:55.566 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T11:37:55.566 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_items": 0, 2024-04-25T11:37:55.566 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_txc_bytes": 0, 2024-04-25T11:37:55.566 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_txc_items": 0, 2024-04-25T11:37:55.566 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T11:37:55.566 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T11:37:55.567 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_bytes": 0, 2024-04-25T11:37:55.567 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_items": 0, 2024-04-25T11:37:55.567 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_bytes": 0, 2024-04-25T11:37:55.567 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_items": 0, 2024-04-25T11:37:55.567 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T11:37:55.567 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_reader_items": 0, 2024-04-25T11:37:55.567 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T11:37:55.567 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_writer_items": 0, 2024-04-25T11:37:55.567 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_anon_bytes": 315186, 2024-04-25T11:37:55.567 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_anon_items": 131, 2024-04-25T11:37:55.567 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_meta_bytes": 0, 2024-04-25T11:37:55.567 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_meta_items": 0, 2024-04-25T11:37:55.567 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_bytes": 0, 2024-04-25T11:37:55.567 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_items": 0, 2024-04-25T11:37:55.568 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mapbl_bytes": 0, 2024-04-25T11:37:55.568 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mapbl_items": 0, 2024-04-25T11:37:55.568 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pglog_bytes": 0, 2024-04-25T11:37:55.568 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pglog_items": 0, 2024-04-25T11:37:55.568 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_bytes": 9088, 2024-04-25T11:37:55.568 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_items": 175, 2024-04-25T11:37:55.568 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T11:37:55.568 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_mapping_items": 0, 2024-04-25T11:37:55.568 INFO:teuthology.orchestra.run.smithi146.stdout: "pgmap_bytes": 0, 2024-04-25T11:37:55.568 INFO:teuthology.orchestra.run.smithi146.stdout: "pgmap_items": 0, 2024-04-25T11:37:55.568 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_co_bytes": 98577, 2024-04-25T11:37:55.568 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_co_items": 696, 2024-04-25T11:37:55.568 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_1_bytes": 0, 2024-04-25T11:37:55.568 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_1_items": 0, 2024-04-25T11:37:55.569 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_2_bytes": 0, 2024-04-25T11:37:55.569 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_2_items": 0 2024-04-25T11:37:55.569 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.569 INFO:teuthology.orchestra.run.smithi146.stdout: "objecter": { 2024-04-25T11:37:55.569 INFO:teuthology.orchestra.run.smithi146.stdout: "op_active": 0, 2024-04-25T11:37:55.569 INFO:teuthology.orchestra.run.smithi146.stdout: "op_laggy": 0, 2024-04-25T11:37:55.569 INFO:teuthology.orchestra.run.smithi146.stdout: "op_send": 3255, 2024-04-25T11:37:55.569 INFO:teuthology.orchestra.run.smithi146.stdout: "op_send_bytes": 6836762, 2024-04-25T11:37:55.569 INFO:teuthology.orchestra.run.smithi146.stdout: "op_resend": 37, 2024-04-25T11:37:55.569 INFO:teuthology.orchestra.run.smithi146.stdout: "op_reply": 3255, 2024-04-25T11:37:55.569 INFO:teuthology.orchestra.run.smithi146.stdout: "op_latency": { 2024-04-25T11:37:55.569 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 3255, 2024-04-25T11:37:55.569 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 28.044572222, 2024-04-25T11:37:55.569 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.008615844 2024-04-25T11:37:55.569 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.569 INFO:teuthology.orchestra.run.smithi146.stdout: "op_inflight": 0, 2024-04-25T11:37:55.570 INFO:teuthology.orchestra.run.smithi146.stdout: "oplen_avg": { 2024-04-25T11:37:55.570 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 3255, 2024-04-25T11:37:55.570 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 3299 2024-04-25T11:37:55.570 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.570 INFO:teuthology.orchestra.run.smithi146.stdout: "op": 3255, 2024-04-25T11:37:55.570 INFO:teuthology.orchestra.run.smithi146.stdout: "op_r": 1, 2024-04-25T11:37:55.570 INFO:teuthology.orchestra.run.smithi146.stdout: "op_w": 3254, 2024-04-25T11:37:55.570 INFO:teuthology.orchestra.run.smithi146.stdout: "op_rmw": 0, 2024-04-25T11:37:55.570 INFO:teuthology.orchestra.run.smithi146.stdout: "op_pg": 0, 2024-04-25T11:37:55.570 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_stat": 0, 2024-04-25T11:37:55.570 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_create": 11, 2024-04-25T11:37:55.570 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_read": 1, 2024-04-25T11:37:55.570 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_write": 2041, 2024-04-25T11:37:55.570 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_writefull": 54, 2024-04-25T11:37:55.570 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_writesame": 0, 2024-04-25T11:37:55.571 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_append": 0, 2024-04-25T11:37:55.571 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_zero": 0, 2024-04-25T11:37:55.571 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_truncate": 0, 2024-04-25T11:37:55.571 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_delete": 516, 2024-04-25T11:37:55.571 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_mapext": 0, 2024-04-25T11:37:55.571 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_sparse_read": 0, 2024-04-25T11:37:55.571 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_clonerange": 0, 2024-04-25T11:37:55.571 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_getxattr": 0, 2024-04-25T11:37:55.571 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_setxattr": 22, 2024-04-25T11:37:55.571 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_cmpxattr": 0, 2024-04-25T11:37:55.571 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_rmxattr": 0, 2024-04-25T11:37:55.571 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_resetxattrs": 0, 2024-04-25T11:37:55.571 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_call": 0, 2024-04-25T11:37:55.571 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_watch": 0, 2024-04-25T11:37:55.571 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_notify": 0, 2024-04-25T11:37:55.571 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T11:37:55.572 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_pgls": 0, 2024-04-25T11:37:55.572 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_pgls_filter": 0, 2024-04-25T11:37:55.572 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_other": 599, 2024-04-25T11:37:55.572 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_active": 0, 2024-04-25T11:37:55.572 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_send": 0, 2024-04-25T11:37:55.572 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_resend": 0, 2024-04-25T11:37:55.572 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_ping": 0, 2024-04-25T11:37:55.572 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_active": 0, 2024-04-25T11:37:55.572 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_send": 0, 2024-04-25T11:37:55.572 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_resend": 0, 2024-04-25T11:37:55.572 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_active": 0, 2024-04-25T11:37:55.572 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_send": 0, 2024-04-25T11:37:55.572 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_resend": 0, 2024-04-25T11:37:55.572 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_active": 0, 2024-04-25T11:37:55.572 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_send": 0, 2024-04-25T11:37:55.572 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_resend": 0, 2024-04-25T11:37:55.573 INFO:teuthology.orchestra.run.smithi146.stdout: "command_active": 0, 2024-04-25T11:37:55.573 INFO:teuthology.orchestra.run.smithi146.stdout: "command_send": 0, 2024-04-25T11:37:55.573 INFO:teuthology.orchestra.run.smithi146.stdout: "command_resend": 0, 2024-04-25T11:37:55.573 INFO:teuthology.orchestra.run.smithi146.stdout: "map_epoch": 84, 2024-04-25T11:37:55.573 INFO:teuthology.orchestra.run.smithi146.stdout: "map_full": 0, 2024-04-25T11:37:55.573 INFO:teuthology.orchestra.run.smithi146.stdout: "map_inc": 10, 2024-04-25T11:37:55.573 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_sessions": 12, 2024-04-25T11:37:55.573 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_session_open": 12, 2024-04-25T11:37:55.573 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_session_close": 0, 2024-04-25T11:37:55.573 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_laggy": 0, 2024-04-25T11:37:55.573 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_wr": 46, 2024-04-25T11:37:55.573 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_rd": 0, 2024-04-25T11:37:55.573 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_del": 9 2024-04-25T11:37:55.573 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.573 INFO:teuthology.orchestra.run.smithi146.stdout: "oft": { 2024-04-25T11:37:55.573 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_objs": 1, 2024-04-25T11:37:55.574 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_kv_pairs": 0, 2024-04-25T11:37:55.574 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_updates": 0, 2024-04-25T11:37:55.574 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_removes": 0 2024-04-25T11:37:55.574 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.574 INFO:teuthology.orchestra.run.smithi146.stdout: "osd-slow-ops": { 2024-04-25T11:37:55.574 INFO:teuthology.orchestra.run.smithi146.stdout: "slow_ops_count": 0 2024-04-25T11:37:55.574 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.574 INFO:teuthology.orchestra.run.smithi146.stdout: "purge_queue": { 2024-04-25T11:37:55.574 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_ops": 0, 2024-04-25T11:37:55.574 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_ops_high_water": 3, 2024-04-25T11:37:55.574 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing": 0, 2024-04-25T11:37:55.574 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_high_water": 2, 2024-04-25T11:37:55.574 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executed_ops": 3, 2024-04-25T11:37:55.574 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executed": 2, 2024-04-25T11:37:55.574 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_item_in_journal": 0 2024-04-25T11:37:55.575 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.575 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T11:37:55.575 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-04-25T11:37:55.575 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 104857600, 2024-04-25T11:37:55.575 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-04-25T11:37:55.575 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 7883, 2024-04-25T11:37:55.575 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 2666445, 2024-04-25T11:37:55.575 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-04-25T11:37:55.575 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 7883, 2024-04-25T11:37:55.575 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-04-25T11:37:55.575 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-04-25T11:37:55.575 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 7883, 2024-04-25T11:37:55.575 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 2666445, 2024-04-25T11:37:55.575 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-04-25T11:37:55.575 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:55.575 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:55.576 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:55.576 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-04-25T11:37:55.576 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.576 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-objecter_bytes": { 2024-04-25T11:37:55.576 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-04-25T11:37:55.576 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 104857600, 2024-04-25T11:37:55.576 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-04-25T11:37:55.576 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-04-25T11:37:55.576 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-04-25T11:37:55.576 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-04-25T11:37:55.576 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-04-25T11:37:55.576 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 3255, 2024-04-25T11:37:55.576 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 6836972, 2024-04-25T11:37:55.576 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 3255, 2024-04-25T11:37:55.576 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 6836972, 2024-04-25T11:37:55.577 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-04-25T11:37:55.577 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:55.577 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:55.577 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:55.577 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-04-25T11:37:55.577 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.577 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-objecter_ops": { 2024-04-25T11:37:55.577 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-04-25T11:37:55.577 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 1024, 2024-04-25T11:37:55.577 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-04-25T11:37:55.577 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-04-25T11:37:55.577 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-04-25T11:37:55.577 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-04-25T11:37:55.577 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-04-25T11:37:55.577 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 3255, 2024-04-25T11:37:55.578 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 3255, 2024-04-25T11:37:55.578 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 3255, 2024-04-25T11:37:55.578 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 3255, 2024-04-25T11:37:55.578 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-04-25T11:37:55.578 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:55.578 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:55.578 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:55.578 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-04-25T11:37:55.578 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.578 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-write_buf_throttle": { 2024-04-25T11:37:55.578 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-04-25T11:37:55.578 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 3758096384, 2024-04-25T11:37:55.578 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-04-25T11:37:55.578 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 2, 2024-04-25T11:37:55.579 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 210, 2024-04-25T11:37:55.579 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-04-25T11:37:55.579 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 2, 2024-04-25T11:37:55.579 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-04-25T11:37:55.579 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-04-25T11:37:55.579 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 1, 2024-04-25T11:37:55.579 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 210, 2024-04-25T11:37:55.579 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-04-25T11:37:55.579 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:55.579 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:55.579 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:55.579 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-04-25T11:37:55.579 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:55.579 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-write_buf_throttle-0x5555c0fac8f0": { 2024-04-25T11:37:55.579 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-04-25T11:37:55.580 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 3758096384, 2024-04-25T11:37:55.580 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-04-25T11:37:55.580 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 2057, 2024-04-25T11:37:55.580 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 6818138, 2024-04-25T11:37:55.580 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-04-25T11:37:55.580 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 2057, 2024-04-25T11:37:55.580 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-04-25T11:37:55.580 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-04-25T11:37:55.580 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 2040, 2024-04-25T11:37:55.580 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 6818138, 2024-04-25T11:37:55.580 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-04-25T11:37:55.580 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:55.580 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:55.580 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:55.580 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-04-25T11:37:55.580 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-04-25T11:37:55.581 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-04-25T11:37:55.724 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:37:55 smithi152 ceph-mon[32701]: pgmap v715: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-04-25T11:37:56.386 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-04-25T11:37:56.386 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-04-25T11:37:56.386 DEBUG:tasks.check_counter:Getting stats from e 2024-04-25T11:37:56.386 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-25T11:37:56.626 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T11:37:57.343 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:37:57 smithi146 ceph-mon[32821]: pgmap v716: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 853 B/s wr, 10 op/s 2024-04-25T11:37:57.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:37:57 smithi069 ceph-mon[29188]: pgmap v716: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 853 B/s wr, 10 op/s 2024-04-25T11:37:57.724 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:37:57 smithi152 ceph-mon[32701]: pgmap v716: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 853 B/s wr, 10 op/s 2024-04-25T11:37:58.421 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-04-25T11:37:58.421 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T11:37:58.421 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 945, 2024-04-25T11:37:58.421 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 883, 2024-04-25T11:37:58.421 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 1022536, 2024-04-25T11:37:58.422 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 420487, 2024-04-25T11:37:58.422 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 7, 2024-04-25T11:37:58.422 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 2, 2024-04-25T11:37:58.422 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.125120663, 2024-04-25T11:37:58.422 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.053435468, 2024-04-25T11:37:58.422 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.065095919, 2024-04-25T11:37:58.422 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.011848959, 2024-04-25T11:37:58.422 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:37:58.422 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 886, 2024-04-25T11:37:58.422 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.056340438, 2024-04-25T11:37:58.422 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000063589 2024-04-25T11:37:58.422 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.422 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:37:58.423 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 1217, 2024-04-25T11:37:58.423 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000386087, 2024-04-25T11:37:58.423 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000317 2024-04-25T11:37:58.423 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.423 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 859328, 2024-04-25T11:37:58.423 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 152320 2024-04-25T11:37:58.423 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.423 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T11:37:58.423 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 21, 2024-04-25T11:37:58.423 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 198, 2024-04-25T11:37:58.423 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 7339, 2024-04-25T11:37:58.423 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 394487, 2024-04-25T11:37:58.423 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 7, 2024-04-25T11:37:58.423 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 5, 2024-04-25T11:37:58.424 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.014695393, 2024-04-25T11:37:58.424 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.007978223, 2024-04-25T11:37:58.424 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.004915547, 2024-04-25T11:37:58.424 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.000178628, 2024-04-25T11:37:58.424 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:37:58.424 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 197, 2024-04-25T11:37:58.424 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.039784495, 2024-04-25T11:37:58.424 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000201951 2024-04-25T11:37:58.424 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.424 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:37:58.424 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 7, 2024-04-25T11:37:58.424 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000954, 2024-04-25T11:37:58.424 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000136 2024-04-25T11:37:58.424 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.425 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 4048, 2024-04-25T11:37:58.425 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 388816 2024-04-25T11:37:58.425 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.425 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T11:37:58.425 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 35, 2024-04-25T11:37:58.425 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 41, 2024-04-25T11:37:58.425 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 7780, 2024-04-25T11:37:58.425 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 63746, 2024-04-25T11:37:58.425 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 12, 2024-04-25T11:37:58.425 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 8, 2024-04-25T11:37:58.425 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.006336108, 2024-04-25T11:37:58.425 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.001025036, 2024-04-25T11:37:58.425 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.004807488, 2024-04-25T11:37:58.425 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.000372636, 2024-04-25T11:37:58.425 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:37:58.426 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 35, 2024-04-25T11:37:58.426 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.009828366, 2024-04-25T11:37:58.426 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000280810 2024-04-25T11:37:58.426 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.426 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:37:58.426 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 37, 2024-04-25T11:37:58.426 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000008440, 2024-04-25T11:37:58.426 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000228 2024-04-25T11:37:58.426 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.426 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 928, 2024-04-25T11:37:58.426 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 736 2024-04-25T11:37:58.426 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.426 INFO:teuthology.orchestra.run.smithi146.stdout: "cct": { 2024-04-25T11:37:58.427 INFO:teuthology.orchestra.run.smithi146.stdout: "total_workers": 1, 2024-04-25T11:37:58.427 INFO:teuthology.orchestra.run.smithi146.stdout: "unhealthy_workers": 0 2024-04-25T11:37:58.427 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.427 INFO:teuthology.orchestra.run.smithi146.stdout: "finisher-MDSRank": { 2024-04-25T11:37:58.427 INFO:teuthology.orchestra.run.smithi146.stdout: "queue_len": 0, 2024-04-25T11:37:58.427 INFO:teuthology.orchestra.run.smithi146.stdout: "complete_latency": { 2024-04-25T11:37:58.427 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 50, 2024-04-25T11:37:58.427 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.001629777, 2024-04-25T11:37:58.427 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000032595 2024-04-25T11:37:58.427 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-04-25T11:37:58.427 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.427 INFO:teuthology.orchestra.run.smithi146.stdout: "finisher-PurgeQueue": { 2024-04-25T11:37:58.427 INFO:teuthology.orchestra.run.smithi146.stdout: "queue_len": 0, 2024-04-25T11:37:58.427 INFO:teuthology.orchestra.run.smithi146.stdout: "complete_latency": { 2024-04-25T11:37:58.428 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 2, 2024-04-25T11:37:58.428 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000007868, 2024-04-25T11:37:58.428 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000003934 2024-04-25T11:37:58.428 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-04-25T11:37:58.428 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.428 INFO:teuthology.orchestra.run.smithi146.stdout: "mds": { 2024-04-25T11:37:58.428 INFO:teuthology.orchestra.run.smithi146.stdout: "request": 2, 2024-04-25T11:37:58.428 INFO:teuthology.orchestra.run.smithi146.stdout: "reply": 1, 2024-04-25T11:37:58.428 INFO:teuthology.orchestra.run.smithi146.stdout: "reply_latency": { 2024-04-25T11:37:58.428 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 1, 2024-04-25T11:37:58.428 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000923177, 2024-04-25T11:37:58.428 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000923177 2024-04-25T11:37:58.428 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.429 INFO:teuthology.orchestra.run.smithi146.stdout: "slow_reply": 0, 2024-04-25T11:37:58.429 INFO:teuthology.orchestra.run.smithi146.stdout: "forward": 1, 2024-04-25T11:37:58.429 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_fetch_complete": 0, 2024-04-25T11:37:58.429 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_fetch_keys": 0, 2024-04-25T11:37:58.429 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_commit": 11, 2024-04-25T11:37:58.429 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_split": 0, 2024-04-25T11:37:58.429 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_merge": 0, 2024-04-25T11:37:58.429 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes": 14, 2024-04-25T11:37:58.429 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_top": 0, 2024-04-25T11:37:58.429 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_bottom": 14, 2024-04-25T11:37:58.429 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_pin_tail": 0, 2024-04-25T11:37:58.429 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_pinned": 14, 2024-04-25T11:37:58.429 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_expired": 0, 2024-04-25T11:37:58.429 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_with_caps": 1, 2024-04-25T11:37:58.429 INFO:teuthology.orchestra.run.smithi146.stdout: "caps": 1, 2024-04-25T11:37:58.429 INFO:teuthology.orchestra.run.smithi146.stdout: "subtrees": 2, 2024-04-25T11:37:58.430 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse": 3, 2024-04-25T11:37:58.430 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_hit": 1, 2024-04-25T11:37:58.430 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_forward": 0, 2024-04-25T11:37:58.430 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_discover": 1, 2024-04-25T11:37:58.430 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_dir_fetch": 0, 2024-04-25T11:37:58.430 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_remote_ino": 0, 2024-04-25T11:37:58.430 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_lock": 0, 2024-04-25T11:37:58.430 INFO:teuthology.orchestra.run.smithi146.stdout: "load_cent": 0, 2024-04-25T11:37:58.430 INFO:teuthology.orchestra.run.smithi146.stdout: "q": 0, 2024-04-25T11:37:58.430 INFO:teuthology.orchestra.run.smithi146.stdout: "exported": 0, 2024-04-25T11:37:58.430 INFO:teuthology.orchestra.run.smithi146.stdout: "exported_inodes": 0, 2024-04-25T11:37:58.430 INFO:teuthology.orchestra.run.smithi146.stdout: "imported": 0, 2024-04-25T11:37:58.431 INFO:teuthology.orchestra.run.smithi146.stdout: "imported_inodes": 0, 2024-04-25T11:37:58.431 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_dir_fetch": 0, 2024-04-25T11:37:58.431 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_backtrace_fetch": 0, 2024-04-25T11:37:58.431 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_peer_discover": 0, 2024-04-25T11:37:58.431 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rfiles": 5, 2024-04-25T11:37:58.431 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rbytes": 298, 2024-04-25T11:37:58.431 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rsnaps": 0, 2024-04-25T11:37:58.431 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T11:37:58.431 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_set_tag": 0, 2024-04-25T11:37:58.431 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T11:37:58.431 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_inotable_repaired": 0, 2024-04-25T11:37:58.431 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dir_inodes": 0, 2024-04-25T11:37:58.431 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T11:37:58.431 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T11:37:58.431 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_file_inodes": 0, 2024-04-25T11:37:58.432 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_inode_file_caps": 0, 2024-04-25T11:37:58.432 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_revoke": 1, 2024-04-25T11:37:58.432 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_grant": 0, 2024-04-25T11:37:58.432 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T11:37:58.432 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T11:37:58.432 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T11:37:58.432 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_caps": 1, 2024-04-25T11:37:58.432 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_caps_dirty": 0, 2024-04-25T11:37:58.432 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_cap_release": 0, 2024-04-25T11:37:58.432 INFO:teuthology.orchestra.run.smithi146.stdout: "process_request_cap_release": 1 2024-04-25T11:37:58.432 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.432 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_cache": { 2024-04-25T11:37:58.432 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_update": 0, 2024-04-25T11:37:58.432 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_update_receipt": 0, 2024-04-25T11:37:58.432 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_try_discover": 0, 2024-04-25T11:37:58.433 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_send_discover": 2, 2024-04-25T11:37:58.433 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_handle_discover": 0, 2024-04-25T11:37:58.433 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays": 0, 2024-04-25T11:37:58.433 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays_delayed": 0, 2024-04-25T11:37:58.433 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays_enqueuing": 0, 2024-04-25T11:37:58.433 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_created": 0, 2024-04-25T11:37:58.433 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_enqueued": 0, 2024-04-25T11:37:58.433 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_reintegrated": 0, 2024-04-25T11:37:58.433 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_migrated": 0, 2024-04-25T11:37:58.433 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_processing": 0, 2024-04-25T11:37:58.433 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_enqueued": 0, 2024-04-25T11:37:58.433 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_prioritized": 0, 2024-04-25T11:37:58.433 INFO:teuthology.orchestra.run.smithi146.stdout: "recovery_started": 0, 2024-04-25T11:37:58.433 INFO:teuthology.orchestra.run.smithi146.stdout: "recovery_completed": 0, 2024-04-25T11:37:58.433 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_quiesce_path": 0, 2024-04-25T11:37:58.434 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_quiesce_inode": 0, 2024-04-25T11:37:58.434 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T11:37:58.434 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_exportdir": 0, 2024-04-25T11:37:58.434 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_flush": 0, 2024-04-25T11:37:58.434 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_fragmentdir": 0, 2024-04-25T11:37:58.434 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_fragstats": 0, 2024-04-25T11:37:58.434 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_inodestats": 0 2024-04-25T11:37:58.434 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.434 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_log": { 2024-04-25T11:37:58.434 INFO:teuthology.orchestra.run.smithi146.stdout: "evlrg": 0, 2024-04-25T11:37:58.434 INFO:teuthology.orchestra.run.smithi146.stdout: "evadd": 7, 2024-04-25T11:37:58.434 INFO:teuthology.orchestra.run.smithi146.stdout: "evex": 0, 2024-04-25T11:37:58.434 INFO:teuthology.orchestra.run.smithi146.stdout: "evtrm": 0, 2024-04-25T11:37:58.434 INFO:teuthology.orchestra.run.smithi146.stdout: "ev": 7, 2024-04-25T11:37:58.435 INFO:teuthology.orchestra.run.smithi146.stdout: "evexg": 0, 2024-04-25T11:37:58.435 INFO:teuthology.orchestra.run.smithi146.stdout: "evexd": 0, 2024-04-25T11:37:58.435 INFO:teuthology.orchestra.run.smithi146.stdout: "segadd": 2, 2024-04-25T11:37:58.435 INFO:teuthology.orchestra.run.smithi146.stdout: "segex": 0, 2024-04-25T11:37:58.435 INFO:teuthology.orchestra.run.smithi146.stdout: "segtrm": 0, 2024-04-25T11:37:58.435 INFO:teuthology.orchestra.run.smithi146.stdout: "seg": 2, 2024-04-25T11:37:58.435 INFO:teuthology.orchestra.run.smithi146.stdout: "segmjr": 2, 2024-04-25T11:37:58.435 INFO:teuthology.orchestra.run.smithi146.stdout: "segexg": 0, 2024-04-25T11:37:58.435 INFO:teuthology.orchestra.run.smithi146.stdout: "segexd": 0, 2024-04-25T11:37:58.435 INFO:teuthology.orchestra.run.smithi146.stdout: "expos": 4194304, 2024-04-25T11:37:58.435 INFO:teuthology.orchestra.run.smithi146.stdout: "wrpos": 4196449, 2024-04-25T11:37:58.435 INFO:teuthology.orchestra.run.smithi146.stdout: "rdpos": 0, 2024-04-25T11:37:58.435 INFO:teuthology.orchestra.run.smithi146.stdout: "jlat": { 2024-04-25T11:37:58.435 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 6, 2024-04-25T11:37:58.435 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.011892557, 2024-04-25T11:37:58.436 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.001982092 2024-04-25T11:37:58.436 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.436 INFO:teuthology.orchestra.run.smithi146.stdout: "replayed": 0 2024-04-25T11:37:58.436 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.436 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_mem": { 2024-04-25T11:37:58.436 INFO:teuthology.orchestra.run.smithi146.stdout: "ino": 17, 2024-04-25T11:37:58.436 INFO:teuthology.orchestra.run.smithi146.stdout: "ino+": 17, 2024-04-25T11:37:58.436 INFO:teuthology.orchestra.run.smithi146.stdout: "ino-": 0, 2024-04-25T11:37:58.436 INFO:teuthology.orchestra.run.smithi146.stdout: "dir": 15, 2024-04-25T11:37:58.436 INFO:teuthology.orchestra.run.smithi146.stdout: "dir+": 16, 2024-04-25T11:37:58.436 INFO:teuthology.orchestra.run.smithi146.stdout: "dir-": 1, 2024-04-25T11:37:58.436 INFO:teuthology.orchestra.run.smithi146.stdout: "dn": 14, 2024-04-25T11:37:58.436 INFO:teuthology.orchestra.run.smithi146.stdout: "dn+": 14, 2024-04-25T11:37:58.436 INFO:teuthology.orchestra.run.smithi146.stdout: "dn-": 0, 2024-04-25T11:37:58.436 INFO:teuthology.orchestra.run.smithi146.stdout: "cap": 1, 2024-04-25T11:37:58.437 INFO:teuthology.orchestra.run.smithi146.stdout: "cap+": 1, 2024-04-25T11:37:58.437 INFO:teuthology.orchestra.run.smithi146.stdout: "cap-": 0, 2024-04-25T11:37:58.437 INFO:teuthology.orchestra.run.smithi146.stdout: "rss": 123512, 2024-04-25T11:37:58.437 INFO:teuthology.orchestra.run.smithi146.stdout: "heap": 223632 2024-04-25T11:37:58.437 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.437 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_server": { 2024-04-25T11:37:58.437 INFO:teuthology.orchestra.run.smithi146.stdout: "dispatch_client_request": 3, 2024-04-25T11:37:58.437 INFO:teuthology.orchestra.run.smithi146.stdout: "dispatch_server_request": 0, 2024-04-25T11:37:58.437 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_request": 2, 2024-04-25T11:37:58.437 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_session": 18, 2024-04-25T11:37:58.437 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_peer_request": 0, 2024-04-25T11:37:58.437 INFO:teuthology.orchestra.run.smithi146.stdout: "req_create_latency": { 2024-04-25T11:37:58.437 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:58.437 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:58.438 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:58.438 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.438 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getattr_latency": { 2024-04-25T11:37:58.438 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 1, 2024-04-25T11:37:58.438 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000923177, 2024-04-25T11:37:58.438 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000923177 2024-04-25T11:37:58.438 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.438 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getfilelock_latency": { 2024-04-25T11:37:58.438 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:58.438 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:58.438 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:58.438 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.438 INFO:teuthology.orchestra.run.smithi146.stdout: "req_link_latency": { 2024-04-25T11:37:58.438 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:58.439 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:58.439 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:58.439 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.439 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookup_latency": { 2024-04-25T11:37:58.439 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:58.439 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:58.439 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:58.439 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.439 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookuphash_latency": { 2024-04-25T11:37:58.439 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:58.439 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:58.439 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:58.439 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.440 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupino_latency": { 2024-04-25T11:37:58.440 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:58.440 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:58.440 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:58.440 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.440 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupname_latency": { 2024-04-25T11:37:58.440 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:58.440 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:58.440 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:58.440 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.440 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupparent_latency": { 2024-04-25T11:37:58.440 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:58.440 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:58.440 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:58.440 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.441 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupsnap_latency": { 2024-04-25T11:37:58.441 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:58.441 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:58.441 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:58.441 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.441 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lssnap_latency": { 2024-04-25T11:37:58.441 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:58.441 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:58.441 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:58.441 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.441 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mkdir_latency": { 2024-04-25T11:37:58.441 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:58.442 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:58.442 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:58.442 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.442 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mknod_latency": { 2024-04-25T11:37:58.442 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:58.443 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:58.443 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:58.443 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.443 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mksnap_latency": { 2024-04-25T11:37:58.443 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:58.443 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:58.445 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:58.446 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.446 INFO:teuthology.orchestra.run.smithi146.stdout: "req_open_latency": { 2024-04-25T11:37:58.446 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:58.446 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:58.446 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:58.446 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.446 INFO:teuthology.orchestra.run.smithi146.stdout: "req_readdir_latency": { 2024-04-25T11:37:58.446 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:58.446 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:58.446 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:58.446 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.446 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rename_latency": { 2024-04-25T11:37:58.446 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:58.446 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:58.447 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:58.447 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.447 INFO:teuthology.orchestra.run.smithi146.stdout: "req_renamesnap_latency": { 2024-04-25T11:37:58.447 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:58.447 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:58.447 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:58.447 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.447 INFO:teuthology.orchestra.run.smithi146.stdout: "req_snapdiff_latency": { 2024-04-25T11:37:58.447 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:58.447 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:58.447 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:58.447 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.447 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmdir_latency": { 2024-04-25T11:37:58.447 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:58.447 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:58.448 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:58.448 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.448 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmsnap_latency": { 2024-04-25T11:37:58.448 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:58.448 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:58.448 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:58.448 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.448 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmxattr_latency": { 2024-04-25T11:37:58.448 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:58.448 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:58.448 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:58.448 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.448 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setattr_latency": { 2024-04-25T11:37:58.448 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:58.448 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:58.448 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:58.449 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.449 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setdirlayout_latency": { 2024-04-25T11:37:58.449 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:58.449 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:58.449 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:58.449 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.449 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setfilelock_latency": { 2024-04-25T11:37:58.449 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:58.449 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:58.449 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:58.449 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.449 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setlayout_latency": { 2024-04-25T11:37:58.449 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:58.449 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:58.450 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:58.450 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.450 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setxattr_latency": { 2024-04-25T11:37:58.450 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:58.450 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:58.450 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:58.450 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.450 INFO:teuthology.orchestra.run.smithi146.stdout: "req_symlink_latency": { 2024-04-25T11:37:58.450 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:58.450 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:58.450 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:58.450 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.450 INFO:teuthology.orchestra.run.smithi146.stdout: "req_unlink_latency": { 2024-04-25T11:37:58.451 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:58.451 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:58.451 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:58.451 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.451 INFO:teuthology.orchestra.run.smithi146.stdout: "cap_revoke_eviction": 0, 2024-04-25T11:37:58.451 INFO:teuthology.orchestra.run.smithi146.stdout: "cap_acquisition_throttle": 0, 2024-04-25T11:37:58.451 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getvxattr_latency": { 2024-04-25T11:37:58.451 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:58.451 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:58.451 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:58.451 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-04-25T11:37:58.451 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.451 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_sessions": { 2024-04-25T11:37:58.451 INFO:teuthology.orchestra.run.smithi146.stdout: "session_count": 1, 2024-04-25T11:37:58.451 INFO:teuthology.orchestra.run.smithi146.stdout: "session_add": 3, 2024-04-25T11:37:58.451 INFO:teuthology.orchestra.run.smithi146.stdout: "session_remove": 2, 2024-04-25T11:37:58.452 INFO:teuthology.orchestra.run.smithi146.stdout: "sessions_open": 1, 2024-04-25T11:37:58.452 INFO:teuthology.orchestra.run.smithi146.stdout: "sessions_stale": 0, 2024-04-25T11:37:58.452 INFO:teuthology.orchestra.run.smithi146.stdout: "total_load": 1, 2024-04-25T11:37:58.452 INFO:teuthology.orchestra.run.smithi146.stdout: "average_load": 1, 2024-04-25T11:37:58.452 INFO:teuthology.orchestra.run.smithi146.stdout: "avg_session_uptime": 709, 2024-04-25T11:37:58.452 INFO:teuthology.orchestra.run.smithi146.stdout: "mdthresh_evicted": 0 2024-04-25T11:37:58.452 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.452 INFO:teuthology.orchestra.run.smithi146.stdout: "mempool": { 2024-04-25T11:37:58.452 INFO:teuthology.orchestra.run.smithi146.stdout: "bloom_filter_bytes": 0, 2024-04-25T11:37:58.452 INFO:teuthology.orchestra.run.smithi146.stdout: "bloom_filter_items": 0, 2024-04-25T11:37:58.452 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T11:37:58.452 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_alloc_items": 0, 2024-04-25T11:37:58.452 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T11:37:58.452 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_data_items": 0, 2024-04-25T11:37:58.453 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T11:37:58.453 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T11:37:58.453 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T11:37:58.453 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T11:37:58.453 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T11:37:58.453 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_other_items": 0, 2024-04-25T11:37:58.453 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T11:37:58.453 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T11:37:58.453 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_extent_bytes": 0, 2024-04-25T11:37:58.453 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_extent_items": 0, 2024-04-25T11:37:58.453 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_blob_bytes": 0, 2024-04-25T11:37:58.453 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_blob_items": 0, 2024-04-25T11:37:58.453 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T11:37:58.454 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T11:37:58.454 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T11:37:58.454 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T11:37:58.454 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T11:37:58.454 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_items": 0, 2024-04-25T11:37:58.454 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_txc_bytes": 0, 2024-04-25T11:37:58.454 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_txc_items": 0, 2024-04-25T11:37:58.454 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T11:37:58.454 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T11:37:58.454 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_bytes": 0, 2024-04-25T11:37:58.454 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_items": 0, 2024-04-25T11:37:58.454 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_bytes": 0, 2024-04-25T11:37:58.454 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_items": 0, 2024-04-25T11:37:58.455 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T11:37:58.455 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_reader_items": 0, 2024-04-25T11:37:58.455 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T11:37:58.455 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_writer_items": 0, 2024-04-25T11:37:58.455 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_anon_bytes": 238879, 2024-04-25T11:37:58.455 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_anon_items": 80, 2024-04-25T11:37:58.455 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_meta_bytes": 0, 2024-04-25T11:37:58.455 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_meta_items": 0, 2024-04-25T11:37:58.455 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_bytes": 0, 2024-04-25T11:37:58.455 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_items": 0, 2024-04-25T11:37:58.455 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mapbl_bytes": 0, 2024-04-25T11:37:58.455 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mapbl_items": 0, 2024-04-25T11:37:58.455 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pglog_bytes": 0, 2024-04-25T11:37:58.455 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pglog_items": 0, 2024-04-25T11:37:58.455 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_bytes": 9088, 2024-04-25T11:37:58.455 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_items": 175, 2024-04-25T11:37:58.456 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T11:37:58.456 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_mapping_items": 0, 2024-04-25T11:37:58.456 INFO:teuthology.orchestra.run.smithi146.stdout: "pgmap_bytes": 0, 2024-04-25T11:37:58.456 INFO:teuthology.orchestra.run.smithi146.stdout: "pgmap_items": 0, 2024-04-25T11:37:58.456 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_co_bytes": 68037, 2024-04-25T11:37:58.456 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_co_items": 288, 2024-04-25T11:37:58.456 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_1_bytes": 0, 2024-04-25T11:37:58.456 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_1_items": 0, 2024-04-25T11:37:58.456 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_2_bytes": 0, 2024-04-25T11:37:58.456 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_2_items": 0 2024-04-25T11:37:58.456 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.456 INFO:teuthology.orchestra.run.smithi146.stdout: "objecter": { 2024-04-25T11:37:58.456 INFO:teuthology.orchestra.run.smithi146.stdout: "op_active": 0, 2024-04-25T11:37:58.456 INFO:teuthology.orchestra.run.smithi146.stdout: "op_laggy": 0, 2024-04-25T11:37:58.456 INFO:teuthology.orchestra.run.smithi146.stdout: "op_send": 44, 2024-04-25T11:37:58.457 INFO:teuthology.orchestra.run.smithi146.stdout: "op_send_bytes": 12806, 2024-04-25T11:37:58.457 INFO:teuthology.orchestra.run.smithi146.stdout: "op_resend": 33, 2024-04-25T11:37:58.457 INFO:teuthology.orchestra.run.smithi146.stdout: "op_reply": 44, 2024-04-25T11:37:58.457 INFO:teuthology.orchestra.run.smithi146.stdout: "op_latency": { 2024-04-25T11:37:58.457 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 44, 2024-04-25T11:37:58.457 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.220996683, 2024-04-25T11:37:58.457 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.005022651 2024-04-25T11:37:58.457 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.457 INFO:teuthology.orchestra.run.smithi146.stdout: "op_inflight": 0, 2024-04-25T11:37:58.457 INFO:teuthology.orchestra.run.smithi146.stdout: "oplen_avg": { 2024-04-25T11:37:58.457 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 44, 2024-04-25T11:37:58.457 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 67 2024-04-25T11:37:58.457 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.457 INFO:teuthology.orchestra.run.smithi146.stdout: "op": 44, 2024-04-25T11:37:58.457 INFO:teuthology.orchestra.run.smithi146.stdout: "op_r": 0, 2024-04-25T11:37:58.458 INFO:teuthology.orchestra.run.smithi146.stdout: "op_w": 44, 2024-04-25T11:37:58.458 INFO:teuthology.orchestra.run.smithi146.stdout: "op_rmw": 0, 2024-04-25T11:37:58.458 INFO:teuthology.orchestra.run.smithi146.stdout: "op_pg": 0, 2024-04-25T11:37:58.458 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_stat": 0, 2024-04-25T11:37:58.458 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_create": 10, 2024-04-25T11:37:58.458 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_read": 0, 2024-04-25T11:37:58.458 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_write": 6, 2024-04-25T11:37:58.458 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_writefull": 8, 2024-04-25T11:37:58.458 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_writesame": 0, 2024-04-25T11:37:58.458 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_append": 0, 2024-04-25T11:37:58.458 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_zero": 0, 2024-04-25T11:37:58.458 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_truncate": 0, 2024-04-25T11:37:58.458 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_delete": 6, 2024-04-25T11:37:58.459 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_mapext": 0, 2024-04-25T11:37:58.459 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_sparse_read": 0, 2024-04-25T11:37:58.459 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_clonerange": 0, 2024-04-25T11:37:58.459 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_getxattr": 0, 2024-04-25T11:37:58.459 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_setxattr": 20, 2024-04-25T11:37:58.459 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_cmpxattr": 0, 2024-04-25T11:37:58.459 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_rmxattr": 0, 2024-04-25T11:37:58.459 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_resetxattrs": 0, 2024-04-25T11:37:58.459 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_call": 0, 2024-04-25T11:37:58.459 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_watch": 0, 2024-04-25T11:37:58.459 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_notify": 0, 2024-04-25T11:37:58.459 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T11:37:58.460 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_pgls": 0, 2024-04-25T11:37:58.460 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_pgls_filter": 0, 2024-04-25T11:37:58.460 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_other": 0, 2024-04-25T11:37:58.460 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_active": 0, 2024-04-25T11:37:58.460 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_send": 0, 2024-04-25T11:37:58.460 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_resend": 0, 2024-04-25T11:37:58.460 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_ping": 0, 2024-04-25T11:37:58.460 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_active": 0, 2024-04-25T11:37:58.460 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_send": 0, 2024-04-25T11:37:58.460 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_resend": 0, 2024-04-25T11:37:58.460 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_active": 0, 2024-04-25T11:37:58.460 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_send": 0, 2024-04-25T11:37:58.460 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_resend": 0, 2024-04-25T11:37:58.460 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_active": 0, 2024-04-25T11:37:58.460 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_send": 0, 2024-04-25T11:37:58.461 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_resend": 0, 2024-04-25T11:37:58.461 INFO:teuthology.orchestra.run.smithi146.stdout: "command_active": 0, 2024-04-25T11:37:58.461 INFO:teuthology.orchestra.run.smithi146.stdout: "command_send": 0, 2024-04-25T11:37:58.461 INFO:teuthology.orchestra.run.smithi146.stdout: "command_resend": 0, 2024-04-25T11:37:58.461 INFO:teuthology.orchestra.run.smithi146.stdout: "map_epoch": 84, 2024-04-25T11:37:58.461 INFO:teuthology.orchestra.run.smithi146.stdout: "map_full": 0, 2024-04-25T11:37:58.461 INFO:teuthology.orchestra.run.smithi146.stdout: "map_inc": 4, 2024-04-25T11:37:58.461 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_sessions": 8, 2024-04-25T11:37:58.461 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_session_open": 8, 2024-04-25T11:37:58.461 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_session_close": 0, 2024-04-25T11:37:58.461 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_laggy": 0, 2024-04-25T11:37:58.461 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_wr": 16, 2024-04-25T11:37:58.461 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_rd": 0, 2024-04-25T11:37:58.461 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_del": 1 2024-04-25T11:37:58.461 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.462 INFO:teuthology.orchestra.run.smithi146.stdout: "oft": { 2024-04-25T11:37:58.462 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_objs": 1, 2024-04-25T11:37:58.462 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_kv_pairs": 0, 2024-04-25T11:37:58.462 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_updates": 0, 2024-04-25T11:37:58.462 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_removes": 0 2024-04-25T11:37:58.462 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.462 INFO:teuthology.orchestra.run.smithi146.stdout: "osd-slow-ops": { 2024-04-25T11:37:58.462 INFO:teuthology.orchestra.run.smithi146.stdout: "slow_ops_count": 0 2024-04-25T11:37:58.462 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.462 INFO:teuthology.orchestra.run.smithi146.stdout: "purge_queue": { 2024-04-25T11:37:58.462 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_ops": 0, 2024-04-25T11:37:58.462 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T11:37:58.463 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing": 0, 2024-04-25T11:37:58.463 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_high_water": 0, 2024-04-25T11:37:58.463 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executed_ops": 0, 2024-04-25T11:37:58.463 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executed": 0, 2024-04-25T11:37:58.463 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_item_in_journal": 0 2024-04-25T11:37:58.463 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.463 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T11:37:58.463 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-04-25T11:37:58.463 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 104857600, 2024-04-25T11:37:58.463 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-04-25T11:37:58.463 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 997, 2024-04-25T11:37:58.463 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 930120, 2024-04-25T11:37:58.463 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-04-25T11:37:58.463 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 997, 2024-04-25T11:37:58.463 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-04-25T11:37:58.464 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-04-25T11:37:58.464 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 997, 2024-04-25T11:37:58.464 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 930120, 2024-04-25T11:37:58.464 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-04-25T11:37:58.464 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:58.464 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:58.464 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:58.464 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-04-25T11:37:58.464 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.464 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-objecter_bytes": { 2024-04-25T11:37:58.464 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-04-25T11:37:58.464 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 104857600, 2024-04-25T11:37:58.464 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-04-25T11:37:58.464 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-04-25T11:37:58.464 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-04-25T11:37:58.465 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-04-25T11:37:58.465 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-04-25T11:37:58.465 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 44, 2024-04-25T11:37:58.465 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 12806, 2024-04-25T11:37:58.465 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 44, 2024-04-25T11:37:58.465 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 12806, 2024-04-25T11:37:58.465 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-04-25T11:37:58.465 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:58.465 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:58.465 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:58.465 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-04-25T11:37:58.465 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.465 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-objecter_ops": { 2024-04-25T11:37:58.465 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-04-25T11:37:58.466 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 1024, 2024-04-25T11:37:58.466 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-04-25T11:37:58.466 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-04-25T11:37:58.466 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-04-25T11:37:58.466 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-04-25T11:37:58.466 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-04-25T11:37:58.466 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 44, 2024-04-25T11:37:58.466 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 44, 2024-04-25T11:37:58.466 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 44, 2024-04-25T11:37:58.466 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 44, 2024-04-25T11:37:58.466 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-04-25T11:37:58.466 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:58.466 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:58.467 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:58.467 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-04-25T11:37:58.467 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.467 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-write_buf_throttle": { 2024-04-25T11:37:58.467 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-04-25T11:37:58.467 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 3758096384, 2024-04-25T11:37:58.467 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-04-25T11:37:58.467 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-04-25T11:37:58.467 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-04-25T11:37:58.467 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-04-25T11:37:58.467 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-04-25T11:37:58.467 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-04-25T11:37:58.467 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-04-25T11:37:58.467 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 0, 2024-04-25T11:37:58.467 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 0, 2024-04-25T11:37:58.468 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-04-25T11:37:58.468 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:58.468 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:58.468 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:58.468 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-04-25T11:37:58.468 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:37:58.468 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-write_buf_throttle-0x55bde7fb2750": { 2024-04-25T11:37:58.468 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-04-25T11:37:58.468 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 3758096384, 2024-04-25T11:37:58.468 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-04-25T11:37:58.468 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 7, 2024-04-25T11:37:58.468 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 2145, 2024-04-25T11:37:58.468 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-04-25T11:37:58.468 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 7, 2024-04-25T11:37:58.468 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-04-25T11:37:58.469 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-04-25T11:37:58.469 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 6, 2024-04-25T11:37:58.469 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 2145, 2024-04-25T11:37:58.469 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-04-25T11:37:58.469 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:37:58.469 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:37:58.469 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:37:58.469 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-04-25T11:37:58.469 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-04-25T11:37:58.469 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-04-25T11:37:59.244 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-25T11:37:59.244 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-25T11:37:59.244 DEBUG:tasks.check_counter:Getting stats from h 2024-04-25T11:37:59.245 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-25T11:37:59.485 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T11:37:59.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:37:59 smithi146 ceph-mon[32821]: pgmap v717: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-25T11:37:59.686 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:37:59 smithi069 ceph-mon[29188]: pgmap v717: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-25T11:37:59.724 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:37:59 smithi152 ceph-mon[32701]: pgmap v717: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-25T11:38:01.291 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:01 smithi146 ceph-mon[32821]: pgmap v718: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-25T11:38:01.291 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-04-25T11:38:01.291 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T11:38:01.291 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 473, 2024-04-25T11:38:01.291 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 565, 2024-04-25T11:38:01.291 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 114662, 2024-04-25T11:38:01.291 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 237951, 2024-04-25T11:38:01.291 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 8, 2024-04-25T11:38:01.292 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 3, 2024-04-25T11:38:01.292 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.076700616, 2024-04-25T11:38:01.292 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.035907942, 2024-04-25T11:38:01.292 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.032913251, 2024-04-25T11:38:01.292 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.005559803, 2024-04-25T11:38:01.292 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:38:01.292 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 564, 2024-04-25T11:38:01.292 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.045851914, 2024-04-25T11:38:01.292 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000081297 2024-04-25T11:38:01.292 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.292 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:38:01.292 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 978, 2024-04-25T11:38:01.292 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000332712, 2024-04-25T11:38:01.292 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000340 2024-04-25T11:38:01.292 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.293 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 2960, 2024-04-25T11:38:01.293 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T11:38:01.293 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.293 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T11:38:01.293 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 31, 2024-04-25T11:38:01.293 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 209, 2024-04-25T11:38:01.293 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 6548, 2024-04-25T11:38:01.293 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 445469, 2024-04-25T11:38:01.293 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 13, 2024-04-25T11:38:01.293 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 7, 2024-04-25T11:38:01.293 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.017162186, 2024-04-25T11:38:01.293 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.008460660, 2024-04-25T11:38:01.293 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.006125953, 2024-04-25T11:38:01.293 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.000879340, 2024-04-25T11:38:01.293 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:38:01.293 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 207, 2024-04-25T11:38:01.294 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.028880402, 2024-04-25T11:38:01.294 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000139518 2024-04-25T11:38:01.294 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.294 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:38:01.294 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 35, 2024-04-25T11:38:01.294 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000008938, 2024-04-25T11:38:01.294 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000255 2024-04-25T11:38:01.294 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.294 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-04-25T11:38:01.294 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 389808 2024-04-25T11:38:01.294 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.294 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T11:38:01.294 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 299, 2024-04-25T11:38:01.294 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 285, 2024-04-25T11:38:01.294 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 870995, 2024-04-25T11:38:01.295 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 161884, 2024-04-25T11:38:01.295 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 6, 2024-04-25T11:38:01.295 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 5, 2024-04-25T11:38:01.295 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.050549518, 2024-04-25T11:38:01.295 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.016507028, 2024-04-25T11:38:01.295 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.030028965, 2024-04-25T11:38:01.295 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.006952727, 2024-04-25T11:38:01.295 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:38:01.295 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 284, 2024-04-25T11:38:01.295 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.029031535, 2024-04-25T11:38:01.295 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000102223 2024-04-25T11:38:01.295 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.295 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:38:01.295 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:01.295 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:01.295 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:01.296 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.296 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 867984, 2024-04-25T11:38:01.296 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 151040 2024-04-25T11:38:01.296 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.296 INFO:teuthology.orchestra.run.smithi146.stdout: "cct": { 2024-04-25T11:38:01.296 INFO:teuthology.orchestra.run.smithi146.stdout: "total_workers": 1, 2024-04-25T11:38:01.296 INFO:teuthology.orchestra.run.smithi146.stdout: "unhealthy_workers": 0 2024-04-25T11:38:01.296 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.296 INFO:teuthology.orchestra.run.smithi146.stdout: "finisher-MDSRank": { 2024-04-25T11:38:01.296 INFO:teuthology.orchestra.run.smithi146.stdout: "queue_len": 0, 2024-04-25T11:38:01.296 INFO:teuthology.orchestra.run.smithi146.stdout: "complete_latency": { 2024-04-25T11:38:01.296 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 32, 2024-04-25T11:38:01.296 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.002373880, 2024-04-25T11:38:01.296 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000074183 2024-04-25T11:38:01.296 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-04-25T11:38:01.296 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.297 INFO:teuthology.orchestra.run.smithi146.stdout: "finisher-PurgeQueue": { 2024-04-25T11:38:01.297 INFO:teuthology.orchestra.run.smithi146.stdout: "queue_len": 0, 2024-04-25T11:38:01.297 INFO:teuthology.orchestra.run.smithi146.stdout: "complete_latency": { 2024-04-25T11:38:01.297 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 2, 2024-04-25T11:38:01.297 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000008119, 2024-04-25T11:38:01.297 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000004059 2024-04-25T11:38:01.297 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-04-25T11:38:01.297 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.297 INFO:teuthology.orchestra.run.smithi146.stdout: "mds": { 2024-04-25T11:38:01.297 INFO:teuthology.orchestra.run.smithi146.stdout: "request": 2, 2024-04-25T11:38:01.297 INFO:teuthology.orchestra.run.smithi146.stdout: "reply": 2, 2024-04-25T11:38:01.297 INFO:teuthology.orchestra.run.smithi146.stdout: "reply_latency": { 2024-04-25T11:38:01.297 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 2, 2024-04-25T11:38:01.297 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.002220810, 2024-04-25T11:38:01.297 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.001110405 2024-04-25T11:38:01.297 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.298 INFO:teuthology.orchestra.run.smithi146.stdout: "slow_reply": 0, 2024-04-25T11:38:01.298 INFO:teuthology.orchestra.run.smithi146.stdout: "forward": 0, 2024-04-25T11:38:01.298 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_fetch_complete": 0, 2024-04-25T11:38:01.298 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_fetch_keys": 0, 2024-04-25T11:38:01.298 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_commit": 11, 2024-04-25T11:38:01.298 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_split": 0, 2024-04-25T11:38:01.298 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_merge": 0, 2024-04-25T11:38:01.298 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes": 12, 2024-04-25T11:38:01.298 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_top": 0, 2024-04-25T11:38:01.298 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_bottom": 12, 2024-04-25T11:38:01.298 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_pin_tail": 0, 2024-04-25T11:38:01.298 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_pinned": 12, 2024-04-25T11:38:01.298 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_expired": 0, 2024-04-25T11:38:01.298 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_with_caps": 2, 2024-04-25T11:38:01.299 INFO:teuthology.orchestra.run.smithi146.stdout: "caps": 2, 2024-04-25T11:38:01.299 INFO:teuthology.orchestra.run.smithi146.stdout: "subtrees": 2, 2024-04-25T11:38:01.299 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse": 4, 2024-04-25T11:38:01.299 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_hit": 3, 2024-04-25T11:38:01.299 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_forward": 0, 2024-04-25T11:38:01.299 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_discover": 1, 2024-04-25T11:38:01.299 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_dir_fetch": 0, 2024-04-25T11:38:01.299 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_remote_ino": 0, 2024-04-25T11:38:01.299 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_lock": 0, 2024-04-25T11:38:01.299 INFO:teuthology.orchestra.run.smithi146.stdout: "load_cent": 0, 2024-04-25T11:38:01.299 INFO:teuthology.orchestra.run.smithi146.stdout: "q": 0, 2024-04-25T11:38:01.299 INFO:teuthology.orchestra.run.smithi146.stdout: "exported": 0, 2024-04-25T11:38:01.299 INFO:teuthology.orchestra.run.smithi146.stdout: "exported_inodes": 0, 2024-04-25T11:38:01.299 INFO:teuthology.orchestra.run.smithi146.stdout: "imported": 0, 2024-04-25T11:38:01.300 INFO:teuthology.orchestra.run.smithi146.stdout: "imported_inodes": 0, 2024-04-25T11:38:01.300 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_dir_fetch": 0, 2024-04-25T11:38:01.300 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_backtrace_fetch": 0, 2024-04-25T11:38:01.300 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_peer_discover": 0, 2024-04-25T11:38:01.300 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rfiles": 5, 2024-04-25T11:38:01.300 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rbytes": 298, 2024-04-25T11:38:01.300 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rsnaps": 0, 2024-04-25T11:38:01.300 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T11:38:01.300 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_set_tag": 0, 2024-04-25T11:38:01.300 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T11:38:01.300 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_inotable_repaired": 0, 2024-04-25T11:38:01.300 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dir_inodes": 0, 2024-04-25T11:38:01.300 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T11:38:01.300 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T11:38:01.301 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_file_inodes": 0, 2024-04-25T11:38:01.301 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_inode_file_caps": 0, 2024-04-25T11:38:01.301 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T11:38:01.301 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_grant": 0, 2024-04-25T11:38:01.301 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T11:38:01.301 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T11:38:01.301 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T11:38:01.301 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_caps": 0, 2024-04-25T11:38:01.301 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_caps_dirty": 0, 2024-04-25T11:38:01.301 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_cap_release": 0, 2024-04-25T11:38:01.301 INFO:teuthology.orchestra.run.smithi146.stdout: "process_request_cap_release": 0 2024-04-25T11:38:01.301 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.301 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_cache": { 2024-04-25T11:38:01.301 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_update": 0, 2024-04-25T11:38:01.301 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_update_receipt": 0, 2024-04-25T11:38:01.301 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_try_discover": 0, 2024-04-25T11:38:01.302 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_send_discover": 2, 2024-04-25T11:38:01.302 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_handle_discover": 0, 2024-04-25T11:38:01.302 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays": 0, 2024-04-25T11:38:01.302 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays_delayed": 0, 2024-04-25T11:38:01.302 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays_enqueuing": 0, 2024-04-25T11:38:01.302 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_created": 0, 2024-04-25T11:38:01.302 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_enqueued": 0, 2024-04-25T11:38:01.302 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_reintegrated": 0, 2024-04-25T11:38:01.302 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_migrated": 0, 2024-04-25T11:38:01.302 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_processing": 0, 2024-04-25T11:38:01.302 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_enqueued": 0, 2024-04-25T11:38:01.302 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_prioritized": 0, 2024-04-25T11:38:01.302 INFO:teuthology.orchestra.run.smithi146.stdout: "recovery_started": 0, 2024-04-25T11:38:01.302 INFO:teuthology.orchestra.run.smithi146.stdout: "recovery_completed": 0, 2024-04-25T11:38:01.302 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_quiesce_path": 0, 2024-04-25T11:38:01.303 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_quiesce_inode": 0, 2024-04-25T11:38:01.303 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T11:38:01.303 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_exportdir": 0, 2024-04-25T11:38:01.303 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_flush": 0, 2024-04-25T11:38:01.303 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_fragmentdir": 0, 2024-04-25T11:38:01.303 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_fragstats": 0, 2024-04-25T11:38:01.303 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_inodestats": 0 2024-04-25T11:38:01.303 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.303 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_log": { 2024-04-25T11:38:01.303 INFO:teuthology.orchestra.run.smithi146.stdout: "evlrg": 0, 2024-04-25T11:38:01.303 INFO:teuthology.orchestra.run.smithi146.stdout: "evadd": 5, 2024-04-25T11:38:01.303 INFO:teuthology.orchestra.run.smithi146.stdout: "evex": 0, 2024-04-25T11:38:01.303 INFO:teuthology.orchestra.run.smithi146.stdout: "evtrm": 0, 2024-04-25T11:38:01.303 INFO:teuthology.orchestra.run.smithi146.stdout: "ev": 5, 2024-04-25T11:38:01.303 INFO:teuthology.orchestra.run.smithi146.stdout: "evexg": 0, 2024-04-25T11:38:01.303 INFO:teuthology.orchestra.run.smithi146.stdout: "evexd": 0, 2024-04-25T11:38:01.303 INFO:teuthology.orchestra.run.smithi146.stdout: "segadd": 2, 2024-04-25T11:38:01.304 INFO:teuthology.orchestra.run.smithi146.stdout: "segex": 0, 2024-04-25T11:38:01.304 INFO:teuthology.orchestra.run.smithi146.stdout: "segtrm": 0, 2024-04-25T11:38:01.304 INFO:teuthology.orchestra.run.smithi146.stdout: "seg": 2, 2024-04-25T11:38:01.304 INFO:teuthology.orchestra.run.smithi146.stdout: "segmjr": 2, 2024-04-25T11:38:01.304 INFO:teuthology.orchestra.run.smithi146.stdout: "segexg": 0, 2024-04-25T11:38:01.304 INFO:teuthology.orchestra.run.smithi146.stdout: "segexd": 0, 2024-04-25T11:38:01.304 INFO:teuthology.orchestra.run.smithi146.stdout: "expos": 4194304, 2024-04-25T11:38:01.304 INFO:teuthology.orchestra.run.smithi146.stdout: "wrpos": 4195917, 2024-04-25T11:38:01.304 INFO:teuthology.orchestra.run.smithi146.stdout: "rdpos": 0, 2024-04-25T11:38:01.304 INFO:teuthology.orchestra.run.smithi146.stdout: "jlat": { 2024-04-25T11:38:01.304 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 4, 2024-04-25T11:38:01.304 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.009448944, 2024-04-25T11:38:01.304 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.002362236 2024-04-25T11:38:01.304 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.304 INFO:teuthology.orchestra.run.smithi146.stdout: "replayed": 0 2024-04-25T11:38:01.304 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.304 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_mem": { 2024-04-25T11:38:01.305 INFO:teuthology.orchestra.run.smithi146.stdout: "ino": 15, 2024-04-25T11:38:01.305 INFO:teuthology.orchestra.run.smithi146.stdout: "ino+": 15, 2024-04-25T11:38:01.305 INFO:teuthology.orchestra.run.smithi146.stdout: "ino-": 0, 2024-04-25T11:38:01.305 INFO:teuthology.orchestra.run.smithi146.stdout: "dir": 13, 2024-04-25T11:38:01.305 INFO:teuthology.orchestra.run.smithi146.stdout: "dir+": 14, 2024-04-25T11:38:01.305 INFO:teuthology.orchestra.run.smithi146.stdout: "dir-": 1, 2024-04-25T11:38:01.305 INFO:teuthology.orchestra.run.smithi146.stdout: "dn": 12, 2024-04-25T11:38:01.305 INFO:teuthology.orchestra.run.smithi146.stdout: "dn+": 12, 2024-04-25T11:38:01.305 INFO:teuthology.orchestra.run.smithi146.stdout: "dn-": 0, 2024-04-25T11:38:01.305 INFO:teuthology.orchestra.run.smithi146.stdout: "cap": 2, 2024-04-25T11:38:01.305 INFO:teuthology.orchestra.run.smithi146.stdout: "cap+": 2, 2024-04-25T11:38:01.305 INFO:teuthology.orchestra.run.smithi146.stdout: "cap-": 0, 2024-04-25T11:38:01.305 INFO:teuthology.orchestra.run.smithi146.stdout: "rss": 120540, 2024-04-25T11:38:01.305 INFO:teuthology.orchestra.run.smithi146.stdout: "heap": 223632 2024-04-25T11:38:01.305 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.305 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_server": { 2024-04-25T11:38:01.305 INFO:teuthology.orchestra.run.smithi146.stdout: "dispatch_client_request": 6, 2024-04-25T11:38:01.306 INFO:teuthology.orchestra.run.smithi146.stdout: "dispatch_server_request": 0, 2024-04-25T11:38:01.306 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_request": 2, 2024-04-25T11:38:01.306 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_session": 14, 2024-04-25T11:38:01.306 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_peer_request": 1, 2024-04-25T11:38:01.306 INFO:teuthology.orchestra.run.smithi146.stdout: "req_create_latency": { 2024-04-25T11:38:01.306 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:01.306 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:01.306 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:01.306 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.306 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getattr_latency": { 2024-04-25T11:38:01.306 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 2, 2024-04-25T11:38:01.306 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.002220810, 2024-04-25T11:38:01.306 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.001110405 2024-04-25T11:38:01.306 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.306 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getfilelock_latency": { 2024-04-25T11:38:01.306 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:01.307 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:01.307 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:01.307 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.307 INFO:teuthology.orchestra.run.smithi146.stdout: "req_link_latency": { 2024-04-25T11:38:01.307 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:01.307 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:01.307 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:01.307 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.307 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookup_latency": { 2024-04-25T11:38:01.307 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:01.307 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:01.307 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:01.307 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.307 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookuphash_latency": { 2024-04-25T11:38:01.307 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:01.308 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:01.308 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:01.308 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.308 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupino_latency": { 2024-04-25T11:38:01.308 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:01.308 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:01.308 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:01.308 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.308 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupname_latency": { 2024-04-25T11:38:01.308 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:01.308 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:01.308 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:01.308 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.308 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupparent_latency": { 2024-04-25T11:38:01.308 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:01.308 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:01.309 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:01.309 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.309 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupsnap_latency": { 2024-04-25T11:38:01.309 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:01.309 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:01.309 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:01.309 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.309 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lssnap_latency": { 2024-04-25T11:38:01.309 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:01.309 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:01.309 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:01.309 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.309 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mkdir_latency": { 2024-04-25T11:38:01.309 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:01.310 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:01.310 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:01.310 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.310 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mknod_latency": { 2024-04-25T11:38:01.310 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:01.310 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:01.310 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:01.311 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.311 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mksnap_latency": { 2024-04-25T11:38:01.311 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:01.311 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:01.311 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:01.311 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.311 INFO:teuthology.orchestra.run.smithi146.stdout: "req_open_latency": { 2024-04-25T11:38:01.311 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:01.311 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:01.311 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:01.311 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.311 INFO:teuthology.orchestra.run.smithi146.stdout: "req_readdir_latency": { 2024-04-25T11:38:01.311 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:01.311 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:01.312 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:01.312 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.312 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rename_latency": { 2024-04-25T11:38:01.312 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:01.312 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:01.312 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:01.312 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.312 INFO:teuthology.orchestra.run.smithi146.stdout: "req_renamesnap_latency": { 2024-04-25T11:38:01.312 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:01.312 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:01.312 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:01.312 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.312 INFO:teuthology.orchestra.run.smithi146.stdout: "req_snapdiff_latency": { 2024-04-25T11:38:01.312 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:01.312 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:01.312 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:01.313 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.313 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmdir_latency": { 2024-04-25T11:38:01.313 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:01.313 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:01.313 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:01.313 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.313 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmsnap_latency": { 2024-04-25T11:38:01.313 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:01.313 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:01.313 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:01.313 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.313 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmxattr_latency": { 2024-04-25T11:38:01.313 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:01.313 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:01.313 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:01.314 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.314 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setattr_latency": { 2024-04-25T11:38:01.314 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:01.314 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:01.314 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:01.314 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.314 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setdirlayout_latency": { 2024-04-25T11:38:01.314 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:01.314 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:01.314 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:01.314 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.314 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setfilelock_latency": { 2024-04-25T11:38:01.314 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:01.314 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:01.315 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:01.315 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.315 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setlayout_latency": { 2024-04-25T11:38:01.315 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:01.315 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:01.315 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:01.315 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.315 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setxattr_latency": { 2024-04-25T11:38:01.315 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:01.315 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:01.315 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:01.315 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.315 INFO:teuthology.orchestra.run.smithi146.stdout: "req_symlink_latency": { 2024-04-25T11:38:01.315 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:01.315 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:01.315 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:01.316 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.316 INFO:teuthology.orchestra.run.smithi146.stdout: "req_unlink_latency": { 2024-04-25T11:38:01.316 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:01.316 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:01.316 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:01.316 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.316 INFO:teuthology.orchestra.run.smithi146.stdout: "cap_revoke_eviction": 0, 2024-04-25T11:38:01.316 INFO:teuthology.orchestra.run.smithi146.stdout: "cap_acquisition_throttle": 0, 2024-04-25T11:38:01.316 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getvxattr_latency": { 2024-04-25T11:38:01.316 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:01.316 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:01.316 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:01.316 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-04-25T11:38:01.316 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.316 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_sessions": { 2024-04-25T11:38:01.316 INFO:teuthology.orchestra.run.smithi146.stdout: "session_count": 1, 2024-04-25T11:38:01.316 INFO:teuthology.orchestra.run.smithi146.stdout: "session_add": 2, 2024-04-25T11:38:01.317 INFO:teuthology.orchestra.run.smithi146.stdout: "session_remove": 1, 2024-04-25T11:38:01.317 INFO:teuthology.orchestra.run.smithi146.stdout: "sessions_open": 1, 2024-04-25T11:38:01.317 INFO:teuthology.orchestra.run.smithi146.stdout: "sessions_stale": 0, 2024-04-25T11:38:01.317 INFO:teuthology.orchestra.run.smithi146.stdout: "total_load": 1, 2024-04-25T11:38:01.317 INFO:teuthology.orchestra.run.smithi146.stdout: "average_load": 1, 2024-04-25T11:38:01.317 INFO:teuthology.orchestra.run.smithi146.stdout: "avg_session_uptime": 712, 2024-04-25T11:38:01.317 INFO:teuthology.orchestra.run.smithi146.stdout: "mdthresh_evicted": 0 2024-04-25T11:38:01.317 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.317 INFO:teuthology.orchestra.run.smithi146.stdout: "mempool": { 2024-04-25T11:38:01.317 INFO:teuthology.orchestra.run.smithi146.stdout: "bloom_filter_bytes": 0, 2024-04-25T11:38:01.317 INFO:teuthology.orchestra.run.smithi146.stdout: "bloom_filter_items": 0, 2024-04-25T11:38:01.317 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T11:38:01.317 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_alloc_items": 0, 2024-04-25T11:38:01.317 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T11:38:01.317 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_data_items": 0, 2024-04-25T11:38:01.318 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T11:38:01.318 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T11:38:01.318 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T11:38:01.318 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T11:38:01.318 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T11:38:01.318 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_other_items": 0, 2024-04-25T11:38:01.318 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T11:38:01.318 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T11:38:01.318 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_extent_bytes": 0, 2024-04-25T11:38:01.318 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_extent_items": 0, 2024-04-25T11:38:01.318 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_blob_bytes": 0, 2024-04-25T11:38:01.318 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_blob_items": 0, 2024-04-25T11:38:01.318 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T11:38:01.318 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T11:38:01.318 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T11:38:01.319 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T11:38:01.319 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T11:38:01.319 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_items": 0, 2024-04-25T11:38:01.319 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_txc_bytes": 0, 2024-04-25T11:38:01.319 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_txc_items": 0, 2024-04-25T11:38:01.319 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T11:38:01.319 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T11:38:01.319 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_bytes": 0, 2024-04-25T11:38:01.319 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_items": 0, 2024-04-25T11:38:01.319 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_bytes": 0, 2024-04-25T11:38:01.319 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_items": 0, 2024-04-25T11:38:01.319 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T11:38:01.319 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_reader_items": 0, 2024-04-25T11:38:01.319 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T11:38:01.319 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_writer_items": 0, 2024-04-25T11:38:01.319 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_anon_bytes": 254785, 2024-04-25T11:38:01.320 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_anon_items": 88, 2024-04-25T11:38:01.320 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_meta_bytes": 0, 2024-04-25T11:38:01.320 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_meta_items": 0, 2024-04-25T11:38:01.320 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_bytes": 0, 2024-04-25T11:38:01.320 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_items": 0, 2024-04-25T11:38:01.320 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mapbl_bytes": 0, 2024-04-25T11:38:01.320 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mapbl_items": 0, 2024-04-25T11:38:01.320 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pglog_bytes": 0, 2024-04-25T11:38:01.320 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pglog_items": 0, 2024-04-25T11:38:01.320 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_bytes": 9088, 2024-04-25T11:38:01.320 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_items": 175, 2024-04-25T11:38:01.320 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T11:38:01.320 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_mapping_items": 0, 2024-04-25T11:38:01.320 INFO:teuthology.orchestra.run.smithi146.stdout: "pgmap_bytes": 0, 2024-04-25T11:38:01.320 INFO:teuthology.orchestra.run.smithi146.stdout: "pgmap_items": 0, 2024-04-25T11:38:01.320 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_co_bytes": 60032, 2024-04-25T11:38:01.321 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_co_items": 233, 2024-04-25T11:38:01.321 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_1_bytes": 0, 2024-04-25T11:38:01.321 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_1_items": 0, 2024-04-25T11:38:01.321 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_2_bytes": 0, 2024-04-25T11:38:01.321 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_2_items": 0 2024-04-25T11:38:01.321 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.321 INFO:teuthology.orchestra.run.smithi146.stdout: "objecter": { 2024-04-25T11:38:01.321 INFO:teuthology.orchestra.run.smithi146.stdout: "op_active": 0, 2024-04-25T11:38:01.321 INFO:teuthology.orchestra.run.smithi146.stdout: "op_laggy": 0, 2024-04-25T11:38:01.321 INFO:teuthology.orchestra.run.smithi146.stdout: "op_send": 41, 2024-04-25T11:38:01.321 INFO:teuthology.orchestra.run.smithi146.stdout: "op_send_bytes": 12070, 2024-04-25T11:38:01.321 INFO:teuthology.orchestra.run.smithi146.stdout: "op_resend": 33, 2024-04-25T11:38:01.321 INFO:teuthology.orchestra.run.smithi146.stdout: "op_reply": 41, 2024-04-25T11:38:01.321 INFO:teuthology.orchestra.run.smithi146.stdout: "op_latency": { 2024-04-25T11:38:01.321 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 41, 2024-04-25T11:38:01.321 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.234996473, 2024-04-25T11:38:01.322 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.005731621 2024-04-25T11:38:01.322 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.322 INFO:teuthology.orchestra.run.smithi146.stdout: "op_inflight": 0, 2024-04-25T11:38:01.322 INFO:teuthology.orchestra.run.smithi146.stdout: "oplen_avg": { 2024-04-25T11:38:01.322 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 41, 2024-04-25T11:38:01.322 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 64 2024-04-25T11:38:01.322 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.322 INFO:teuthology.orchestra.run.smithi146.stdout: "op": 41, 2024-04-25T11:38:01.322 INFO:teuthology.orchestra.run.smithi146.stdout: "op_r": 0, 2024-04-25T11:38:01.322 INFO:teuthology.orchestra.run.smithi146.stdout: "op_w": 41, 2024-04-25T11:38:01.322 INFO:teuthology.orchestra.run.smithi146.stdout: "op_rmw": 0, 2024-04-25T11:38:01.322 INFO:teuthology.orchestra.run.smithi146.stdout: "op_pg": 0, 2024-04-25T11:38:01.322 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_stat": 0, 2024-04-25T11:38:01.322 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_create": 10, 2024-04-25T11:38:01.322 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_read": 0, 2024-04-25T11:38:01.323 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_write": 4, 2024-04-25T11:38:01.323 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_writefull": 7, 2024-04-25T11:38:01.323 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_writesame": 0, 2024-04-25T11:38:01.323 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_append": 0, 2024-04-25T11:38:01.323 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_zero": 0, 2024-04-25T11:38:01.323 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_truncate": 0, 2024-04-25T11:38:01.323 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_delete": 6, 2024-04-25T11:38:01.323 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_mapext": 0, 2024-04-25T11:38:01.323 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_sparse_read": 0, 2024-04-25T11:38:01.323 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_clonerange": 0, 2024-04-25T11:38:01.323 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_getxattr": 0, 2024-04-25T11:38:01.323 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_setxattr": 20, 2024-04-25T11:38:01.323 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_cmpxattr": 0, 2024-04-25T11:38:01.323 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_rmxattr": 0, 2024-04-25T11:38:01.323 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_resetxattrs": 0, 2024-04-25T11:38:01.323 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_call": 0, 2024-04-25T11:38:01.323 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_watch": 0, 2024-04-25T11:38:01.324 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_notify": 0, 2024-04-25T11:38:01.324 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T11:38:01.324 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_pgls": 0, 2024-04-25T11:38:01.324 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_pgls_filter": 0, 2024-04-25T11:38:01.324 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_other": 0, 2024-04-25T11:38:01.324 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_active": 0, 2024-04-25T11:38:01.324 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_send": 0, 2024-04-25T11:38:01.324 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_resend": 0, 2024-04-25T11:38:01.324 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_ping": 0, 2024-04-25T11:38:01.324 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_active": 0, 2024-04-25T11:38:01.324 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_send": 0, 2024-04-25T11:38:01.324 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_resend": 0, 2024-04-25T11:38:01.324 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_active": 0, 2024-04-25T11:38:01.324 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_send": 0, 2024-04-25T11:38:01.324 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_resend": 0, 2024-04-25T11:38:01.324 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_active": 0, 2024-04-25T11:38:01.324 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_send": 0, 2024-04-25T11:38:01.325 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_resend": 0, 2024-04-25T11:38:01.325 INFO:teuthology.orchestra.run.smithi146.stdout: "command_active": 0, 2024-04-25T11:38:01.325 INFO:teuthology.orchestra.run.smithi146.stdout: "command_send": 0, 2024-04-25T11:38:01.325 INFO:teuthology.orchestra.run.smithi146.stdout: "command_resend": 0, 2024-04-25T11:38:01.325 INFO:teuthology.orchestra.run.smithi146.stdout: "map_epoch": 84, 2024-04-25T11:38:01.325 INFO:teuthology.orchestra.run.smithi146.stdout: "map_full": 0, 2024-04-25T11:38:01.325 INFO:teuthology.orchestra.run.smithi146.stdout: "map_inc": 4, 2024-04-25T11:38:01.325 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_sessions": 10, 2024-04-25T11:38:01.325 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_session_open": 10, 2024-04-25T11:38:01.325 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_session_close": 0, 2024-04-25T11:38:01.325 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_laggy": 0, 2024-04-25T11:38:01.325 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_wr": 16, 2024-04-25T11:38:01.325 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_rd": 0, 2024-04-25T11:38:01.325 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_del": 1 2024-04-25T11:38:01.325 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.325 INFO:teuthology.orchestra.run.smithi146.stdout: "oft": { 2024-04-25T11:38:01.326 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_objs": 1, 2024-04-25T11:38:01.326 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_kv_pairs": 0, 2024-04-25T11:38:01.326 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_updates": 0, 2024-04-25T11:38:01.326 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_removes": 0 2024-04-25T11:38:01.326 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.326 INFO:teuthology.orchestra.run.smithi146.stdout: "osd-slow-ops": { 2024-04-25T11:38:01.326 INFO:teuthology.orchestra.run.smithi146.stdout: "slow_ops_count": 0 2024-04-25T11:38:01.326 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.326 INFO:teuthology.orchestra.run.smithi146.stdout: "purge_queue": { 2024-04-25T11:38:01.326 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_ops": 0, 2024-04-25T11:38:01.326 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T11:38:01.326 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing": 0, 2024-04-25T11:38:01.326 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_high_water": 0, 2024-04-25T11:38:01.326 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executed_ops": 0, 2024-04-25T11:38:01.326 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executed": 0, 2024-04-25T11:38:01.326 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_item_in_journal": 0 2024-04-25T11:38:01.327 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.327 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T11:38:01.327 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-04-25T11:38:01.327 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 104857600, 2024-04-25T11:38:01.327 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-04-25T11:38:01.327 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 799, 2024-04-25T11:38:01.327 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 902580, 2024-04-25T11:38:01.327 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-04-25T11:38:01.327 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 799, 2024-04-25T11:38:01.327 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-04-25T11:38:01.327 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-04-25T11:38:01.327 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 799, 2024-04-25T11:38:01.327 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 902580, 2024-04-25T11:38:01.327 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-04-25T11:38:01.327 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:01.327 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:01.327 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:01.328 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-04-25T11:38:01.328 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.328 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-objecter_bytes": { 2024-04-25T11:38:01.328 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-04-25T11:38:01.328 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 104857600, 2024-04-25T11:38:01.328 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-04-25T11:38:01.328 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-04-25T11:38:01.328 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-04-25T11:38:01.328 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-04-25T11:38:01.328 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-04-25T11:38:01.328 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 41, 2024-04-25T11:38:01.328 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 12070, 2024-04-25T11:38:01.328 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 41, 2024-04-25T11:38:01.328 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 12070, 2024-04-25T11:38:01.328 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-04-25T11:38:01.328 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:01.329 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:01.329 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:01.329 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-04-25T11:38:01.329 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.329 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-objecter_ops": { 2024-04-25T11:38:01.329 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-04-25T11:38:01.329 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 1024, 2024-04-25T11:38:01.329 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-04-25T11:38:01.329 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-04-25T11:38:01.329 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-04-25T11:38:01.329 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-04-25T11:38:01.329 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-04-25T11:38:01.329 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 41, 2024-04-25T11:38:01.329 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 41, 2024-04-25T11:38:01.329 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 41, 2024-04-25T11:38:01.329 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 41, 2024-04-25T11:38:01.330 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-04-25T11:38:01.330 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:01.330 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:01.330 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:01.330 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-04-25T11:38:01.330 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.330 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-write_buf_throttle": { 2024-04-25T11:38:01.330 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-04-25T11:38:01.330 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 3758096384, 2024-04-25T11:38:01.330 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-04-25T11:38:01.330 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-04-25T11:38:01.330 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-04-25T11:38:01.330 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-04-25T11:38:01.330 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-04-25T11:38:01.330 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-04-25T11:38:01.330 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-04-25T11:38:01.331 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 0, 2024-04-25T11:38:01.331 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 0, 2024-04-25T11:38:01.331 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-04-25T11:38:01.331 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:01.331 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:01.331 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:01.331 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-04-25T11:38:01.331 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:01.331 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-write_buf_throttle-0x5596e7dda750": { 2024-04-25T11:38:01.331 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-04-25T11:38:01.331 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 3758096384, 2024-04-25T11:38:01.331 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-04-25T11:38:01.331 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 5, 2024-04-25T11:38:01.331 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 1613, 2024-04-25T11:38:01.331 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-04-25T11:38:01.331 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 5, 2024-04-25T11:38:01.331 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-04-25T11:38:01.332 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-04-25T11:38:01.332 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 4, 2024-04-25T11:38:01.332 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 1613, 2024-04-25T11:38:01.332 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-04-25T11:38:01.332 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:01.332 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:01.332 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:01.332 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-04-25T11:38:01.332 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-04-25T11:38:01.332 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-04-25T11:38:01.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:01 smithi069 ceph-mon[29188]: pgmap v718: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-25T11:38:01.724 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:01 smithi152 ceph-mon[32701]: pgmap v718: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-25T11:38:02.106 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-25T11:38:02.106 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-25T11:38:02.107 DEBUG:tasks.check_counter:Getting stats from k 2024-04-25T11:38:02.107 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-25T11:38:02.343 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T11:38:03.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:03 smithi146 ceph-mon[32821]: pgmap v719: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:38:03.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:03 smithi069 ceph-mon[29188]: pgmap v719: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:38:03.724 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:03 smithi152 ceph-mon[32701]: pgmap v719: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:38:03.879 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-04-25T11:38:03.879 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T11:38:03.879 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 259, 2024-04-25T11:38:03.879 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 250, 2024-04-25T11:38:03.879 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 852962, 2024-04-25T11:38:03.879 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 149764, 2024-04-25T11:38:03.879 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 5, 2024-04-25T11:38:03.879 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 1, 2024-04-25T11:38:03.879 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.045994485, 2024-04-25T11:38:03.879 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.015384586, 2024-04-25T11:38:03.879 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.025572210, 2024-04-25T11:38:03.880 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.006853056, 2024-04-25T11:38:03.880 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:38:03.880 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 249, 2024-04-25T11:38:03.880 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.013543685, 2024-04-25T11:38:03.880 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000054392 2024-04-25T11:38:03.880 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.880 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:38:03.880 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:03.880 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:03.880 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:03.880 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.880 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 852128, 2024-04-25T11:38:03.880 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 149440 2024-04-25T11:38:03.881 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.881 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T11:38:03.881 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 725, 2024-04-25T11:38:03.881 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 905, 2024-04-25T11:38:03.881 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 180262, 2024-04-25T11:38:03.881 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 619189, 2024-04-25T11:38:03.881 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 5, 2024-04-25T11:38:03.881 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 4, 2024-04-25T11:38:03.881 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.084291038, 2024-04-25T11:38:03.881 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.031378150, 2024-04-25T11:38:03.881 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.017364156, 2024-04-25T11:38:03.881 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.026851845, 2024-04-25T11:38:03.881 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:38:03.882 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 904, 2024-04-25T11:38:03.882 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.044161356, 2024-04-25T11:38:03.882 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000048851 2024-04-25T11:38:03.882 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.882 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:38:03.882 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:03.882 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:03.882 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:03.882 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.882 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 3120, 2024-04-25T11:38:03.882 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 390224 2024-04-25T11:38:03.882 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.882 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T11:38:03.883 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 1676, 2024-04-25T11:38:03.883 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 1678, 2024-04-25T11:38:03.883 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 502518, 2024-04-25T11:38:03.883 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 533026, 2024-04-25T11:38:03.883 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 4, 2024-04-25T11:38:03.883 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 3, 2024-04-25T11:38:03.883 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.188992877, 2024-04-25T11:38:03.883 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.067214849, 2024-04-25T11:38:03.883 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.047246832, 2024-04-25T11:38:03.883 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.057921885, 2024-04-25T11:38:03.883 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:38:03.883 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 1676, 2024-04-25T11:38:03.883 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.073668232, 2024-04-25T11:38:03.883 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000043954 2024-04-25T11:38:03.884 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.884 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:38:03.884 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:03.884 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:03.884 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:03.884 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.884 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T11:38:03.884 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T11:38:03.884 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.884 INFO:teuthology.orchestra.run.smithi146.stdout: "cct": { 2024-04-25T11:38:03.884 INFO:teuthology.orchestra.run.smithi146.stdout: "total_workers": 1, 2024-04-25T11:38:03.884 INFO:teuthology.orchestra.run.smithi146.stdout: "unhealthy_workers": 0 2024-04-25T11:38:03.884 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.885 INFO:teuthology.orchestra.run.smithi146.stdout: "finisher-MDSRank": { 2024-04-25T11:38:03.885 INFO:teuthology.orchestra.run.smithi146.stdout: "queue_len": 0, 2024-04-25T11:38:03.885 INFO:teuthology.orchestra.run.smithi146.stdout: "complete_latency": { 2024-04-25T11:38:03.885 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 4019, 2024-04-25T11:38:03.885 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.132404238, 2024-04-25T11:38:03.885 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000032944 2024-04-25T11:38:03.885 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-04-25T11:38:03.885 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.885 INFO:teuthology.orchestra.run.smithi146.stdout: "finisher-PurgeQueue": { 2024-04-25T11:38:03.885 INFO:teuthology.orchestra.run.smithi146.stdout: "queue_len": 0, 2024-04-25T11:38:03.885 INFO:teuthology.orchestra.run.smithi146.stdout: "complete_latency": { 2024-04-25T11:38:03.885 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:03.885 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:03.885 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:03.886 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-04-25T11:38:03.886 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.886 INFO:teuthology.orchestra.run.smithi146.stdout: "mds": { 2024-04-25T11:38:03.886 INFO:teuthology.orchestra.run.smithi146.stdout: "request": 0, 2024-04-25T11:38:03.886 INFO:teuthology.orchestra.run.smithi146.stdout: "reply": 0, 2024-04-25T11:38:03.886 INFO:teuthology.orchestra.run.smithi146.stdout: "reply_latency": { 2024-04-25T11:38:03.886 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:03.886 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:03.886 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:03.886 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.886 INFO:teuthology.orchestra.run.smithi146.stdout: "slow_reply": 0, 2024-04-25T11:38:03.886 INFO:teuthology.orchestra.run.smithi146.stdout: "forward": 0, 2024-04-25T11:38:03.886 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_fetch_complete": 0, 2024-04-25T11:38:03.887 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_fetch_keys": 0, 2024-04-25T11:38:03.887 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_commit": 0, 2024-04-25T11:38:03.887 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_split": 0, 2024-04-25T11:38:03.887 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_merge": 0, 2024-04-25T11:38:03.887 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes": 0, 2024-04-25T11:38:03.887 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_top": 0, 2024-04-25T11:38:03.887 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_bottom": 0, 2024-04-25T11:38:03.887 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_pin_tail": 0, 2024-04-25T11:38:03.887 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_pinned": 0, 2024-04-25T11:38:03.887 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_expired": 0, 2024-04-25T11:38:03.887 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_with_caps": 0, 2024-04-25T11:38:03.887 INFO:teuthology.orchestra.run.smithi146.stdout: "caps": 0, 2024-04-25T11:38:03.887 INFO:teuthology.orchestra.run.smithi146.stdout: "subtrees": 1, 2024-04-25T11:38:03.888 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse": 0, 2024-04-25T11:38:03.888 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_hit": 0, 2024-04-25T11:38:03.888 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_forward": 0, 2024-04-25T11:38:03.888 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_discover": 0, 2024-04-25T11:38:03.888 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_dir_fetch": 0, 2024-04-25T11:38:03.888 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_remote_ino": 0, 2024-04-25T11:38:03.888 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_lock": 0, 2024-04-25T11:38:03.888 INFO:teuthology.orchestra.run.smithi146.stdout: "load_cent": 0, 2024-04-25T11:38:03.888 INFO:teuthology.orchestra.run.smithi146.stdout: "q": 0, 2024-04-25T11:38:03.888 INFO:teuthology.orchestra.run.smithi146.stdout: "exported": 0, 2024-04-25T11:38:03.888 INFO:teuthology.orchestra.run.smithi146.stdout: "exported_inodes": 0, 2024-04-25T11:38:03.888 INFO:teuthology.orchestra.run.smithi146.stdout: "imported": 0, 2024-04-25T11:38:03.888 INFO:teuthology.orchestra.run.smithi146.stdout: "imported_inodes": 0, 2024-04-25T11:38:03.888 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_dir_fetch": 0, 2024-04-25T11:38:03.889 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_backtrace_fetch": 0, 2024-04-25T11:38:03.889 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_peer_discover": 0, 2024-04-25T11:38:03.889 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rfiles": 0, 2024-04-25T11:38:03.889 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rbytes": 0, 2024-04-25T11:38:03.889 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rsnaps": 0, 2024-04-25T11:38:03.889 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T11:38:03.889 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_set_tag": 0, 2024-04-25T11:38:03.889 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T11:38:03.889 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_inotable_repaired": 0, 2024-04-25T11:38:03.889 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dir_inodes": 0, 2024-04-25T11:38:03.889 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T11:38:03.889 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T11:38:03.889 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_file_inodes": 0, 2024-04-25T11:38:03.889 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_inode_file_caps": 0, 2024-04-25T11:38:03.890 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T11:38:03.890 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_grant": 0, 2024-04-25T11:38:03.890 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T11:38:03.890 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T11:38:03.890 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T11:38:03.890 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_caps": 0, 2024-04-25T11:38:03.890 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_caps_dirty": 0, 2024-04-25T11:38:03.890 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_cap_release": 0, 2024-04-25T11:38:03.890 INFO:teuthology.orchestra.run.smithi146.stdout: "process_request_cap_release": 0 2024-04-25T11:38:03.890 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.890 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_cache": { 2024-04-25T11:38:03.890 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_update": 0, 2024-04-25T11:38:03.890 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_update_receipt": 0, 2024-04-25T11:38:03.891 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_try_discover": 0, 2024-04-25T11:38:03.891 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_send_discover": 0, 2024-04-25T11:38:03.891 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_handle_discover": 0, 2024-04-25T11:38:03.891 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays": 0, 2024-04-25T11:38:03.891 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays_delayed": 0, 2024-04-25T11:38:03.891 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays_enqueuing": 0, 2024-04-25T11:38:03.891 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_created": 0, 2024-04-25T11:38:03.891 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_enqueued": 0, 2024-04-25T11:38:03.891 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_reintegrated": 0, 2024-04-25T11:38:03.891 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_migrated": 0, 2024-04-25T11:38:03.891 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_processing": 0, 2024-04-25T11:38:03.891 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_enqueued": 0, 2024-04-25T11:38:03.891 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_prioritized": 0, 2024-04-25T11:38:03.891 INFO:teuthology.orchestra.run.smithi146.stdout: "recovery_started": 0, 2024-04-25T11:38:03.891 INFO:teuthology.orchestra.run.smithi146.stdout: "recovery_completed": 0, 2024-04-25T11:38:03.892 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_quiesce_path": 0, 2024-04-25T11:38:03.892 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_quiesce_inode": 0, 2024-04-25T11:38:03.892 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T11:38:03.892 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_exportdir": 0, 2024-04-25T11:38:03.892 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_flush": 0, 2024-04-25T11:38:03.892 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_fragmentdir": 0, 2024-04-25T11:38:03.892 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_fragstats": 0, 2024-04-25T11:38:03.892 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_inodestats": 0 2024-04-25T11:38:03.892 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.892 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_log": { 2024-04-25T11:38:03.892 INFO:teuthology.orchestra.run.smithi146.stdout: "evlrg": 0, 2024-04-25T11:38:03.892 INFO:teuthology.orchestra.run.smithi146.stdout: "evadd": 0, 2024-04-25T11:38:03.892 INFO:teuthology.orchestra.run.smithi146.stdout: "evex": 0, 2024-04-25T11:38:03.892 INFO:teuthology.orchestra.run.smithi146.stdout: "evtrm": 0, 2024-04-25T11:38:03.893 INFO:teuthology.orchestra.run.smithi146.stdout: "ev": 5, 2024-04-25T11:38:03.893 INFO:teuthology.orchestra.run.smithi146.stdout: "evexg": 0, 2024-04-25T11:38:03.893 INFO:teuthology.orchestra.run.smithi146.stdout: "evexd": 0, 2024-04-25T11:38:03.893 INFO:teuthology.orchestra.run.smithi146.stdout: "segadd": 0, 2024-04-25T11:38:03.893 INFO:teuthology.orchestra.run.smithi146.stdout: "segex": 0, 2024-04-25T11:38:03.893 INFO:teuthology.orchestra.run.smithi146.stdout: "segtrm": 0, 2024-04-25T11:38:03.893 INFO:teuthology.orchestra.run.smithi146.stdout: "seg": 2, 2024-04-25T11:38:03.893 INFO:teuthology.orchestra.run.smithi146.stdout: "segmjr": 2, 2024-04-25T11:38:03.893 INFO:teuthology.orchestra.run.smithi146.stdout: "segexg": 0, 2024-04-25T11:38:03.893 INFO:teuthology.orchestra.run.smithi146.stdout: "segexd": 0, 2024-04-25T11:38:03.893 INFO:teuthology.orchestra.run.smithi146.stdout: "expos": 4194304, 2024-04-25T11:38:03.893 INFO:teuthology.orchestra.run.smithi146.stdout: "wrpos": 4195917, 2024-04-25T11:38:03.893 INFO:teuthology.orchestra.run.smithi146.stdout: "rdpos": 4195776, 2024-04-25T11:38:03.893 INFO:teuthology.orchestra.run.smithi146.stdout: "jlat": { 2024-04-25T11:38:03.894 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:03.894 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:03.894 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:03.894 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.894 INFO:teuthology.orchestra.run.smithi146.stdout: "replayed": 5 2024-04-25T11:38:03.894 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.894 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_mem": { 2024-04-25T11:38:03.894 INFO:teuthology.orchestra.run.smithi146.stdout: "ino": 3, 2024-04-25T11:38:03.894 INFO:teuthology.orchestra.run.smithi146.stdout: "ino+": 3, 2024-04-25T11:38:03.894 INFO:teuthology.orchestra.run.smithi146.stdout: "ino-": 0, 2024-04-25T11:38:03.894 INFO:teuthology.orchestra.run.smithi146.stdout: "dir": 1, 2024-04-25T11:38:03.894 INFO:teuthology.orchestra.run.smithi146.stdout: "dir+": 1, 2024-04-25T11:38:03.894 INFO:teuthology.orchestra.run.smithi146.stdout: "dir-": 0, 2024-04-25T11:38:03.894 INFO:teuthology.orchestra.run.smithi146.stdout: "dn": 0, 2024-04-25T11:38:03.895 INFO:teuthology.orchestra.run.smithi146.stdout: "dn+": 0, 2024-04-25T11:38:03.895 INFO:teuthology.orchestra.run.smithi146.stdout: "dn-": 0, 2024-04-25T11:38:03.895 INFO:teuthology.orchestra.run.smithi146.stdout: "cap": 0, 2024-04-25T11:38:03.895 INFO:teuthology.orchestra.run.smithi146.stdout: "cap+": 0, 2024-04-25T11:38:03.895 INFO:teuthology.orchestra.run.smithi146.stdout: "cap-": 0, 2024-04-25T11:38:03.895 INFO:teuthology.orchestra.run.smithi146.stdout: "rss": 118184, 2024-04-25T11:38:03.895 INFO:teuthology.orchestra.run.smithi146.stdout: "heap": 215440 2024-04-25T11:38:03.895 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.895 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_server": { 2024-04-25T11:38:03.895 INFO:teuthology.orchestra.run.smithi146.stdout: "dispatch_client_request": 0, 2024-04-25T11:38:03.895 INFO:teuthology.orchestra.run.smithi146.stdout: "dispatch_server_request": 0, 2024-04-25T11:38:03.895 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_request": 0, 2024-04-25T11:38:03.895 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_session": 0, 2024-04-25T11:38:03.895 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_peer_request": 0, 2024-04-25T11:38:03.896 INFO:teuthology.orchestra.run.smithi146.stdout: "req_create_latency": { 2024-04-25T11:38:03.896 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:03.896 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:03.896 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:03.896 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.896 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getattr_latency": { 2024-04-25T11:38:03.896 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:03.896 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:03.896 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:03.896 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.896 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getfilelock_latency": { 2024-04-25T11:38:03.896 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:03.896 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:03.896 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:03.896 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.897 INFO:teuthology.orchestra.run.smithi146.stdout: "req_link_latency": { 2024-04-25T11:38:03.897 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:03.897 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:03.897 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:03.897 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.897 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookup_latency": { 2024-04-25T11:38:03.897 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:03.897 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:03.897 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:03.897 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.897 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookuphash_latency": { 2024-04-25T11:38:03.897 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:03.897 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:03.897 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:03.897 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.898 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupino_latency": { 2024-04-25T11:38:03.898 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:03.898 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:03.898 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:03.898 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.898 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupname_latency": { 2024-04-25T11:38:03.898 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:03.898 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:03.898 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:03.898 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.898 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupparent_latency": { 2024-04-25T11:38:03.898 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:03.898 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:03.899 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:03.899 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.899 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupsnap_latency": { 2024-04-25T11:38:03.899 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:03.899 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:03.899 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:03.899 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.899 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lssnap_latency": { 2024-04-25T11:38:03.899 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:03.899 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:03.899 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:03.899 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.899 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mkdir_latency": { 2024-04-25T11:38:03.899 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:03.900 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:03.900 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:03.900 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.900 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mknod_latency": { 2024-04-25T11:38:03.901 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:03.901 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:03.901 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:03.901 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.901 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mksnap_latency": { 2024-04-25T11:38:03.901 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:03.901 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:03.901 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:03.901 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.901 INFO:teuthology.orchestra.run.smithi146.stdout: "req_open_latency": { 2024-04-25T11:38:03.901 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:03.901 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:03.901 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:03.901 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.901 INFO:teuthology.orchestra.run.smithi146.stdout: "req_readdir_latency": { 2024-04-25T11:38:03.902 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:03.902 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:03.902 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:03.902 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.902 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rename_latency": { 2024-04-25T11:38:03.902 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:03.902 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:03.902 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:03.902 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.902 INFO:teuthology.orchestra.run.smithi146.stdout: "req_renamesnap_latency": { 2024-04-25T11:38:03.902 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:03.902 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:03.902 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:03.903 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.903 INFO:teuthology.orchestra.run.smithi146.stdout: "req_snapdiff_latency": { 2024-04-25T11:38:03.903 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:03.903 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:03.903 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:03.903 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.903 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmdir_latency": { 2024-04-25T11:38:03.903 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:03.903 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:03.903 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:03.903 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.903 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmsnap_latency": { 2024-04-25T11:38:03.903 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:03.903 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:03.903 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:03.904 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.904 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmxattr_latency": { 2024-04-25T11:38:03.904 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:03.904 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:03.904 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:03.904 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.904 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setattr_latency": { 2024-04-25T11:38:03.904 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:03.904 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:03.904 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:03.904 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.904 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setdirlayout_latency": { 2024-04-25T11:38:03.904 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:03.904 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:03.904 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:03.905 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.905 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setfilelock_latency": { 2024-04-25T11:38:03.905 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:03.905 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:03.905 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:03.905 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.905 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setlayout_latency": { 2024-04-25T11:38:03.905 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:03.905 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:03.905 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:03.905 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.905 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setxattr_latency": { 2024-04-25T11:38:03.905 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:03.905 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:03.906 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:03.906 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.906 INFO:teuthology.orchestra.run.smithi146.stdout: "req_symlink_latency": { 2024-04-25T11:38:03.906 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:03.906 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:03.906 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:03.906 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.906 INFO:teuthology.orchestra.run.smithi146.stdout: "req_unlink_latency": { 2024-04-25T11:38:03.906 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:03.906 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:03.906 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:03.906 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.906 INFO:teuthology.orchestra.run.smithi146.stdout: "cap_revoke_eviction": 0, 2024-04-25T11:38:03.906 INFO:teuthology.orchestra.run.smithi146.stdout: "cap_acquisition_throttle": 0, 2024-04-25T11:38:03.907 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getvxattr_latency": { 2024-04-25T11:38:03.907 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:03.907 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:03.907 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:03.907 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-04-25T11:38:03.907 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.907 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_sessions": { 2024-04-25T11:38:03.907 INFO:teuthology.orchestra.run.smithi146.stdout: "session_count": 1, 2024-04-25T11:38:03.907 INFO:teuthology.orchestra.run.smithi146.stdout: "session_add": 2, 2024-04-25T11:38:03.907 INFO:teuthology.orchestra.run.smithi146.stdout: "session_remove": 1, 2024-04-25T11:38:03.907 INFO:teuthology.orchestra.run.smithi146.stdout: "sessions_open": 2, 2024-04-25T11:38:03.907 INFO:teuthology.orchestra.run.smithi146.stdout: "sessions_stale": 0, 2024-04-25T11:38:03.907 INFO:teuthology.orchestra.run.smithi146.stdout: "total_load": 0, 2024-04-25T11:38:03.907 INFO:teuthology.orchestra.run.smithi146.stdout: "average_load": 0, 2024-04-25T11:38:03.907 INFO:teuthology.orchestra.run.smithi146.stdout: "avg_session_uptime": 2967, 2024-04-25T11:38:03.908 INFO:teuthology.orchestra.run.smithi146.stdout: "mdthresh_evicted": 0 2024-04-25T11:38:03.908 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.908 INFO:teuthology.orchestra.run.smithi146.stdout: "mempool": { 2024-04-25T11:38:03.908 INFO:teuthology.orchestra.run.smithi146.stdout: "bloom_filter_bytes": 0, 2024-04-25T11:38:03.908 INFO:teuthology.orchestra.run.smithi146.stdout: "bloom_filter_items": 0, 2024-04-25T11:38:03.908 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T11:38:03.908 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_alloc_items": 0, 2024-04-25T11:38:03.908 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T11:38:03.908 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_data_items": 0, 2024-04-25T11:38:03.908 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T11:38:03.908 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T11:38:03.908 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T11:38:03.908 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T11:38:03.908 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T11:38:03.908 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_other_items": 0, 2024-04-25T11:38:03.909 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T11:38:03.909 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T11:38:03.909 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_extent_bytes": 0, 2024-04-25T11:38:03.909 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_extent_items": 0, 2024-04-25T11:38:03.909 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_blob_bytes": 0, 2024-04-25T11:38:03.909 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_blob_items": 0, 2024-04-25T11:38:03.909 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T11:38:03.909 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T11:38:03.909 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T11:38:03.909 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T11:38:03.909 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T11:38:03.909 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_items": 0, 2024-04-25T11:38:03.909 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_txc_bytes": 0, 2024-04-25T11:38:03.909 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_txc_items": 0, 2024-04-25T11:38:03.909 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T11:38:03.910 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T11:38:03.910 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_bytes": 0, 2024-04-25T11:38:03.910 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_items": 0, 2024-04-25T11:38:03.910 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_bytes": 0, 2024-04-25T11:38:03.910 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_items": 0, 2024-04-25T11:38:03.910 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T11:38:03.910 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_reader_items": 0, 2024-04-25T11:38:03.910 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T11:38:03.910 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_writer_items": 0, 2024-04-25T11:38:03.910 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_anon_bytes": 189400, 2024-04-25T11:38:03.910 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_anon_items": 51, 2024-04-25T11:38:03.910 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_meta_bytes": 0, 2024-04-25T11:38:03.910 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_meta_items": 0, 2024-04-25T11:38:03.911 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_bytes": 0, 2024-04-25T11:38:03.911 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_items": 0, 2024-04-25T11:38:03.911 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mapbl_bytes": 0, 2024-04-25T11:38:03.911 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mapbl_items": 0, 2024-04-25T11:38:03.911 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pglog_bytes": 0, 2024-04-25T11:38:03.911 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pglog_items": 0, 2024-04-25T11:38:03.911 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_bytes": 9088, 2024-04-25T11:38:03.911 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_items": 175, 2024-04-25T11:38:03.911 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T11:38:03.911 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_mapping_items": 0, 2024-04-25T11:38:03.911 INFO:teuthology.orchestra.run.smithi146.stdout: "pgmap_bytes": 0, 2024-04-25T11:38:03.911 INFO:teuthology.orchestra.run.smithi146.stdout: "pgmap_items": 0, 2024-04-25T11:38:03.911 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_co_bytes": 8512, 2024-04-25T11:38:03.911 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_co_items": 13, 2024-04-25T11:38:03.912 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_1_bytes": 0, 2024-04-25T11:38:03.912 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_1_items": 0, 2024-04-25T11:38:03.912 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_2_bytes": 0, 2024-04-25T11:38:03.912 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_2_items": 0 2024-04-25T11:38:03.912 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.912 INFO:teuthology.orchestra.run.smithi146.stdout: "objecter": { 2024-04-25T11:38:03.912 INFO:teuthology.orchestra.run.smithi146.stdout: "op_active": 0, 2024-04-25T11:38:03.912 INFO:teuthology.orchestra.run.smithi146.stdout: "op_laggy": 0, 2024-04-25T11:38:03.912 INFO:teuthology.orchestra.run.smithi146.stdout: "op_send": 2397, 2024-04-25T11:38:03.912 INFO:teuthology.orchestra.run.smithi146.stdout: "op_send_bytes": 21, 2024-04-25T11:38:03.912 INFO:teuthology.orchestra.run.smithi146.stdout: "op_resend": 3, 2024-04-25T11:38:03.912 INFO:teuthology.orchestra.run.smithi146.stdout: "op_reply": 2397, 2024-04-25T11:38:03.912 INFO:teuthology.orchestra.run.smithi146.stdout: "op_latency": { 2024-04-25T11:38:03.912 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 2397, 2024-04-25T11:38:03.913 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 1.222981076, 2024-04-25T11:38:03.913 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000510213 2024-04-25T11:38:03.913 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.913 INFO:teuthology.orchestra.run.smithi146.stdout: "op_inflight": 0, 2024-04-25T11:38:03.913 INFO:teuthology.orchestra.run.smithi146.stdout: "oplen_avg": { 2024-04-25T11:38:03.913 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 2397, 2024-04-25T11:38:03.913 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 2398 2024-04-25T11:38:03.913 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.913 INFO:teuthology.orchestra.run.smithi146.stdout: "op": 2397, 2024-04-25T11:38:03.913 INFO:teuthology.orchestra.run.smithi146.stdout: "op_r": 2397, 2024-04-25T11:38:03.913 INFO:teuthology.orchestra.run.smithi146.stdout: "op_w": 0, 2024-04-25T11:38:03.913 INFO:teuthology.orchestra.run.smithi146.stdout: "op_rmw": 0, 2024-04-25T11:38:03.913 INFO:teuthology.orchestra.run.smithi146.stdout: "op_pg": 0, 2024-04-25T11:38:03.913 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_stat": 1553, 2024-04-25T11:38:03.914 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_create": 0, 2024-04-25T11:38:03.914 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_read": 842, 2024-04-25T11:38:03.914 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_write": 0, 2024-04-25T11:38:03.914 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_writefull": 0, 2024-04-25T11:38:03.914 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_writesame": 0, 2024-04-25T11:38:03.914 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_append": 0, 2024-04-25T11:38:03.914 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_zero": 0, 2024-04-25T11:38:03.914 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_truncate": 0, 2024-04-25T11:38:03.914 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_delete": 0, 2024-04-25T11:38:03.914 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_mapext": 0, 2024-04-25T11:38:03.914 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_sparse_read": 0, 2024-04-25T11:38:03.914 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_clonerange": 0, 2024-04-25T11:38:03.914 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_getxattr": 1, 2024-04-25T11:38:03.915 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_setxattr": 0, 2024-04-25T11:38:03.915 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_cmpxattr": 0, 2024-04-25T11:38:03.915 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_rmxattr": 0, 2024-04-25T11:38:03.915 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_resetxattrs": 0, 2024-04-25T11:38:03.915 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_call": 0, 2024-04-25T11:38:03.915 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_watch": 0, 2024-04-25T11:38:03.915 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_notify": 0, 2024-04-25T11:38:03.915 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T11:38:03.915 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_pgls": 0, 2024-04-25T11:38:03.915 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_pgls_filter": 0, 2024-04-25T11:38:03.915 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_other": 0, 2024-04-25T11:38:03.915 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_active": 0, 2024-04-25T11:38:03.915 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_send": 0, 2024-04-25T11:38:03.915 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_resend": 0, 2024-04-25T11:38:03.916 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_ping": 0, 2024-04-25T11:38:03.916 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_active": 0, 2024-04-25T11:38:03.916 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_send": 0, 2024-04-25T11:38:03.916 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_resend": 0, 2024-04-25T11:38:03.916 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_active": 0, 2024-04-25T11:38:03.916 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_send": 0, 2024-04-25T11:38:03.916 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_resend": 0, 2024-04-25T11:38:03.916 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_active": 0, 2024-04-25T11:38:03.916 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_send": 0, 2024-04-25T11:38:03.916 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_resend": 0, 2024-04-25T11:38:03.916 INFO:teuthology.orchestra.run.smithi146.stdout: "command_active": 0, 2024-04-25T11:38:03.916 INFO:teuthology.orchestra.run.smithi146.stdout: "command_send": 0, 2024-04-25T11:38:03.916 INFO:teuthology.orchestra.run.smithi146.stdout: "command_resend": 0, 2024-04-25T11:38:03.916 INFO:teuthology.orchestra.run.smithi146.stdout: "map_epoch": 84, 2024-04-25T11:38:03.916 INFO:teuthology.orchestra.run.smithi146.stdout: "map_full": 0, 2024-04-25T11:38:03.917 INFO:teuthology.orchestra.run.smithi146.stdout: "map_inc": 4, 2024-04-25T11:38:03.917 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_sessions": 5, 2024-04-25T11:38:03.917 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_session_open": 5, 2024-04-25T11:38:03.917 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_session_close": 0, 2024-04-25T11:38:03.917 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_laggy": 0, 2024-04-25T11:38:03.917 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_wr": 0, 2024-04-25T11:38:03.917 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_rd": 2, 2024-04-25T11:38:03.917 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_del": 0 2024-04-25T11:38:03.917 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.917 INFO:teuthology.orchestra.run.smithi146.stdout: "oft": { 2024-04-25T11:38:03.917 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_objs": 0, 2024-04-25T11:38:03.917 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_kv_pairs": 0, 2024-04-25T11:38:03.917 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_updates": 0, 2024-04-25T11:38:03.917 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_removes": 0 2024-04-25T11:38:03.917 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.918 INFO:teuthology.orchestra.run.smithi146.stdout: "osd-slow-ops": { 2024-04-25T11:38:03.918 INFO:teuthology.orchestra.run.smithi146.stdout: "slow_ops_count": 0 2024-04-25T11:38:03.918 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.918 INFO:teuthology.orchestra.run.smithi146.stdout: "purge_queue": { 2024-04-25T11:38:03.918 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_ops": 0, 2024-04-25T11:38:03.918 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T11:38:03.918 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing": 0, 2024-04-25T11:38:03.918 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_high_water": 0, 2024-04-25T11:38:03.918 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executed_ops": 0, 2024-04-25T11:38:03.918 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executed": 0, 2024-04-25T11:38:03.918 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_item_in_journal": 0 2024-04-25T11:38:03.918 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.918 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T11:38:03.918 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-04-25T11:38:03.919 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 104857600, 2024-04-25T11:38:03.919 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-04-25T11:38:03.919 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 2656, 2024-04-25T11:38:03.919 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 1280512, 2024-04-25T11:38:03.919 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-04-25T11:38:03.919 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 2656, 2024-04-25T11:38:03.919 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-04-25T11:38:03.919 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-04-25T11:38:03.919 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 2656, 2024-04-25T11:38:03.919 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 1280512, 2024-04-25T11:38:03.919 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-04-25T11:38:03.919 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:03.919 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:03.919 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:03.919 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-04-25T11:38:03.920 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.920 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-objecter_bytes": { 2024-04-25T11:38:03.920 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-04-25T11:38:03.920 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 104857600, 2024-04-25T11:38:03.920 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-04-25T11:38:03.920 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-04-25T11:38:03.920 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-04-25T11:38:03.920 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-04-25T11:38:03.920 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-04-25T11:38:03.920 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 2397, 2024-04-25T11:38:03.920 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 1618, 2024-04-25T11:38:03.920 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 2397, 2024-04-25T11:38:03.920 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 1618, 2024-04-25T11:38:03.920 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-04-25T11:38:03.920 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:03.921 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:03.921 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:03.921 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-04-25T11:38:03.921 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.921 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-objecter_ops": { 2024-04-25T11:38:03.921 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-04-25T11:38:03.921 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 1024, 2024-04-25T11:38:03.921 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-04-25T11:38:03.921 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-04-25T11:38:03.921 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-04-25T11:38:03.921 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-04-25T11:38:03.921 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-04-25T11:38:03.921 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 2397, 2024-04-25T11:38:03.921 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 2397, 2024-04-25T11:38:03.922 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 2397, 2024-04-25T11:38:03.922 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 2397, 2024-04-25T11:38:03.922 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-04-25T11:38:03.922 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:03.922 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:03.922 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:03.922 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-04-25T11:38:03.922 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.922 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-write_buf_throttle": { 2024-04-25T11:38:03.922 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-04-25T11:38:03.922 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 3758096384, 2024-04-25T11:38:03.922 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-04-25T11:38:03.922 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-04-25T11:38:03.923 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-04-25T11:38:03.923 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-04-25T11:38:03.923 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-04-25T11:38:03.923 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-04-25T11:38:03.923 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-04-25T11:38:03.923 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 0, 2024-04-25T11:38:03.923 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 0, 2024-04-25T11:38:03.923 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-04-25T11:38:03.923 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:03.923 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:03.923 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:03.923 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-04-25T11:38:03.923 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-04-25T11:38:03.923 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-write_buf_throttle-0x561af6aca8f0": { 2024-04-25T11:38:03.924 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-04-25T11:38:03.924 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 3758096384, 2024-04-25T11:38:03.924 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-04-25T11:38:03.924 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-04-25T11:38:03.924 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-04-25T11:38:03.924 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-04-25T11:38:03.924 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-04-25T11:38:03.924 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-04-25T11:38:03.924 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-04-25T11:38:03.924 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 0, 2024-04-25T11:38:03.924 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 0, 2024-04-25T11:38:03.924 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-04-25T11:38:03.924 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-04-25T11:38:03.924 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-04-25T11:38:03.924 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-04-25T11:38:03.925 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-04-25T11:38:03.925 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-04-25T11:38:03.925 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-04-25T11:38:04.693 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-04-25T11:38:04.693 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-04-25T11:38:04.693 DEBUG:tasks.check_counter:Getting stats from c 2024-04-25T11:38:04.693 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-25T11:38:04.945 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T11:38:05.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:05 smithi152 ceph-mon[32701]: pgmap v720: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T11:38:05.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:05 smithi146 ceph-mon[32821]: pgmap v720: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T11:38:05.686 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:05 smithi069 ceph-mon[29188]: pgmap v720: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T11:38:06.893 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-25T11:38:06.894 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T11:38:06.894 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 2060, 2024-04-25T11:38:06.894 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 2049, 2024-04-25T11:38:06.894 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 5576748, 2024-04-25T11:38:06.894 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 720532, 2024-04-25T11:38:06.894 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 5, 2024-04-25T11:38:06.894 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 1, 2024-04-25T11:38:06.894 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.239828215, 2024-04-25T11:38:06.894 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.075420992, 2024-04-25T11:38:06.894 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.083190734, 2024-04-25T11:38:06.895 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.066415852, 2024-04-25T11:38:06.895 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:38:06.895 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 2049, 2024-04-25T11:38:06.895 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.088563920, 2024-04-25T11:38:06.895 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000043222 2024-04-25T11:38:06.895 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.895 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:38:06.895 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:06.895 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:06.895 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:06.895 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.895 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 854432, 2024-04-25T11:38:06.895 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 148128 2024-04-25T11:38:06.895 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.896 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T11:38:06.896 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 261, 2024-04-25T11:38:06.896 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 441, 2024-04-25T11:38:06.896 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 64792, 2024-04-25T11:38:06.896 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 473786, 2024-04-25T11:38:06.896 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 5, 2024-04-25T11:38:06.896 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 4, 2024-04-25T11:38:06.896 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.039193547, 2024-04-25T11:38:06.896 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.017689019, 2024-04-25T11:38:06.896 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.007742407, 2024-04-25T11:38:06.896 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.009533100, 2024-04-25T11:38:06.896 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:38:06.896 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 439, 2024-04-25T11:38:06.897 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.033283351, 2024-04-25T11:38:06.897 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000075816 2024-04-25T11:38:06.897 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.897 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:38:06.897 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:06.897 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:06.897 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:06.897 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.897 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T11:38:06.897 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 391040 2024-04-25T11:38:06.897 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.897 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T11:38:06.897 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 1078, 2024-04-25T11:38:06.897 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 1079, 2024-04-25T11:38:06.898 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 2893524, 2024-04-25T11:38:06.898 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 342493, 2024-04-25T11:38:06.898 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 5, 2024-04-25T11:38:06.898 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 4, 2024-04-25T11:38:06.898 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.107606689, 2024-04-25T11:38:06.898 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.039536673, 2024-04-25T11:38:06.898 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.026282013, 2024-04-25T11:38:06.898 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.032497436, 2024-04-25T11:38:06.898 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:38:06.898 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 1077, 2024-04-25T11:38:06.898 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.043672180, 2024-04-25T11:38:06.898 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000040549 2024-04-25T11:38:06.898 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.898 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:38:06.899 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:06.899 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:06.899 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:06.899 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.899 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T11:38:06.899 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T11:38:06.899 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.899 INFO:teuthology.orchestra.run.smithi152.stdout: "cct": { 2024-04-25T11:38:06.899 INFO:teuthology.orchestra.run.smithi152.stdout: "total_workers": 1, 2024-04-25T11:38:06.899 INFO:teuthology.orchestra.run.smithi152.stdout: "unhealthy_workers": 0 2024-04-25T11:38:06.899 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.899 INFO:teuthology.orchestra.run.smithi152.stdout: "finisher-MDSRank": { 2024-04-25T11:38:06.899 INFO:teuthology.orchestra.run.smithi152.stdout: "queue_len": 0, 2024-04-25T11:38:06.899 INFO:teuthology.orchestra.run.smithi152.stdout: "complete_latency": { 2024-04-25T11:38:06.900 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 5353, 2024-04-25T11:38:06.900 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.153889198, 2024-04-25T11:38:06.900 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000028748 2024-04-25T11:38:06.900 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T11:38:06.900 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.900 INFO:teuthology.orchestra.run.smithi152.stdout: "finisher-PurgeQueue": { 2024-04-25T11:38:06.900 INFO:teuthology.orchestra.run.smithi152.stdout: "queue_len": 0, 2024-04-25T11:38:06.900 INFO:teuthology.orchestra.run.smithi152.stdout: "complete_latency": { 2024-04-25T11:38:06.900 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:06.900 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:06.900 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:06.900 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T11:38:06.900 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.900 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": { 2024-04-25T11:38:06.901 INFO:teuthology.orchestra.run.smithi152.stdout: "request": 0, 2024-04-25T11:38:06.901 INFO:teuthology.orchestra.run.smithi152.stdout: "reply": 0, 2024-04-25T11:38:06.901 INFO:teuthology.orchestra.run.smithi152.stdout: "reply_latency": { 2024-04-25T11:38:06.901 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:06.901 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:06.901 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:06.901 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.901 INFO:teuthology.orchestra.run.smithi152.stdout: "slow_reply": 0, 2024-04-25T11:38:06.901 INFO:teuthology.orchestra.run.smithi152.stdout: "forward": 0, 2024-04-25T11:38:06.901 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_fetch_complete": 0, 2024-04-25T11:38:06.901 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_fetch_keys": 0, 2024-04-25T11:38:06.901 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_commit": 0, 2024-04-25T11:38:06.901 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_split": 0, 2024-04-25T11:38:06.901 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_merge": 0, 2024-04-25T11:38:06.902 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes": 18, 2024-04-25T11:38:06.902 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_top": 0, 2024-04-25T11:38:06.902 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_bottom": 18, 2024-04-25T11:38:06.902 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_pin_tail": 0, 2024-04-25T11:38:06.902 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_pinned": 18, 2024-04-25T11:38:06.902 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_expired": 0, 2024-04-25T11:38:06.902 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_with_caps": 0, 2024-04-25T11:38:06.902 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": 0, 2024-04-25T11:38:06.902 INFO:teuthology.orchestra.run.smithi152.stdout: "subtrees": 2, 2024-04-25T11:38:06.902 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse": 0, 2024-04-25T11:38:06.902 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_hit": 0, 2024-04-25T11:38:06.902 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_forward": 0, 2024-04-25T11:38:06.902 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_discover": 0, 2024-04-25T11:38:06.903 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_dir_fetch": 0, 2024-04-25T11:38:06.903 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_remote_ino": 0, 2024-04-25T11:38:06.903 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_lock": 0, 2024-04-25T11:38:06.903 INFO:teuthology.orchestra.run.smithi152.stdout: "load_cent": 0, 2024-04-25T11:38:06.903 INFO:teuthology.orchestra.run.smithi152.stdout: "q": 0, 2024-04-25T11:38:06.903 INFO:teuthology.orchestra.run.smithi152.stdout: "exported": 0, 2024-04-25T11:38:06.903 INFO:teuthology.orchestra.run.smithi152.stdout: "exported_inodes": 0, 2024-04-25T11:38:06.903 INFO:teuthology.orchestra.run.smithi152.stdout: "imported": 0, 2024-04-25T11:38:06.903 INFO:teuthology.orchestra.run.smithi152.stdout: "imported_inodes": 0, 2024-04-25T11:38:06.903 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_dir_fetch": 0, 2024-04-25T11:38:06.903 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_backtrace_fetch": 0, 2024-04-25T11:38:06.903 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_peer_discover": 0, 2024-04-25T11:38:06.903 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rfiles": 4, 2024-04-25T11:38:06.903 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rbytes": 234, 2024-04-25T11:38:06.903 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rsnaps": 0, 2024-04-25T11:38:06.904 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T11:38:06.904 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_set_tag": 0, 2024-04-25T11:38:06.904 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T11:38:06.904 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_inotable_repaired": 0, 2024-04-25T11:38:06.904 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dir_inodes": 0, 2024-04-25T11:38:06.904 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T11:38:06.904 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T11:38:06.904 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_file_inodes": 0, 2024-04-25T11:38:06.904 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_inode_file_caps": 0, 2024-04-25T11:38:06.904 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T11:38:06.904 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_grant": 0, 2024-04-25T11:38:06.904 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T11:38:06.904 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T11:38:06.904 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T11:38:06.905 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_caps": 0, 2024-04-25T11:38:06.905 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_caps_dirty": 0, 2024-04-25T11:38:06.905 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_cap_release": 0, 2024-04-25T11:38:06.905 INFO:teuthology.orchestra.run.smithi152.stdout: "process_request_cap_release": 0 2024-04-25T11:38:06.905 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.905 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_cache": { 2024-04-25T11:38:06.905 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_update": 0, 2024-04-25T11:38:06.905 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_update_receipt": 0, 2024-04-25T11:38:06.905 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_try_discover": 0, 2024-04-25T11:38:06.905 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_send_discover": 0, 2024-04-25T11:38:06.905 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_handle_discover": 0, 2024-04-25T11:38:06.905 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays": 0, 2024-04-25T11:38:06.905 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays_delayed": 0, 2024-04-25T11:38:06.905 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays_enqueuing": 0, 2024-04-25T11:38:06.906 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_created": 0, 2024-04-25T11:38:06.906 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_enqueued": 0, 2024-04-25T11:38:06.906 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_reintegrated": 0, 2024-04-25T11:38:06.906 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_migrated": 0, 2024-04-25T11:38:06.906 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_processing": 0, 2024-04-25T11:38:06.906 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_enqueued": 0, 2024-04-25T11:38:06.906 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_prioritized": 0, 2024-04-25T11:38:06.906 INFO:teuthology.orchestra.run.smithi152.stdout: "recovery_started": 0, 2024-04-25T11:38:06.906 INFO:teuthology.orchestra.run.smithi152.stdout: "recovery_completed": 0, 2024-04-25T11:38:06.906 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_quiesce_path": 0, 2024-04-25T11:38:06.906 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_quiesce_inode": 0, 2024-04-25T11:38:06.906 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T11:38:06.906 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_exportdir": 0, 2024-04-25T11:38:06.907 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_flush": 0, 2024-04-25T11:38:06.907 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_fragmentdir": 0, 2024-04-25T11:38:06.907 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_fragstats": 0, 2024-04-25T11:38:06.907 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_inodestats": 0 2024-04-25T11:38:06.907 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.907 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_log": { 2024-04-25T11:38:06.907 INFO:teuthology.orchestra.run.smithi152.stdout: "evlrg": 0, 2024-04-25T11:38:06.907 INFO:teuthology.orchestra.run.smithi152.stdout: "evadd": 0, 2024-04-25T11:38:06.907 INFO:teuthology.orchestra.run.smithi152.stdout: "evex": 0, 2024-04-25T11:38:06.907 INFO:teuthology.orchestra.run.smithi152.stdout: "evtrm": 0, 2024-04-25T11:38:06.907 INFO:teuthology.orchestra.run.smithi152.stdout: "ev": 2057, 2024-04-25T11:38:06.907 INFO:teuthology.orchestra.run.smithi152.stdout: "evexg": 0, 2024-04-25T11:38:06.907 INFO:teuthology.orchestra.run.smithi152.stdout: "evexd": 0, 2024-04-25T11:38:06.907 INFO:teuthology.orchestra.run.smithi152.stdout: "segadd": 0, 2024-04-25T11:38:06.908 INFO:teuthology.orchestra.run.smithi152.stdout: "segex": 0, 2024-04-25T11:38:06.908 INFO:teuthology.orchestra.run.smithi152.stdout: "segtrm": 0, 2024-04-25T11:38:06.908 INFO:teuthology.orchestra.run.smithi152.stdout: "seg": 4, 2024-04-25T11:38:06.908 INFO:teuthology.orchestra.run.smithi152.stdout: "segmjr": 2, 2024-04-25T11:38:06.908 INFO:teuthology.orchestra.run.smithi152.stdout: "segexg": 0, 2024-04-25T11:38:06.908 INFO:teuthology.orchestra.run.smithi152.stdout: "segexd": 0, 2024-04-25T11:38:06.908 INFO:teuthology.orchestra.run.smithi152.stdout: "expos": 4194304, 2024-04-25T11:38:06.908 INFO:teuthology.orchestra.run.smithi152.stdout: "wrpos": 11012442, 2024-04-25T11:38:06.908 INFO:teuthology.orchestra.run.smithi152.stdout: "rdpos": 11007805, 2024-04-25T11:38:06.908 INFO:teuthology.orchestra.run.smithi152.stdout: "jlat": { 2024-04-25T11:38:06.908 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:06.908 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:06.908 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:06.908 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.908 INFO:teuthology.orchestra.run.smithi152.stdout: "replayed": 2057 2024-04-25T11:38:06.909 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.909 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_mem": { 2024-04-25T11:38:06.909 INFO:teuthology.orchestra.run.smithi152.stdout: "ino": 15, 2024-04-25T11:38:06.909 INFO:teuthology.orchestra.run.smithi152.stdout: "ino+": 17, 2024-04-25T11:38:06.909 INFO:teuthology.orchestra.run.smithi152.stdout: "ino-": 2, 2024-04-25T11:38:06.909 INFO:teuthology.orchestra.run.smithi152.stdout: "dir": 10, 2024-04-25T11:38:06.909 INFO:teuthology.orchestra.run.smithi152.stdout: "dir+": 11, 2024-04-25T11:38:06.909 INFO:teuthology.orchestra.run.smithi152.stdout: "dir-": 1, 2024-04-25T11:38:06.909 INFO:teuthology.orchestra.run.smithi152.stdout: "dn": 18, 2024-04-25T11:38:06.909 INFO:teuthology.orchestra.run.smithi152.stdout: "dn+": 19, 2024-04-25T11:38:06.910 INFO:teuthology.orchestra.run.smithi152.stdout: "dn-": 1, 2024-04-25T11:38:06.910 INFO:teuthology.orchestra.run.smithi152.stdout: "cap": 0, 2024-04-25T11:38:06.910 INFO:teuthology.orchestra.run.smithi152.stdout: "cap+": 0, 2024-04-25T11:38:06.910 INFO:teuthology.orchestra.run.smithi152.stdout: "cap-": 0, 2024-04-25T11:38:06.910 INFO:teuthology.orchestra.run.smithi152.stdout: "rss": 120120, 2024-04-25T11:38:06.910 INFO:teuthology.orchestra.run.smithi152.stdout: "heap": 215440 2024-04-25T11:38:06.910 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.910 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_server": { 2024-04-25T11:38:06.910 INFO:teuthology.orchestra.run.smithi152.stdout: "dispatch_client_request": 0, 2024-04-25T11:38:06.910 INFO:teuthology.orchestra.run.smithi152.stdout: "dispatch_server_request": 0, 2024-04-25T11:38:06.910 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_request": 0, 2024-04-25T11:38:06.910 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_session": 0, 2024-04-25T11:38:06.910 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_peer_request": 0, 2024-04-25T11:38:06.910 INFO:teuthology.orchestra.run.smithi152.stdout: "req_create_latency": { 2024-04-25T11:38:06.911 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:06.911 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:06.911 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:06.911 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.911 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getattr_latency": { 2024-04-25T11:38:06.911 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:06.911 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:06.911 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:06.911 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.911 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getfilelock_latency": { 2024-04-25T11:38:06.911 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:06.911 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:06.911 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:06.911 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.912 INFO:teuthology.orchestra.run.smithi152.stdout: "req_link_latency": { 2024-04-25T11:38:06.912 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:06.912 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:06.912 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:06.912 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.912 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookup_latency": { 2024-04-25T11:38:06.912 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:06.912 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:06.912 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:06.912 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.912 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookuphash_latency": { 2024-04-25T11:38:06.912 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:06.912 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:06.912 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:06.913 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.913 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupino_latency": { 2024-04-25T11:38:06.913 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:06.913 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:06.913 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:06.913 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.913 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupname_latency": { 2024-04-25T11:38:06.913 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:06.913 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:06.913 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:06.913 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.913 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupparent_latency": { 2024-04-25T11:38:06.913 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:06.914 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:06.914 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:06.914 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.914 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupsnap_latency": { 2024-04-25T11:38:06.914 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:06.914 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:06.914 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:06.914 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.914 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lssnap_latency": { 2024-04-25T11:38:06.914 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:06.914 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:06.914 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:06.914 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.914 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mkdir_latency": { 2024-04-25T11:38:06.915 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:06.915 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:06.915 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:06.916 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.916 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mknod_latency": { 2024-04-25T11:38:06.916 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:06.916 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:06.916 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:06.916 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.916 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mksnap_latency": { 2024-04-25T11:38:06.916 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:06.916 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:06.916 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:06.916 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.916 INFO:teuthology.orchestra.run.smithi152.stdout: "req_open_latency": { 2024-04-25T11:38:06.916 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:06.916 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:06.917 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:06.917 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.917 INFO:teuthology.orchestra.run.smithi152.stdout: "req_readdir_latency": { 2024-04-25T11:38:06.917 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:06.917 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:06.917 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:06.917 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.917 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rename_latency": { 2024-04-25T11:38:06.917 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:06.917 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:06.917 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:06.917 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.917 INFO:teuthology.orchestra.run.smithi152.stdout: "req_renamesnap_latency": { 2024-04-25T11:38:06.917 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:06.918 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:06.918 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:06.918 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.918 INFO:teuthology.orchestra.run.smithi152.stdout: "req_snapdiff_latency": { 2024-04-25T11:38:06.918 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:06.918 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:06.918 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:06.918 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.918 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmdir_latency": { 2024-04-25T11:38:06.918 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:06.918 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:06.918 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:06.919 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.919 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmsnap_latency": { 2024-04-25T11:38:06.919 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:06.919 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:06.919 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:06.919 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.919 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmxattr_latency": { 2024-04-25T11:38:06.919 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:06.919 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:06.919 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:06.919 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.919 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setattr_latency": { 2024-04-25T11:38:06.919 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:06.920 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:06.920 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:06.920 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.920 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setdirlayout_latency": { 2024-04-25T11:38:06.920 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:06.920 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:06.920 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:06.920 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.920 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setfilelock_latency": { 2024-04-25T11:38:06.920 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:06.920 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:06.920 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:06.920 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.920 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setlayout_latency": { 2024-04-25T11:38:06.921 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:06.921 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:06.921 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:06.921 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.921 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setxattr_latency": { 2024-04-25T11:38:06.921 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:06.921 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:06.921 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:06.921 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.921 INFO:teuthology.orchestra.run.smithi152.stdout: "req_symlink_latency": { 2024-04-25T11:38:06.921 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:06.921 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:06.921 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:06.921 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.922 INFO:teuthology.orchestra.run.smithi152.stdout: "req_unlink_latency": { 2024-04-25T11:38:06.922 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:06.922 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:06.922 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:06.922 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.922 INFO:teuthology.orchestra.run.smithi152.stdout: "cap_revoke_eviction": 0, 2024-04-25T11:38:06.922 INFO:teuthology.orchestra.run.smithi152.stdout: "cap_acquisition_throttle": 0, 2024-04-25T11:38:06.922 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getvxattr_latency": { 2024-04-25T11:38:06.922 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:06.922 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:06.922 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:06.922 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T11:38:06.922 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.923 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_sessions": { 2024-04-25T11:38:06.923 INFO:teuthology.orchestra.run.smithi152.stdout: "session_count": 1, 2024-04-25T11:38:06.923 INFO:teuthology.orchestra.run.smithi152.stdout: "session_add": 4, 2024-04-25T11:38:06.923 INFO:teuthology.orchestra.run.smithi152.stdout: "session_remove": 3, 2024-04-25T11:38:06.923 INFO:teuthology.orchestra.run.smithi152.stdout: "sessions_open": 2, 2024-04-25T11:38:06.923 INFO:teuthology.orchestra.run.smithi152.stdout: "sessions_stale": 0, 2024-04-25T11:38:06.923 INFO:teuthology.orchestra.run.smithi152.stdout: "total_load": 0, 2024-04-25T11:38:06.923 INFO:teuthology.orchestra.run.smithi152.stdout: "average_load": 0, 2024-04-25T11:38:06.923 INFO:teuthology.orchestra.run.smithi152.stdout: "avg_session_uptime": 2972, 2024-04-25T11:38:06.923 INFO:teuthology.orchestra.run.smithi152.stdout: "mdthresh_evicted": 0 2024-04-25T11:38:06.923 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.923 INFO:teuthology.orchestra.run.smithi152.stdout: "mempool": { 2024-04-25T11:38:06.923 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_bytes": 0, 2024-04-25T11:38:06.923 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_items": 0, 2024-04-25T11:38:06.924 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T11:38:06.924 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_items": 0, 2024-04-25T11:38:06.924 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T11:38:06.924 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_items": 0, 2024-04-25T11:38:06.924 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T11:38:06.924 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T11:38:06.924 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T11:38:06.924 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T11:38:06.924 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T11:38:06.924 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_items": 0, 2024-04-25T11:38:06.924 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T11:38:06.924 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T11:38:06.924 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_bytes": 0, 2024-04-25T11:38:06.924 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_items": 0, 2024-04-25T11:38:06.925 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_bytes": 0, 2024-04-25T11:38:06.925 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_items": 0, 2024-04-25T11:38:06.925 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T11:38:06.925 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T11:38:06.925 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T11:38:06.925 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T11:38:06.925 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T11:38:06.925 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_items": 0, 2024-04-25T11:38:06.925 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_bytes": 0, 2024-04-25T11:38:06.925 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_items": 0, 2024-04-25T11:38:06.925 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T11:38:06.925 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T11:38:06.925 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_bytes": 0, 2024-04-25T11:38:06.925 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_items": 0, 2024-04-25T11:38:06.925 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_bytes": 0, 2024-04-25T11:38:06.926 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_items": 0, 2024-04-25T11:38:06.926 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T11:38:06.926 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_items": 0, 2024-04-25T11:38:06.926 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T11:38:06.926 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_items": 0, 2024-04-25T11:38:06.926 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_bytes": 197543, 2024-04-25T11:38:06.926 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_items": 56, 2024-04-25T11:38:06.926 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_bytes": 0, 2024-04-25T11:38:06.926 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_items": 0, 2024-04-25T11:38:06.926 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_bytes": 0, 2024-04-25T11:38:06.926 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_items": 0, 2024-04-25T11:38:06.926 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_bytes": 0, 2024-04-25T11:38:06.926 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_items": 0, 2024-04-25T11:38:06.926 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_bytes": 0, 2024-04-25T11:38:06.927 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_items": 0, 2024-04-25T11:38:06.927 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_bytes": 9088, 2024-04-25T11:38:06.927 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_items": 175, 2024-04-25T11:38:06.927 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T11:38:06.927 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_items": 0, 2024-04-25T11:38:06.927 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_bytes": 0, 2024-04-25T11:38:06.927 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_items": 0, 2024-04-25T11:38:06.927 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_bytes": 59249, 2024-04-25T11:38:06.927 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_items": 339, 2024-04-25T11:38:06.927 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_bytes": 0, 2024-04-25T11:38:06.927 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_items": 0, 2024-04-25T11:38:06.927 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_bytes": 0, 2024-04-25T11:38:06.927 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_items": 0 2024-04-25T11:38:06.927 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.927 INFO:teuthology.orchestra.run.smithi152.stdout: "objecter": { 2024-04-25T11:38:06.928 INFO:teuthology.orchestra.run.smithi152.stdout: "op_active": 0, 2024-04-25T11:38:06.928 INFO:teuthology.orchestra.run.smithi152.stdout: "op_laggy": 0, 2024-04-25T11:38:06.928 INFO:teuthology.orchestra.run.smithi152.stdout: "op_send": 3137, 2024-04-25T11:38:06.928 INFO:teuthology.orchestra.run.smithi152.stdout: "op_send_bytes": 26, 2024-04-25T11:38:06.928 INFO:teuthology.orchestra.run.smithi152.stdout: "op_resend": 4, 2024-04-25T11:38:06.928 INFO:teuthology.orchestra.run.smithi152.stdout: "op_reply": 3137, 2024-04-25T11:38:06.928 INFO:teuthology.orchestra.run.smithi152.stdout: "op_latency": { 2024-04-25T11:38:06.928 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 3137, 2024-04-25T11:38:06.928 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 2.380969276, 2024-04-25T11:38:06.928 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000758995 2024-04-25T11:38:06.928 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.928 INFO:teuthology.orchestra.run.smithi152.stdout: "op_inflight": 0, 2024-04-25T11:38:06.928 INFO:teuthology.orchestra.run.smithi152.stdout: "oplen_avg": { 2024-04-25T11:38:06.928 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 3137, 2024-04-25T11:38:06.929 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 3138 2024-04-25T11:38:06.929 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.929 INFO:teuthology.orchestra.run.smithi152.stdout: "op": 3137, 2024-04-25T11:38:06.929 INFO:teuthology.orchestra.run.smithi152.stdout: "op_r": 3137, 2024-04-25T11:38:06.929 INFO:teuthology.orchestra.run.smithi152.stdout: "op_w": 0, 2024-04-25T11:38:06.929 INFO:teuthology.orchestra.run.smithi152.stdout: "op_rmw": 0, 2024-04-25T11:38:06.929 INFO:teuthology.orchestra.run.smithi152.stdout: "op_pg": 0, 2024-04-25T11:38:06.929 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_stat": 1674, 2024-04-25T11:38:06.929 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_create": 0, 2024-04-25T11:38:06.929 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_read": 1460, 2024-04-25T11:38:06.929 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_write": 0, 2024-04-25T11:38:06.929 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_writefull": 0, 2024-04-25T11:38:06.929 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_writesame": 0, 2024-04-25T11:38:06.929 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_append": 0, 2024-04-25T11:38:06.930 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_zero": 0, 2024-04-25T11:38:06.930 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_truncate": 0, 2024-04-25T11:38:06.930 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_delete": 0, 2024-04-25T11:38:06.930 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_mapext": 0, 2024-04-25T11:38:06.930 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_sparse_read": 0, 2024-04-25T11:38:06.930 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_clonerange": 0, 2024-04-25T11:38:06.930 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_getxattr": 2, 2024-04-25T11:38:06.930 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_setxattr": 0, 2024-04-25T11:38:06.930 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_cmpxattr": 0, 2024-04-25T11:38:06.930 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_rmxattr": 0, 2024-04-25T11:38:06.930 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_resetxattrs": 0, 2024-04-25T11:38:06.930 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_call": 0, 2024-04-25T11:38:06.930 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_watch": 0, 2024-04-25T11:38:06.930 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_notify": 0, 2024-04-25T11:38:06.931 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T11:38:06.931 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_pgls": 0, 2024-04-25T11:38:06.931 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_pgls_filter": 0, 2024-04-25T11:38:06.931 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_other": 0, 2024-04-25T11:38:06.931 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_active": 0, 2024-04-25T11:38:06.931 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_send": 0, 2024-04-25T11:38:06.931 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_resend": 0, 2024-04-25T11:38:06.931 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_ping": 0, 2024-04-25T11:38:06.931 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_active": 0, 2024-04-25T11:38:06.931 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_send": 0, 2024-04-25T11:38:06.931 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_resend": 0, 2024-04-25T11:38:06.931 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_active": 0, 2024-04-25T11:38:06.931 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_send": 0, 2024-04-25T11:38:06.932 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_resend": 0, 2024-04-25T11:38:06.932 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_active": 0, 2024-04-25T11:38:06.932 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_send": 0, 2024-04-25T11:38:06.932 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_resend": 0, 2024-04-25T11:38:06.932 INFO:teuthology.orchestra.run.smithi152.stdout: "command_active": 0, 2024-04-25T11:38:06.932 INFO:teuthology.orchestra.run.smithi152.stdout: "command_send": 0, 2024-04-25T11:38:06.932 INFO:teuthology.orchestra.run.smithi152.stdout: "command_resend": 0, 2024-04-25T11:38:06.932 INFO:teuthology.orchestra.run.smithi152.stdout: "map_epoch": 84, 2024-04-25T11:38:06.932 INFO:teuthology.orchestra.run.smithi152.stdout: "map_full": 0, 2024-04-25T11:38:06.932 INFO:teuthology.orchestra.run.smithi152.stdout: "map_inc": 4, 2024-04-25T11:38:06.932 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_sessions": 6, 2024-04-25T11:38:06.932 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_session_open": 6, 2024-04-25T11:38:06.932 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_session_close": 0, 2024-04-25T11:38:06.932 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_laggy": 0, 2024-04-25T11:38:06.933 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_wr": 0, 2024-04-25T11:38:06.933 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_rd": 2, 2024-04-25T11:38:06.933 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_del": 0 2024-04-25T11:38:06.933 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.933 INFO:teuthology.orchestra.run.smithi152.stdout: "oft": { 2024-04-25T11:38:06.933 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_objs": 0, 2024-04-25T11:38:06.933 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_kv_pairs": 0, 2024-04-25T11:38:06.933 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_updates": 0, 2024-04-25T11:38:06.933 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_removes": 0 2024-04-25T11:38:06.933 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.933 INFO:teuthology.orchestra.run.smithi152.stdout: "osd-slow-ops": { 2024-04-25T11:38:06.933 INFO:teuthology.orchestra.run.smithi152.stdout: "slow_ops_count": 0 2024-04-25T11:38:06.933 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.933 INFO:teuthology.orchestra.run.smithi152.stdout: "purge_queue": { 2024-04-25T11:38:06.934 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_ops": 0, 2024-04-25T11:38:06.934 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T11:38:06.934 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing": 0, 2024-04-25T11:38:06.934 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_high_water": 0, 2024-04-25T11:38:06.934 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executed_ops": 0, 2024-04-25T11:38:06.934 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executed": 0, 2024-04-25T11:38:06.934 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_item_in_journal": 0 2024-04-25T11:38:06.934 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.934 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T11:38:06.934 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-04-25T11:38:06.934 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 104857600, 2024-04-25T11:38:06.934 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-04-25T11:38:06.934 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 3395, 2024-04-25T11:38:06.934 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 8213434, 2024-04-25T11:38:06.935 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-04-25T11:38:06.935 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 3395, 2024-04-25T11:38:06.935 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-04-25T11:38:06.935 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-04-25T11:38:06.935 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 3395, 2024-04-25T11:38:06.935 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 8213434, 2024-04-25T11:38:06.935 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-04-25T11:38:06.935 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:06.935 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:06.935 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:06.935 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T11:38:06.935 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.935 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-objecter_bytes": { 2024-04-25T11:38:06.936 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-04-25T11:38:06.936 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 104857600, 2024-04-25T11:38:06.936 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-04-25T11:38:06.936 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-04-25T11:38:06.936 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-04-25T11:38:06.936 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-04-25T11:38:06.936 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-04-25T11:38:06.936 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 3137, 2024-04-25T11:38:06.936 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 6818148, 2024-04-25T11:38:06.936 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 3137, 2024-04-25T11:38:06.936 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 6818148, 2024-04-25T11:38:06.936 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-04-25T11:38:06.936 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:06.936 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:06.937 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:06.937 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T11:38:06.937 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.937 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-objecter_ops": { 2024-04-25T11:38:06.937 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-04-25T11:38:06.937 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 1024, 2024-04-25T11:38:06.937 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-04-25T11:38:06.937 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-04-25T11:38:06.937 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-04-25T11:38:06.937 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-04-25T11:38:06.937 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-04-25T11:38:06.937 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 3137, 2024-04-25T11:38:06.937 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 3137, 2024-04-25T11:38:06.937 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 3137, 2024-04-25T11:38:06.937 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 3137, 2024-04-25T11:38:06.938 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-04-25T11:38:06.938 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:06.938 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:06.938 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:06.938 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T11:38:06.938 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.938 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-write_buf_throttle": { 2024-04-25T11:38:06.938 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-04-25T11:38:06.938 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 3758096384, 2024-04-25T11:38:06.938 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-04-25T11:38:06.938 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-04-25T11:38:06.938 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-04-25T11:38:06.938 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-04-25T11:38:06.939 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-04-25T11:38:06.939 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-04-25T11:38:06.939 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-04-25T11:38:06.939 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 0, 2024-04-25T11:38:06.939 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 0, 2024-04-25T11:38:06.939 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-04-25T11:38:06.939 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:06.939 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:06.939 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:06.939 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T11:38:06.939 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:06.939 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-write_buf_throttle-0x55c29e5088f0": { 2024-04-25T11:38:06.939 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-04-25T11:38:06.939 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 3758096384, 2024-04-25T11:38:06.940 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-04-25T11:38:06.940 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-04-25T11:38:06.940 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-04-25T11:38:06.940 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-04-25T11:38:06.940 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-04-25T11:38:06.940 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-04-25T11:38:06.940 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-04-25T11:38:06.940 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 0, 2024-04-25T11:38:06.940 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 0, 2024-04-25T11:38:06.940 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-04-25T11:38:06.940 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:06.940 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:06.940 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:06.940 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T11:38:06.940 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T11:38:06.941 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-25T11:38:07.474 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:07 smithi152 ceph-mon[32701]: pgmap v721: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T11:38:07.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:07 smithi146 ceph-mon[32821]: pgmap v721: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T11:38:07.669 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-25T11:38:07.669 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-25T11:38:07.670 DEBUG:tasks.check_counter:Getting stats from f 2024-04-25T11:38:07.670 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-25T11:38:07.686 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:07 smithi069 ceph-mon[29188]: pgmap v721: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T11:38:07.920 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T11:38:09.442 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:09 smithi152 ceph-mon[32701]: pgmap v722: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:38:09.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:09 smithi146 ceph-mon[32821]: pgmap v722: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:38:09.686 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:09 smithi069 ceph-mon[29188]: pgmap v722: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:38:09.712 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-25T11:38:09.712 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T11:38:09.713 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 872, 2024-04-25T11:38:09.713 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 872, 2024-04-25T11:38:09.713 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 294074, 2024-04-25T11:38:09.713 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 276479, 2024-04-25T11:38:09.713 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 6, 2024-04-25T11:38:09.713 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 2, 2024-04-25T11:38:09.713 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.123631601, 2024-04-25T11:38:09.713 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.037236895, 2024-04-25T11:38:09.713 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.034071450, 2024-04-25T11:38:09.713 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.040411796, 2024-04-25T11:38:09.713 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:38:09.713 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 871, 2024-04-25T11:38:09.713 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.054481607, 2024-04-25T11:38:09.714 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000062550 2024-04-25T11:38:09.714 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.714 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:38:09.714 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:09.714 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:09.714 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:09.714 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.714 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 2960, 2024-04-25T11:38:09.714 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T11:38:09.714 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.714 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T11:38:09.714 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 1835, 2024-04-25T11:38:09.714 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 1825, 2024-04-25T11:38:09.715 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 1253986, 2024-04-25T11:38:09.715 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 650348, 2024-04-25T11:38:09.715 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 4, 2024-04-25T11:38:09.715 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 3, 2024-04-25T11:38:09.715 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.174791629, 2024-04-25T11:38:09.715 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.055528359, 2024-04-25T11:38:09.715 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.062996046, 2024-04-25T11:38:09.715 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.050930934, 2024-04-25T11:38:09.715 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:38:09.715 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 1824, 2024-04-25T11:38:09.715 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.080630995, 2024-04-25T11:38:09.715 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000044205 2024-04-25T11:38:09.715 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.715 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:38:09.716 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:09.716 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:09.716 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:09.716 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.716 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 851856, 2024-04-25T11:38:09.716 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 148800 2024-04-25T11:38:09.716 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.716 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T11:38:09.716 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 2, 2024-04-25T11:38:09.716 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 183, 2024-04-25T11:38:09.716 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 428, 2024-04-25T11:38:09.716 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 391534, 2024-04-25T11:38:09.716 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 4, 2024-04-25T11:38:09.716 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 3, 2024-04-25T11:38:09.716 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.010746116, 2024-04-25T11:38:09.716 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.007441425, 2024-04-25T11:38:09.717 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.001417085, 2024-04-25T11:38:09.717 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.000035148, 2024-04-25T11:38:09.717 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:38:09.717 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 181, 2024-04-25T11:38:09.717 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.012073692, 2024-04-25T11:38:09.717 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000066705 2024-04-25T11:38:09.717 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.717 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:38:09.717 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:09.717 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:09.717 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:09.717 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.717 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T11:38:09.717 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 391216 2024-04-25T11:38:09.717 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.717 INFO:teuthology.orchestra.run.smithi152.stdout: "cct": { 2024-04-25T11:38:09.718 INFO:teuthology.orchestra.run.smithi152.stdout: "total_workers": 1, 2024-04-25T11:38:09.718 INFO:teuthology.orchestra.run.smithi152.stdout: "unhealthy_workers": 0 2024-04-25T11:38:09.718 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.718 INFO:teuthology.orchestra.run.smithi152.stdout: "finisher-MDSRank": { 2024-04-25T11:38:09.718 INFO:teuthology.orchestra.run.smithi152.stdout: "queue_len": 0, 2024-04-25T11:38:09.718 INFO:teuthology.orchestra.run.smithi152.stdout: "complete_latency": { 2024-04-25T11:38:09.718 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 4128, 2024-04-25T11:38:09.718 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.136067016, 2024-04-25T11:38:09.718 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000032961 2024-04-25T11:38:09.718 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T11:38:09.718 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.718 INFO:teuthology.orchestra.run.smithi152.stdout: "finisher-PurgeQueue": { 2024-04-25T11:38:09.718 INFO:teuthology.orchestra.run.smithi152.stdout: "queue_len": 0, 2024-04-25T11:38:09.719 INFO:teuthology.orchestra.run.smithi152.stdout: "complete_latency": { 2024-04-25T11:38:09.719 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:09.719 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:09.719 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:09.719 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T11:38:09.719 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.719 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": { 2024-04-25T11:38:09.719 INFO:teuthology.orchestra.run.smithi152.stdout: "request": 0, 2024-04-25T11:38:09.719 INFO:teuthology.orchestra.run.smithi152.stdout: "reply": 0, 2024-04-25T11:38:09.719 INFO:teuthology.orchestra.run.smithi152.stdout: "reply_latency": { 2024-04-25T11:38:09.719 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:09.719 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:09.719 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:09.719 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.720 INFO:teuthology.orchestra.run.smithi152.stdout: "slow_reply": 0, 2024-04-25T11:38:09.720 INFO:teuthology.orchestra.run.smithi152.stdout: "forward": 0, 2024-04-25T11:38:09.720 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_fetch_complete": 0, 2024-04-25T11:38:09.720 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_fetch_keys": 0, 2024-04-25T11:38:09.720 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_commit": 0, 2024-04-25T11:38:09.720 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_split": 0, 2024-04-25T11:38:09.720 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_merge": 0, 2024-04-25T11:38:09.720 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes": 0, 2024-04-25T11:38:09.720 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_top": 0, 2024-04-25T11:38:09.720 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_bottom": 0, 2024-04-25T11:38:09.720 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_pin_tail": 0, 2024-04-25T11:38:09.720 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_pinned": 0, 2024-04-25T11:38:09.720 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_expired": 0, 2024-04-25T11:38:09.720 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_with_caps": 0, 2024-04-25T11:38:09.720 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": 0, 2024-04-25T11:38:09.721 INFO:teuthology.orchestra.run.smithi152.stdout: "subtrees": 1, 2024-04-25T11:38:09.721 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse": 0, 2024-04-25T11:38:09.721 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_hit": 0, 2024-04-25T11:38:09.721 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_forward": 0, 2024-04-25T11:38:09.721 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_discover": 0, 2024-04-25T11:38:09.721 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_dir_fetch": 0, 2024-04-25T11:38:09.721 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_remote_ino": 0, 2024-04-25T11:38:09.721 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_lock": 0, 2024-04-25T11:38:09.721 INFO:teuthology.orchestra.run.smithi152.stdout: "load_cent": 0, 2024-04-25T11:38:09.721 INFO:teuthology.orchestra.run.smithi152.stdout: "q": 0, 2024-04-25T11:38:09.721 INFO:teuthology.orchestra.run.smithi152.stdout: "exported": 0, 2024-04-25T11:38:09.721 INFO:teuthology.orchestra.run.smithi152.stdout: "exported_inodes": 0, 2024-04-25T11:38:09.721 INFO:teuthology.orchestra.run.smithi152.stdout: "imported": 0, 2024-04-25T11:38:09.721 INFO:teuthology.orchestra.run.smithi152.stdout: "imported_inodes": 0, 2024-04-25T11:38:09.722 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_dir_fetch": 0, 2024-04-25T11:38:09.722 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_backtrace_fetch": 0, 2024-04-25T11:38:09.722 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_peer_discover": 0, 2024-04-25T11:38:09.722 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rfiles": 0, 2024-04-25T11:38:09.722 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rbytes": 0, 2024-04-25T11:38:09.722 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rsnaps": 0, 2024-04-25T11:38:09.722 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T11:38:09.722 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_set_tag": 0, 2024-04-25T11:38:09.722 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T11:38:09.722 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_inotable_repaired": 0, 2024-04-25T11:38:09.722 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dir_inodes": 0, 2024-04-25T11:38:09.722 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T11:38:09.722 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T11:38:09.722 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_file_inodes": 0, 2024-04-25T11:38:09.723 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_inode_file_caps": 0, 2024-04-25T11:38:09.723 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T11:38:09.723 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_grant": 0, 2024-04-25T11:38:09.723 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T11:38:09.723 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T11:38:09.723 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T11:38:09.723 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_caps": 0, 2024-04-25T11:38:09.723 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_caps_dirty": 0, 2024-04-25T11:38:09.723 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_cap_release": 0, 2024-04-25T11:38:09.723 INFO:teuthology.orchestra.run.smithi152.stdout: "process_request_cap_release": 0 2024-04-25T11:38:09.723 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.723 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_cache": { 2024-04-25T11:38:09.723 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_update": 0, 2024-04-25T11:38:09.723 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_update_receipt": 0, 2024-04-25T11:38:09.724 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_try_discover": 0, 2024-04-25T11:38:09.724 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_send_discover": 0, 2024-04-25T11:38:09.724 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_handle_discover": 0, 2024-04-25T11:38:09.724 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays": 0, 2024-04-25T11:38:09.724 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays_delayed": 0, 2024-04-25T11:38:09.724 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays_enqueuing": 0, 2024-04-25T11:38:09.724 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_created": 0, 2024-04-25T11:38:09.724 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_enqueued": 0, 2024-04-25T11:38:09.724 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_reintegrated": 0, 2024-04-25T11:38:09.724 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_migrated": 0, 2024-04-25T11:38:09.724 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_processing": 0, 2024-04-25T11:38:09.724 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_enqueued": 0, 2024-04-25T11:38:09.724 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_prioritized": 0, 2024-04-25T11:38:09.725 INFO:teuthology.orchestra.run.smithi152.stdout: "recovery_started": 0, 2024-04-25T11:38:09.725 INFO:teuthology.orchestra.run.smithi152.stdout: "recovery_completed": 0, 2024-04-25T11:38:09.725 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_quiesce_path": 0, 2024-04-25T11:38:09.725 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_quiesce_inode": 0, 2024-04-25T11:38:09.725 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T11:38:09.725 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_exportdir": 0, 2024-04-25T11:38:09.725 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_flush": 0, 2024-04-25T11:38:09.725 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_fragmentdir": 0, 2024-04-25T11:38:09.725 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_fragstats": 0, 2024-04-25T11:38:09.725 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_inodestats": 0 2024-04-25T11:38:09.725 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.725 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_log": { 2024-04-25T11:38:09.725 INFO:teuthology.orchestra.run.smithi152.stdout: "evlrg": 0, 2024-04-25T11:38:09.725 INFO:teuthology.orchestra.run.smithi152.stdout: "evadd": 0, 2024-04-25T11:38:09.725 INFO:teuthology.orchestra.run.smithi152.stdout: "evex": 0, 2024-04-25T11:38:09.726 INFO:teuthology.orchestra.run.smithi152.stdout: "evtrm": 0, 2024-04-25T11:38:09.726 INFO:teuthology.orchestra.run.smithi152.stdout: "ev": 5, 2024-04-25T11:38:09.726 INFO:teuthology.orchestra.run.smithi152.stdout: "evexg": 0, 2024-04-25T11:38:09.726 INFO:teuthology.orchestra.run.smithi152.stdout: "evexd": 0, 2024-04-25T11:38:09.726 INFO:teuthology.orchestra.run.smithi152.stdout: "segadd": 0, 2024-04-25T11:38:09.726 INFO:teuthology.orchestra.run.smithi152.stdout: "segex": 0, 2024-04-25T11:38:09.726 INFO:teuthology.orchestra.run.smithi152.stdout: "segtrm": 0, 2024-04-25T11:38:09.726 INFO:teuthology.orchestra.run.smithi152.stdout: "seg": 2, 2024-04-25T11:38:09.726 INFO:teuthology.orchestra.run.smithi152.stdout: "segmjr": 2, 2024-04-25T11:38:09.726 INFO:teuthology.orchestra.run.smithi152.stdout: "segexg": 0, 2024-04-25T11:38:09.726 INFO:teuthology.orchestra.run.smithi152.stdout: "segexd": 0, 2024-04-25T11:38:09.726 INFO:teuthology.orchestra.run.smithi152.stdout: "expos": 4194304, 2024-04-25T11:38:09.726 INFO:teuthology.orchestra.run.smithi152.stdout: "wrpos": 4195917, 2024-04-25T11:38:09.726 INFO:teuthology.orchestra.run.smithi152.stdout: "rdpos": 4195424, 2024-04-25T11:38:09.727 INFO:teuthology.orchestra.run.smithi152.stdout: "jlat": { 2024-04-25T11:38:09.727 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:09.727 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:09.727 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:09.727 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.727 INFO:teuthology.orchestra.run.smithi152.stdout: "replayed": 5 2024-04-25T11:38:09.727 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.727 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_mem": { 2024-04-25T11:38:09.727 INFO:teuthology.orchestra.run.smithi152.stdout: "ino": 3, 2024-04-25T11:38:09.727 INFO:teuthology.orchestra.run.smithi152.stdout: "ino+": 3, 2024-04-25T11:38:09.727 INFO:teuthology.orchestra.run.smithi152.stdout: "ino-": 0, 2024-04-25T11:38:09.727 INFO:teuthology.orchestra.run.smithi152.stdout: "dir": 1, 2024-04-25T11:38:09.727 INFO:teuthology.orchestra.run.smithi152.stdout: "dir+": 1, 2024-04-25T11:38:09.727 INFO:teuthology.orchestra.run.smithi152.stdout: "dir-": 0, 2024-04-25T11:38:09.728 INFO:teuthology.orchestra.run.smithi152.stdout: "dn": 0, 2024-04-25T11:38:09.728 INFO:teuthology.orchestra.run.smithi152.stdout: "dn+": 0, 2024-04-25T11:38:09.728 INFO:teuthology.orchestra.run.smithi152.stdout: "dn-": 0, 2024-04-25T11:38:09.728 INFO:teuthology.orchestra.run.smithi152.stdout: "cap": 0, 2024-04-25T11:38:09.728 INFO:teuthology.orchestra.run.smithi152.stdout: "cap+": 0, 2024-04-25T11:38:09.728 INFO:teuthology.orchestra.run.smithi152.stdout: "cap-": 0, 2024-04-25T11:38:09.728 INFO:teuthology.orchestra.run.smithi152.stdout: "rss": 118152, 2024-04-25T11:38:09.728 INFO:teuthology.orchestra.run.smithi152.stdout: "heap": 215440 2024-04-25T11:38:09.728 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.728 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_server": { 2024-04-25T11:38:09.728 INFO:teuthology.orchestra.run.smithi152.stdout: "dispatch_client_request": 0, 2024-04-25T11:38:09.728 INFO:teuthology.orchestra.run.smithi152.stdout: "dispatch_server_request": 0, 2024-04-25T11:38:09.728 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_request": 0, 2024-04-25T11:38:09.728 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_session": 0, 2024-04-25T11:38:09.729 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_peer_request": 0, 2024-04-25T11:38:09.729 INFO:teuthology.orchestra.run.smithi152.stdout: "req_create_latency": { 2024-04-25T11:38:09.729 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:09.729 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:09.729 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:09.729 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.729 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getattr_latency": { 2024-04-25T11:38:09.729 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:09.729 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:09.729 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:09.729 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.729 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getfilelock_latency": { 2024-04-25T11:38:09.730 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:09.730 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:09.730 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:09.730 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.730 INFO:teuthology.orchestra.run.smithi152.stdout: "req_link_latency": { 2024-04-25T11:38:09.730 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:09.730 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:09.730 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:09.730 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.730 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookup_latency": { 2024-04-25T11:38:09.730 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:09.730 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:09.730 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:09.731 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.731 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookuphash_latency": { 2024-04-25T11:38:09.731 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:09.731 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:09.731 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:09.731 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.731 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupino_latency": { 2024-04-25T11:38:09.731 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:09.731 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:09.731 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:09.731 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.731 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupname_latency": { 2024-04-25T11:38:09.731 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:09.731 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:09.731 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:09.731 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.732 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupparent_latency": { 2024-04-25T11:38:09.732 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:09.732 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:09.732 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:09.732 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.732 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupsnap_latency": { 2024-04-25T11:38:09.732 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:09.732 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:09.732 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:09.732 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.732 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lssnap_latency": { 2024-04-25T11:38:09.732 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:09.732 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:09.733 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:09.733 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.733 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mkdir_latency": { 2024-04-25T11:38:09.733 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:09.733 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:09.733 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:09.734 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.734 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mknod_latency": { 2024-04-25T11:38:09.734 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:09.734 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:09.734 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:09.734 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.734 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mksnap_latency": { 2024-04-25T11:38:09.734 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:09.734 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:09.734 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:09.734 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.734 INFO:teuthology.orchestra.run.smithi152.stdout: "req_open_latency": { 2024-04-25T11:38:09.734 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:09.735 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:09.735 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:09.735 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.735 INFO:teuthology.orchestra.run.smithi152.stdout: "req_readdir_latency": { 2024-04-25T11:38:09.735 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:09.735 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:09.735 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:09.735 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.735 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rename_latency": { 2024-04-25T11:38:09.735 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:09.735 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:09.735 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:09.735 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.735 INFO:teuthology.orchestra.run.smithi152.stdout: "req_renamesnap_latency": { 2024-04-25T11:38:09.736 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:09.736 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:09.736 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:09.736 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.736 INFO:teuthology.orchestra.run.smithi152.stdout: "req_snapdiff_latency": { 2024-04-25T11:38:09.736 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:09.736 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:09.736 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:09.736 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.736 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmdir_latency": { 2024-04-25T11:38:09.736 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:09.736 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:09.736 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:09.737 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.737 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmsnap_latency": { 2024-04-25T11:38:09.737 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:09.737 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:09.737 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:09.737 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.737 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmxattr_latency": { 2024-04-25T11:38:09.737 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:09.737 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:09.737 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:09.737 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.737 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setattr_latency": { 2024-04-25T11:38:09.738 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:09.738 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:09.738 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:09.738 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.738 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setdirlayout_latency": { 2024-04-25T11:38:09.738 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:09.738 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:09.738 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:09.738 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.738 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setfilelock_latency": { 2024-04-25T11:38:09.738 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:09.738 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:09.738 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:09.739 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.739 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setlayout_latency": { 2024-04-25T11:38:09.739 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:09.739 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:09.739 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:09.739 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.739 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setxattr_latency": { 2024-04-25T11:38:09.739 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:09.739 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:09.739 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:09.739 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.739 INFO:teuthology.orchestra.run.smithi152.stdout: "req_symlink_latency": { 2024-04-25T11:38:09.739 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:09.739 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:09.740 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:09.740 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.740 INFO:teuthology.orchestra.run.smithi152.stdout: "req_unlink_latency": { 2024-04-25T11:38:09.740 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:09.740 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:09.740 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:09.740 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.740 INFO:teuthology.orchestra.run.smithi152.stdout: "cap_revoke_eviction": 0, 2024-04-25T11:38:09.740 INFO:teuthology.orchestra.run.smithi152.stdout: "cap_acquisition_throttle": 0, 2024-04-25T11:38:09.740 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getvxattr_latency": { 2024-04-25T11:38:09.740 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:09.740 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:09.740 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:09.740 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T11:38:09.740 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.741 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_sessions": { 2024-04-25T11:38:09.741 INFO:teuthology.orchestra.run.smithi152.stdout: "session_count": 1, 2024-04-25T11:38:09.741 INFO:teuthology.orchestra.run.smithi152.stdout: "session_add": 2, 2024-04-25T11:38:09.741 INFO:teuthology.orchestra.run.smithi152.stdout: "session_remove": 1, 2024-04-25T11:38:09.741 INFO:teuthology.orchestra.run.smithi152.stdout: "sessions_open": 1, 2024-04-25T11:38:09.741 INFO:teuthology.orchestra.run.smithi152.stdout: "sessions_stale": 0, 2024-04-25T11:38:09.741 INFO:teuthology.orchestra.run.smithi152.stdout: "total_load": 0, 2024-04-25T11:38:09.741 INFO:teuthology.orchestra.run.smithi152.stdout: "average_load": 0, 2024-04-25T11:38:09.741 INFO:teuthology.orchestra.run.smithi152.stdout: "avg_session_uptime": 1846, 2024-04-25T11:38:09.741 INFO:teuthology.orchestra.run.smithi152.stdout: "mdthresh_evicted": 0 2024-04-25T11:38:09.741 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.741 INFO:teuthology.orchestra.run.smithi152.stdout: "mempool": { 2024-04-25T11:38:09.741 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_bytes": 0, 2024-04-25T11:38:09.742 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_items": 0, 2024-04-25T11:38:09.742 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T11:38:09.742 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_items": 0, 2024-04-25T11:38:09.742 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T11:38:09.742 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_items": 0, 2024-04-25T11:38:09.742 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T11:38:09.742 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T11:38:09.742 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T11:38:09.742 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T11:38:09.742 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T11:38:09.742 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_items": 0, 2024-04-25T11:38:09.742 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T11:38:09.742 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T11:38:09.742 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_bytes": 0, 2024-04-25T11:38:09.743 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_items": 0, 2024-04-25T11:38:09.743 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_bytes": 0, 2024-04-25T11:38:09.743 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_items": 0, 2024-04-25T11:38:09.743 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T11:38:09.743 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T11:38:09.743 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T11:38:09.743 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T11:38:09.743 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T11:38:09.743 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_items": 0, 2024-04-25T11:38:09.743 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_bytes": 0, 2024-04-25T11:38:09.743 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_items": 0, 2024-04-25T11:38:09.743 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T11:38:09.743 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T11:38:09.743 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_bytes": 0, 2024-04-25T11:38:09.743 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_items": 0, 2024-04-25T11:38:09.744 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_bytes": 0, 2024-04-25T11:38:09.744 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_items": 0, 2024-04-25T11:38:09.744 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T11:38:09.744 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_items": 0, 2024-04-25T11:38:09.744 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T11:38:09.744 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_items": 0, 2024-04-25T11:38:09.744 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_bytes": 189383, 2024-04-25T11:38:09.744 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_items": 50, 2024-04-25T11:38:09.744 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_bytes": 0, 2024-04-25T11:38:09.744 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_items": 0, 2024-04-25T11:38:09.744 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_bytes": 0, 2024-04-25T11:38:09.744 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_items": 0, 2024-04-25T11:38:09.744 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_bytes": 0, 2024-04-25T11:38:09.744 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_items": 0, 2024-04-25T11:38:09.744 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_bytes": 0, 2024-04-25T11:38:09.744 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_items": 0, 2024-04-25T11:38:09.745 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_bytes": 9088, 2024-04-25T11:38:09.745 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_items": 175, 2024-04-25T11:38:09.745 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T11:38:09.745 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_items": 0, 2024-04-25T11:38:09.745 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_bytes": 0, 2024-04-25T11:38:09.745 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_items": 0, 2024-04-25T11:38:09.745 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_bytes": 8512, 2024-04-25T11:38:09.745 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_items": 13, 2024-04-25T11:38:09.745 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_bytes": 0, 2024-04-25T11:38:09.745 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_items": 0, 2024-04-25T11:38:09.745 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_bytes": 0, 2024-04-25T11:38:09.745 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_items": 0 2024-04-25T11:38:09.745 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.745 INFO:teuthology.orchestra.run.smithi152.stdout: "objecter": { 2024-04-25T11:38:09.746 INFO:teuthology.orchestra.run.smithi152.stdout: "op_active": 0, 2024-04-25T11:38:09.746 INFO:teuthology.orchestra.run.smithi152.stdout: "op_laggy": 0, 2024-04-25T11:38:09.746 INFO:teuthology.orchestra.run.smithi152.stdout: "op_send": 2446, 2024-04-25T11:38:09.746 INFO:teuthology.orchestra.run.smithi152.stdout: "op_send_bytes": 21, 2024-04-25T11:38:09.746 INFO:teuthology.orchestra.run.smithi152.stdout: "op_resend": 3, 2024-04-25T11:38:09.746 INFO:teuthology.orchestra.run.smithi152.stdout: "op_reply": 2446, 2024-04-25T11:38:09.746 INFO:teuthology.orchestra.run.smithi152.stdout: "op_latency": { 2024-04-25T11:38:09.746 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 2446, 2024-04-25T11:38:09.746 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 2.151972262, 2024-04-25T11:38:09.746 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000879792 2024-04-25T11:38:09.746 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.746 INFO:teuthology.orchestra.run.smithi152.stdout: "op_inflight": 0, 2024-04-25T11:38:09.746 INFO:teuthology.orchestra.run.smithi152.stdout: "oplen_avg": { 2024-04-25T11:38:09.747 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 2446, 2024-04-25T11:38:09.747 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 2447 2024-04-25T11:38:09.747 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.747 INFO:teuthology.orchestra.run.smithi152.stdout: "op": 2446, 2024-04-25T11:38:09.747 INFO:teuthology.orchestra.run.smithi152.stdout: "op_r": 2446, 2024-04-25T11:38:09.747 INFO:teuthology.orchestra.run.smithi152.stdout: "op_w": 0, 2024-04-25T11:38:09.747 INFO:teuthology.orchestra.run.smithi152.stdout: "op_rmw": 0, 2024-04-25T11:38:09.747 INFO:teuthology.orchestra.run.smithi152.stdout: "op_pg": 0, 2024-04-25T11:38:09.747 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_stat": 1597, 2024-04-25T11:38:09.747 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_create": 0, 2024-04-25T11:38:09.747 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_read": 847, 2024-04-25T11:38:09.747 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_write": 0, 2024-04-25T11:38:09.747 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_writefull": 0, 2024-04-25T11:38:09.747 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_writesame": 0, 2024-04-25T11:38:09.747 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_append": 0, 2024-04-25T11:38:09.748 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_zero": 0, 2024-04-25T11:38:09.748 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_truncate": 0, 2024-04-25T11:38:09.748 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_delete": 0, 2024-04-25T11:38:09.748 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_mapext": 0, 2024-04-25T11:38:09.748 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_sparse_read": 0, 2024-04-25T11:38:09.748 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_clonerange": 0, 2024-04-25T11:38:09.748 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_getxattr": 1, 2024-04-25T11:38:09.748 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_setxattr": 0, 2024-04-25T11:38:09.748 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_cmpxattr": 0, 2024-04-25T11:38:09.748 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_rmxattr": 0, 2024-04-25T11:38:09.748 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_resetxattrs": 0, 2024-04-25T11:38:09.748 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_call": 0, 2024-04-25T11:38:09.748 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_watch": 0, 2024-04-25T11:38:09.748 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_notify": 0, 2024-04-25T11:38:09.748 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T11:38:09.749 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_pgls": 0, 2024-04-25T11:38:09.749 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_pgls_filter": 0, 2024-04-25T11:38:09.749 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_other": 0, 2024-04-25T11:38:09.749 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_active": 0, 2024-04-25T11:38:09.749 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_send": 0, 2024-04-25T11:38:09.749 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_resend": 0, 2024-04-25T11:38:09.749 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_ping": 0, 2024-04-25T11:38:09.749 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_active": 0, 2024-04-25T11:38:09.749 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_send": 0, 2024-04-25T11:38:09.749 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_resend": 0, 2024-04-25T11:38:09.749 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_active": 0, 2024-04-25T11:38:09.749 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_send": 0, 2024-04-25T11:38:09.749 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_resend": 0, 2024-04-25T11:38:09.749 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_active": 0, 2024-04-25T11:38:09.750 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_send": 0, 2024-04-25T11:38:09.750 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_resend": 0, 2024-04-25T11:38:09.750 INFO:teuthology.orchestra.run.smithi152.stdout: "command_active": 0, 2024-04-25T11:38:09.750 INFO:teuthology.orchestra.run.smithi152.stdout: "command_send": 0, 2024-04-25T11:38:09.750 INFO:teuthology.orchestra.run.smithi152.stdout: "command_resend": 0, 2024-04-25T11:38:09.750 INFO:teuthology.orchestra.run.smithi152.stdout: "map_epoch": 84, 2024-04-25T11:38:09.750 INFO:teuthology.orchestra.run.smithi152.stdout: "map_full": 0, 2024-04-25T11:38:09.750 INFO:teuthology.orchestra.run.smithi152.stdout: "map_inc": 4, 2024-04-25T11:38:09.750 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_sessions": 5, 2024-04-25T11:38:09.750 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_session_open": 5, 2024-04-25T11:38:09.750 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_session_close": 0, 2024-04-25T11:38:09.750 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_laggy": 0, 2024-04-25T11:38:09.750 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_wr": 0, 2024-04-25T11:38:09.751 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_rd": 2, 2024-04-25T11:38:09.751 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_del": 0 2024-04-25T11:38:09.751 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.751 INFO:teuthology.orchestra.run.smithi152.stdout: "oft": { 2024-04-25T11:38:09.751 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_objs": 0, 2024-04-25T11:38:09.751 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_kv_pairs": 0, 2024-04-25T11:38:09.751 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_updates": 0, 2024-04-25T11:38:09.751 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_removes": 0 2024-04-25T11:38:09.751 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.751 INFO:teuthology.orchestra.run.smithi152.stdout: "osd-slow-ops": { 2024-04-25T11:38:09.751 INFO:teuthology.orchestra.run.smithi152.stdout: "slow_ops_count": 0 2024-04-25T11:38:09.751 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.751 INFO:teuthology.orchestra.run.smithi152.stdout: "purge_queue": { 2024-04-25T11:38:09.751 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_ops": 0, 2024-04-25T11:38:09.752 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T11:38:09.752 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing": 0, 2024-04-25T11:38:09.752 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_high_water": 0, 2024-04-25T11:38:09.752 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executed_ops": 0, 2024-04-25T11:38:09.752 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executed": 0, 2024-04-25T11:38:09.752 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_item_in_journal": 0 2024-04-25T11:38:09.752 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.752 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T11:38:09.752 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-04-25T11:38:09.752 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 104857600, 2024-04-25T11:38:09.752 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-04-25T11:38:09.752 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 2705, 2024-04-25T11:38:09.752 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 1288900, 2024-04-25T11:38:09.752 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-04-25T11:38:09.753 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 2705, 2024-04-25T11:38:09.753 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-04-25T11:38:09.753 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-04-25T11:38:09.753 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 2705, 2024-04-25T11:38:09.753 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 1288900, 2024-04-25T11:38:09.753 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-04-25T11:38:09.753 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:09.753 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:09.753 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:09.753 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T11:38:09.753 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.753 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-objecter_bytes": { 2024-04-25T11:38:09.753 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-04-25T11:38:09.753 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 104857600, 2024-04-25T11:38:09.753 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-04-25T11:38:09.754 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-04-25T11:38:09.754 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-04-25T11:38:09.754 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-04-25T11:38:09.754 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-04-25T11:38:09.754 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 2446, 2024-04-25T11:38:09.754 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 1618, 2024-04-25T11:38:09.754 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 2446, 2024-04-25T11:38:09.754 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 1618, 2024-04-25T11:38:09.754 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-04-25T11:38:09.754 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:09.754 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:09.754 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:09.754 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T11:38:09.754 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.755 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-objecter_ops": { 2024-04-25T11:38:09.755 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-04-25T11:38:09.755 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 1024, 2024-04-25T11:38:09.755 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-04-25T11:38:09.755 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-04-25T11:38:09.755 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-04-25T11:38:09.755 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-04-25T11:38:09.755 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-04-25T11:38:09.755 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 2446, 2024-04-25T11:38:09.755 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 2446, 2024-04-25T11:38:09.755 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 2446, 2024-04-25T11:38:09.755 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 2446, 2024-04-25T11:38:09.755 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-04-25T11:38:09.755 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:09.755 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:09.755 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:09.756 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T11:38:09.756 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.756 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-write_buf_throttle": { 2024-04-25T11:38:09.756 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-04-25T11:38:09.756 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 3758096384, 2024-04-25T11:38:09.756 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-04-25T11:38:09.756 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-04-25T11:38:09.756 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-04-25T11:38:09.756 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-04-25T11:38:09.756 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-04-25T11:38:09.756 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-04-25T11:38:09.756 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-04-25T11:38:09.756 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 0, 2024-04-25T11:38:09.756 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 0, 2024-04-25T11:38:09.756 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-04-25T11:38:09.756 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:09.757 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:09.757 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:09.757 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T11:38:09.757 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:09.757 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-write_buf_throttle-0x55d5770c8750": { 2024-04-25T11:38:09.757 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-04-25T11:38:09.757 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 3758096384, 2024-04-25T11:38:09.757 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-04-25T11:38:09.757 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-04-25T11:38:09.757 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-04-25T11:38:09.757 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-04-25T11:38:09.757 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-04-25T11:38:09.757 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-04-25T11:38:09.757 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-04-25T11:38:09.757 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 0, 2024-04-25T11:38:09.757 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 0, 2024-04-25T11:38:09.758 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-04-25T11:38:09.758 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:09.758 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:09.758 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:09.758 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T11:38:09.758 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T11:38:09.758 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-25T11:38:10.490 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-25T11:38:10.491 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-25T11:38:10.491 DEBUG:tasks.check_counter:Getting stats from i 2024-04-25T11:38:10.491 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-25T11:38:10.730 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T11:38:11.474 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:11 smithi152 ceph-mon[32701]: pgmap v723: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-25T11:38:11.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:11 smithi146 ceph-mon[32821]: pgmap v723: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-25T11:38:11.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:11 smithi069 ceph-mon[29188]: pgmap v723: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-25T11:38:12.494 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-25T11:38:12.494 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T11:38:12.494 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 36, 2024-04-25T11:38:12.494 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 38, 2024-04-25T11:38:12.495 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 7993, 2024-04-25T11:38:12.495 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 61021, 2024-04-25T11:38:12.495 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 12, 2024-04-25T11:38:12.495 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 4, 2024-04-25T11:38:12.495 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.007543448, 2024-04-25T11:38:12.495 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.000868048, 2024-04-25T11:38:12.495 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.004494550, 2024-04-25T11:38:12.495 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.000708189, 2024-04-25T11:38:12.495 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:38:12.495 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 36, 2024-04-25T11:38:12.495 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.017273497, 2024-04-25T11:38:12.495 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000479819 2024-04-25T11:38:12.495 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.495 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:38:12.495 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 26, 2024-04-25T11:38:12.495 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000005779, 2024-04-25T11:38:12.496 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000222 2024-04-25T11:38:12.496 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.496 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T11:38:12.496 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T11:38:12.496 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.496 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T11:38:12.496 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 312, 2024-04-25T11:38:12.496 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 298, 2024-04-25T11:38:12.496 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 872786, 2024-04-25T11:38:12.496 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 166702, 2024-04-25T11:38:12.496 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 6, 2024-04-25T11:38:12.496 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 5, 2024-04-25T11:38:12.496 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.049223099, 2024-04-25T11:38:12.497 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.017376892, 2024-04-25T11:38:12.497 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.029123099, 2024-04-25T11:38:12.497 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.006810652, 2024-04-25T11:38:12.497 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:38:12.497 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 298, 2024-04-25T11:38:12.497 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.032595126, 2024-04-25T11:38:12.497 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000109379 2024-04-25T11:38:12.497 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.497 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:38:12.497 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 7, 2024-04-25T11:38:12.497 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000001175, 2024-04-25T11:38:12.497 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000167 2024-04-25T11:38:12.497 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.498 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 868704, 2024-04-25T11:38:12.498 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 154144 2024-04-25T11:38:12.498 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.498 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T11:38:12.498 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 335, 2024-04-25T11:38:12.498 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 683, 2024-04-25T11:38:12.498 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 78652, 2024-04-25T11:38:12.498 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 611392, 2024-04-25T11:38:12.498 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 6, 2024-04-25T11:38:12.498 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 4, 2024-04-25T11:38:12.498 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.070248404, 2024-04-25T11:38:12.498 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.032263477, 2024-04-25T11:38:12.498 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.027816923, 2024-04-25T11:38:12.498 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.005840636, 2024-04-25T11:38:12.499 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:38:12.499 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 681, 2024-04-25T11:38:12.499 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.044590001, 2024-04-25T11:38:12.499 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000065477 2024-04-25T11:38:12.499 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.499 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:38:12.499 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 816, 2024-04-25T11:38:12.499 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000354870, 2024-04-25T11:38:12.499 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000434 2024-04-25T11:38:12.499 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.499 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T11:38:12.499 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 392656 2024-04-25T11:38:12.499 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.499 INFO:teuthology.orchestra.run.smithi152.stdout: "cct": { 2024-04-25T11:38:12.500 INFO:teuthology.orchestra.run.smithi152.stdout: "total_workers": 1, 2024-04-25T11:38:12.500 INFO:teuthology.orchestra.run.smithi152.stdout: "unhealthy_workers": 0 2024-04-25T11:38:12.500 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.500 INFO:teuthology.orchestra.run.smithi152.stdout: "finisher-MDSRank": { 2024-04-25T11:38:12.500 INFO:teuthology.orchestra.run.smithi152.stdout: "queue_len": 0, 2024-04-25T11:38:12.500 INFO:teuthology.orchestra.run.smithi152.stdout: "complete_latency": { 2024-04-25T11:38:12.500 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 43, 2024-04-25T11:38:12.500 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.001751413, 2024-04-25T11:38:12.500 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000040730 2024-04-25T11:38:12.500 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T11:38:12.500 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.500 INFO:teuthology.orchestra.run.smithi152.stdout: "finisher-PurgeQueue": { 2024-04-25T11:38:12.500 INFO:teuthology.orchestra.run.smithi152.stdout: "queue_len": 0, 2024-04-25T11:38:12.500 INFO:teuthology.orchestra.run.smithi152.stdout: "complete_latency": { 2024-04-25T11:38:12.500 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 2, 2024-04-25T11:38:12.501 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000009956, 2024-04-25T11:38:12.501 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000004978 2024-04-25T11:38:12.501 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T11:38:12.501 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.501 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": { 2024-04-25T11:38:12.501 INFO:teuthology.orchestra.run.smithi152.stdout: "request": 1, 2024-04-25T11:38:12.501 INFO:teuthology.orchestra.run.smithi152.stdout: "reply": 1, 2024-04-25T11:38:12.501 INFO:teuthology.orchestra.run.smithi152.stdout: "reply_latency": { 2024-04-25T11:38:12.501 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 1, 2024-04-25T11:38:12.501 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.001082117, 2024-04-25T11:38:12.501 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.001082117 2024-04-25T11:38:12.501 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.501 INFO:teuthology.orchestra.run.smithi152.stdout: "slow_reply": 0, 2024-04-25T11:38:12.501 INFO:teuthology.orchestra.run.smithi152.stdout: "forward": 0, 2024-04-25T11:38:12.501 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_fetch_complete": 0, 2024-04-25T11:38:12.501 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_fetch_keys": 0, 2024-04-25T11:38:12.502 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_commit": 11, 2024-04-25T11:38:12.502 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_split": 0, 2024-04-25T11:38:12.502 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_merge": 0, 2024-04-25T11:38:12.502 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes": 11, 2024-04-25T11:38:12.502 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_top": 0, 2024-04-25T11:38:12.502 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_bottom": 11, 2024-04-25T11:38:12.502 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_pin_tail": 0, 2024-04-25T11:38:12.502 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_pinned": 11, 2024-04-25T11:38:12.502 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_expired": 0, 2024-04-25T11:38:12.502 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_with_caps": 1, 2024-04-25T11:38:12.502 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": 1, 2024-04-25T11:38:12.502 INFO:teuthology.orchestra.run.smithi152.stdout: "subtrees": 2, 2024-04-25T11:38:12.502 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse": 2, 2024-04-25T11:38:12.502 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_hit": 1, 2024-04-25T11:38:12.502 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_forward": 0, 2024-04-25T11:38:12.503 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_discover": 1, 2024-04-25T11:38:12.503 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_dir_fetch": 0, 2024-04-25T11:38:12.503 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_remote_ino": 0, 2024-04-25T11:38:12.503 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_lock": 0, 2024-04-25T11:38:12.503 INFO:teuthology.orchestra.run.smithi152.stdout: "load_cent": 0, 2024-04-25T11:38:12.503 INFO:teuthology.orchestra.run.smithi152.stdout: "q": 0, 2024-04-25T11:38:12.503 INFO:teuthology.orchestra.run.smithi152.stdout: "exported": 0, 2024-04-25T11:38:12.503 INFO:teuthology.orchestra.run.smithi152.stdout: "exported_inodes": 0, 2024-04-25T11:38:12.503 INFO:teuthology.orchestra.run.smithi152.stdout: "imported": 0, 2024-04-25T11:38:12.503 INFO:teuthology.orchestra.run.smithi152.stdout: "imported_inodes": 0, 2024-04-25T11:38:12.503 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_dir_fetch": 0, 2024-04-25T11:38:12.503 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_backtrace_fetch": 0, 2024-04-25T11:38:12.503 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_peer_discover": 0, 2024-04-25T11:38:12.503 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rfiles": 5, 2024-04-25T11:38:12.503 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rbytes": 298, 2024-04-25T11:38:12.503 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rsnaps": 0, 2024-04-25T11:38:12.504 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T11:38:12.504 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_set_tag": 0, 2024-04-25T11:38:12.504 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T11:38:12.504 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_inotable_repaired": 0, 2024-04-25T11:38:12.504 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dir_inodes": 0, 2024-04-25T11:38:12.504 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T11:38:12.504 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T11:38:12.504 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_file_inodes": 0, 2024-04-25T11:38:12.504 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_inode_file_caps": 0, 2024-04-25T11:38:12.504 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T11:38:12.504 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_grant": 0, 2024-04-25T11:38:12.504 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T11:38:12.504 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T11:38:12.504 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T11:38:12.504 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_caps": 0, 2024-04-25T11:38:12.504 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_caps_dirty": 0, 2024-04-25T11:38:12.505 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_cap_release": 0, 2024-04-25T11:38:12.505 INFO:teuthology.orchestra.run.smithi152.stdout: "process_request_cap_release": 0 2024-04-25T11:38:12.505 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.505 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_cache": { 2024-04-25T11:38:12.505 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_update": 0, 2024-04-25T11:38:12.505 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_update_receipt": 0, 2024-04-25T11:38:12.505 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_try_discover": 0, 2024-04-25T11:38:12.505 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_send_discover": 2, 2024-04-25T11:38:12.505 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_handle_discover": 0, 2024-04-25T11:38:12.505 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays": 0, 2024-04-25T11:38:12.505 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays_delayed": 0, 2024-04-25T11:38:12.505 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays_enqueuing": 0, 2024-04-25T11:38:12.505 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_created": 0, 2024-04-25T11:38:12.505 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_enqueued": 0, 2024-04-25T11:38:12.505 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_reintegrated": 0, 2024-04-25T11:38:12.505 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_migrated": 0, 2024-04-25T11:38:12.506 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_processing": 0, 2024-04-25T11:38:12.506 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_enqueued": 0, 2024-04-25T11:38:12.506 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_prioritized": 0, 2024-04-25T11:38:12.506 INFO:teuthology.orchestra.run.smithi152.stdout: "recovery_started": 0, 2024-04-25T11:38:12.506 INFO:teuthology.orchestra.run.smithi152.stdout: "recovery_completed": 0, 2024-04-25T11:38:12.506 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_quiesce_path": 0, 2024-04-25T11:38:12.506 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_quiesce_inode": 0, 2024-04-25T11:38:12.506 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T11:38:12.506 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_exportdir": 0, 2024-04-25T11:38:12.506 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_flush": 0, 2024-04-25T11:38:12.506 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_fragmentdir": 0, 2024-04-25T11:38:12.506 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_fragstats": 0, 2024-04-25T11:38:12.506 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_inodestats": 0 2024-04-25T11:38:12.506 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.507 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_log": { 2024-04-25T11:38:12.507 INFO:teuthology.orchestra.run.smithi152.stdout: "evlrg": 0, 2024-04-25T11:38:12.507 INFO:teuthology.orchestra.run.smithi152.stdout: "evadd": 5, 2024-04-25T11:38:12.507 INFO:teuthology.orchestra.run.smithi152.stdout: "evex": 0, 2024-04-25T11:38:12.507 INFO:teuthology.orchestra.run.smithi152.stdout: "evtrm": 0, 2024-04-25T11:38:12.507 INFO:teuthology.orchestra.run.smithi152.stdout: "ev": 5, 2024-04-25T11:38:12.507 INFO:teuthology.orchestra.run.smithi152.stdout: "evexg": 0, 2024-04-25T11:38:12.507 INFO:teuthology.orchestra.run.smithi152.stdout: "evexd": 0, 2024-04-25T11:38:12.507 INFO:teuthology.orchestra.run.smithi152.stdout: "segadd": 2, 2024-04-25T11:38:12.507 INFO:teuthology.orchestra.run.smithi152.stdout: "segex": 0, 2024-04-25T11:38:12.507 INFO:teuthology.orchestra.run.smithi152.stdout: "segtrm": 0, 2024-04-25T11:38:12.507 INFO:teuthology.orchestra.run.smithi152.stdout: "seg": 2, 2024-04-25T11:38:12.507 INFO:teuthology.orchestra.run.smithi152.stdout: "segmjr": 2, 2024-04-25T11:38:12.507 INFO:teuthology.orchestra.run.smithi152.stdout: "segexg": 0, 2024-04-25T11:38:12.508 INFO:teuthology.orchestra.run.smithi152.stdout: "segexd": 0, 2024-04-25T11:38:12.508 INFO:teuthology.orchestra.run.smithi152.stdout: "expos": 4194304, 2024-04-25T11:38:12.508 INFO:teuthology.orchestra.run.smithi152.stdout: "wrpos": 4195917, 2024-04-25T11:38:12.508 INFO:teuthology.orchestra.run.smithi152.stdout: "rdpos": 0, 2024-04-25T11:38:12.508 INFO:teuthology.orchestra.run.smithi152.stdout: "jlat": { 2024-04-25T11:38:12.508 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 4, 2024-04-25T11:38:12.508 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.008526631, 2024-04-25T11:38:12.508 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.002131657 2024-04-25T11:38:12.508 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.508 INFO:teuthology.orchestra.run.smithi152.stdout: "replayed": 0 2024-04-25T11:38:12.508 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.508 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_mem": { 2024-04-25T11:38:12.508 INFO:teuthology.orchestra.run.smithi152.stdout: "ino": 14, 2024-04-25T11:38:12.508 INFO:teuthology.orchestra.run.smithi152.stdout: "ino+": 14, 2024-04-25T11:38:12.509 INFO:teuthology.orchestra.run.smithi152.stdout: "ino-": 0, 2024-04-25T11:38:12.509 INFO:teuthology.orchestra.run.smithi152.stdout: "dir": 12, 2024-04-25T11:38:12.509 INFO:teuthology.orchestra.run.smithi152.stdout: "dir+": 13, 2024-04-25T11:38:12.509 INFO:teuthology.orchestra.run.smithi152.stdout: "dir-": 1, 2024-04-25T11:38:12.509 INFO:teuthology.orchestra.run.smithi152.stdout: "dn": 11, 2024-04-25T11:38:12.509 INFO:teuthology.orchestra.run.smithi152.stdout: "dn+": 11, 2024-04-25T11:38:12.509 INFO:teuthology.orchestra.run.smithi152.stdout: "dn-": 0, 2024-04-25T11:38:12.509 INFO:teuthology.orchestra.run.smithi152.stdout: "cap": 1, 2024-04-25T11:38:12.509 INFO:teuthology.orchestra.run.smithi152.stdout: "cap+": 1, 2024-04-25T11:38:12.509 INFO:teuthology.orchestra.run.smithi152.stdout: "cap-": 0, 2024-04-25T11:38:12.509 INFO:teuthology.orchestra.run.smithi152.stdout: "rss": 120396, 2024-04-25T11:38:12.509 INFO:teuthology.orchestra.run.smithi152.stdout: "heap": 223632 2024-04-25T11:38:12.510 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.510 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_server": { 2024-04-25T11:38:12.510 INFO:teuthology.orchestra.run.smithi152.stdout: "dispatch_client_request": 2, 2024-04-25T11:38:12.510 INFO:teuthology.orchestra.run.smithi152.stdout: "dispatch_server_request": 0, 2024-04-25T11:38:12.510 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_request": 1, 2024-04-25T11:38:12.510 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_session": 14, 2024-04-25T11:38:12.510 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_peer_request": 0, 2024-04-25T11:38:12.510 INFO:teuthology.orchestra.run.smithi152.stdout: "req_create_latency": { 2024-04-25T11:38:12.510 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:12.510 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:12.510 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:12.510 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.510 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getattr_latency": { 2024-04-25T11:38:12.510 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 1, 2024-04-25T11:38:12.510 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.001082117, 2024-04-25T11:38:12.510 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.001082117 2024-04-25T11:38:12.511 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.511 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getfilelock_latency": { 2024-04-25T11:38:12.511 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:12.511 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:12.511 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:12.511 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.511 INFO:teuthology.orchestra.run.smithi152.stdout: "req_link_latency": { 2024-04-25T11:38:12.511 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:12.511 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:12.511 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:12.511 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.511 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookup_latency": { 2024-04-25T11:38:12.511 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:12.511 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:12.511 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:12.511 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.512 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookuphash_latency": { 2024-04-25T11:38:12.512 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:12.512 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:12.512 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:12.512 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.512 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupino_latency": { 2024-04-25T11:38:12.512 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:12.512 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:12.512 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:12.512 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.512 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupname_latency": { 2024-04-25T11:38:12.512 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:12.512 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:12.512 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:12.512 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.512 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupparent_latency": { 2024-04-25T11:38:12.513 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:12.513 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:12.513 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:12.513 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.513 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupsnap_latency": { 2024-04-25T11:38:12.513 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:12.513 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:12.513 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:12.513 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.513 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lssnap_latency": { 2024-04-25T11:38:12.513 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:12.513 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:12.513 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:12.513 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.513 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mkdir_latency": { 2024-04-25T11:38:12.513 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:12.514 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:12.514 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:12.514 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.514 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mknod_latency": { 2024-04-25T11:38:12.514 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:12.514 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:12.515 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:12.515 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.515 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mksnap_latency": { 2024-04-25T11:38:12.515 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:12.515 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:12.515 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:12.515 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.515 INFO:teuthology.orchestra.run.smithi152.stdout: "req_open_latency": { 2024-04-25T11:38:12.515 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:12.515 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:12.515 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:12.515 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.515 INFO:teuthology.orchestra.run.smithi152.stdout: "req_readdir_latency": { 2024-04-25T11:38:12.516 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:12.516 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:12.516 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:12.516 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.516 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rename_latency": { 2024-04-25T11:38:12.516 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:12.516 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:12.516 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:12.516 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.516 INFO:teuthology.orchestra.run.smithi152.stdout: "req_renamesnap_latency": { 2024-04-25T11:38:12.516 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:12.516 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:12.516 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:12.516 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.516 INFO:teuthology.orchestra.run.smithi152.stdout: "req_snapdiff_latency": { 2024-04-25T11:38:12.516 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:12.517 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:12.517 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:12.517 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.517 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmdir_latency": { 2024-04-25T11:38:12.517 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:12.517 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:12.517 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:12.517 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.517 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmsnap_latency": { 2024-04-25T11:38:12.517 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:12.517 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:12.517 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:12.517 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.517 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmxattr_latency": { 2024-04-25T11:38:12.518 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:12.518 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:12.518 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:12.518 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.518 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setattr_latency": { 2024-04-25T11:38:12.518 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:12.518 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:12.518 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:12.518 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.518 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setdirlayout_latency": { 2024-04-25T11:38:12.518 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:12.518 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:12.518 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:12.518 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.518 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setfilelock_latency": { 2024-04-25T11:38:12.519 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:12.519 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:12.519 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:12.519 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.519 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setlayout_latency": { 2024-04-25T11:38:12.519 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:12.519 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:12.519 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:12.519 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.519 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setxattr_latency": { 2024-04-25T11:38:12.519 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:12.519 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:12.519 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:12.519 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.519 INFO:teuthology.orchestra.run.smithi152.stdout: "req_symlink_latency": { 2024-04-25T11:38:12.520 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:12.520 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:12.520 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:12.520 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.520 INFO:teuthology.orchestra.run.smithi152.stdout: "req_unlink_latency": { 2024-04-25T11:38:12.520 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:12.520 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:12.520 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:12.520 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.520 INFO:teuthology.orchestra.run.smithi152.stdout: "cap_revoke_eviction": 0, 2024-04-25T11:38:12.520 INFO:teuthology.orchestra.run.smithi152.stdout: "cap_acquisition_throttle": 0, 2024-04-25T11:38:12.520 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getvxattr_latency": { 2024-04-25T11:38:12.520 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:12.521 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:12.521 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:12.521 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T11:38:12.521 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.521 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_sessions": { 2024-04-25T11:38:12.521 INFO:teuthology.orchestra.run.smithi152.stdout: "session_count": 1, 2024-04-25T11:38:12.521 INFO:teuthology.orchestra.run.smithi152.stdout: "session_add": 2, 2024-04-25T11:38:12.521 INFO:teuthology.orchestra.run.smithi152.stdout: "session_remove": 1, 2024-04-25T11:38:12.521 INFO:teuthology.orchestra.run.smithi152.stdout: "sessions_open": 1, 2024-04-25T11:38:12.521 INFO:teuthology.orchestra.run.smithi152.stdout: "sessions_stale": 0, 2024-04-25T11:38:12.521 INFO:teuthology.orchestra.run.smithi152.stdout: "total_load": 1, 2024-04-25T11:38:12.521 INFO:teuthology.orchestra.run.smithi152.stdout: "average_load": 1, 2024-04-25T11:38:12.521 INFO:teuthology.orchestra.run.smithi152.stdout: "avg_session_uptime": 725, 2024-04-25T11:38:12.522 INFO:teuthology.orchestra.run.smithi152.stdout: "mdthresh_evicted": 0 2024-04-25T11:38:12.522 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.522 INFO:teuthology.orchestra.run.smithi152.stdout: "mempool": { 2024-04-25T11:38:12.522 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_bytes": 0, 2024-04-25T11:38:12.522 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_items": 0, 2024-04-25T11:38:12.522 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T11:38:12.522 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_items": 0, 2024-04-25T11:38:12.522 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T11:38:12.522 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_items": 0, 2024-04-25T11:38:12.522 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T11:38:12.522 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T11:38:12.522 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T11:38:12.522 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T11:38:12.522 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T11:38:12.523 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_items": 0, 2024-04-25T11:38:12.523 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T11:38:12.523 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T11:38:12.523 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_bytes": 0, 2024-04-25T11:38:12.523 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_items": 0, 2024-04-25T11:38:12.523 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_bytes": 0, 2024-04-25T11:38:12.523 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_items": 0, 2024-04-25T11:38:12.523 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T11:38:12.523 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T11:38:12.523 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T11:38:12.523 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T11:38:12.523 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T11:38:12.523 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_items": 0, 2024-04-25T11:38:12.524 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_bytes": 0, 2024-04-25T11:38:12.524 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_items": 0, 2024-04-25T11:38:12.524 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T11:38:12.524 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T11:38:12.524 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_bytes": 0, 2024-04-25T11:38:12.524 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_items": 0, 2024-04-25T11:38:12.524 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_bytes": 0, 2024-04-25T11:38:12.524 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_items": 0, 2024-04-25T11:38:12.524 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T11:38:12.524 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_items": 0, 2024-04-25T11:38:12.524 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T11:38:12.524 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_items": 0, 2024-04-25T11:38:12.524 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_bytes": 230438, 2024-04-25T11:38:12.524 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_items": 75, 2024-04-25T11:38:12.524 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_bytes": 0, 2024-04-25T11:38:12.524 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_items": 0, 2024-04-25T11:38:12.525 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_bytes": 0, 2024-04-25T11:38:12.525 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_items": 0, 2024-04-25T11:38:12.525 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_bytes": 0, 2024-04-25T11:38:12.525 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_items": 0, 2024-04-25T11:38:12.525 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_bytes": 0, 2024-04-25T11:38:12.525 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_items": 0, 2024-04-25T11:38:12.525 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_bytes": 9088, 2024-04-25T11:38:12.525 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_items": 175, 2024-04-25T11:38:12.525 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T11:38:12.525 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_items": 0, 2024-04-25T11:38:12.525 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_bytes": 0, 2024-04-25T11:38:12.525 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_items": 0, 2024-04-25T11:38:12.525 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_bytes": 55448, 2024-04-25T11:38:12.525 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_items": 215, 2024-04-25T11:38:12.525 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_bytes": 0, 2024-04-25T11:38:12.525 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_items": 0, 2024-04-25T11:38:12.526 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_bytes": 0, 2024-04-25T11:38:12.526 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_items": 0 2024-04-25T11:38:12.526 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.526 INFO:teuthology.orchestra.run.smithi152.stdout: "objecter": { 2024-04-25T11:38:12.526 INFO:teuthology.orchestra.run.smithi152.stdout: "op_active": 0, 2024-04-25T11:38:12.526 INFO:teuthology.orchestra.run.smithi152.stdout: "op_laggy": 0, 2024-04-25T11:38:12.526 INFO:teuthology.orchestra.run.smithi152.stdout: "op_send": 41, 2024-04-25T11:38:12.526 INFO:teuthology.orchestra.run.smithi152.stdout: "op_send_bytes": 12009, 2024-04-25T11:38:12.526 INFO:teuthology.orchestra.run.smithi152.stdout: "op_resend": 33, 2024-04-25T11:38:12.526 INFO:teuthology.orchestra.run.smithi152.stdout: "op_reply": 41, 2024-04-25T11:38:12.526 INFO:teuthology.orchestra.run.smithi152.stdout: "op_latency": { 2024-04-25T11:38:12.526 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 41, 2024-04-25T11:38:12.526 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.250996853, 2024-04-25T11:38:12.526 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.006121874 2024-04-25T11:38:12.527 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.527 INFO:teuthology.orchestra.run.smithi152.stdout: "op_inflight": 0, 2024-04-25T11:38:12.527 INFO:teuthology.orchestra.run.smithi152.stdout: "oplen_avg": { 2024-04-25T11:38:12.527 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 41, 2024-04-25T11:38:12.527 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 64 2024-04-25T11:38:12.527 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.527 INFO:teuthology.orchestra.run.smithi152.stdout: "op": 41, 2024-04-25T11:38:12.527 INFO:teuthology.orchestra.run.smithi152.stdout: "op_r": 0, 2024-04-25T11:38:12.527 INFO:teuthology.orchestra.run.smithi152.stdout: "op_w": 41, 2024-04-25T11:38:12.527 INFO:teuthology.orchestra.run.smithi152.stdout: "op_rmw": 0, 2024-04-25T11:38:12.527 INFO:teuthology.orchestra.run.smithi152.stdout: "op_pg": 0, 2024-04-25T11:38:12.527 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_stat": 0, 2024-04-25T11:38:12.527 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_create": 10, 2024-04-25T11:38:12.527 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_read": 0, 2024-04-25T11:38:12.527 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_write": 4, 2024-04-25T11:38:12.527 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_writefull": 7, 2024-04-25T11:38:12.528 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_writesame": 0, 2024-04-25T11:38:12.528 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_append": 0, 2024-04-25T11:38:12.528 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_zero": 0, 2024-04-25T11:38:12.528 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_truncate": 0, 2024-04-25T11:38:12.528 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_delete": 6, 2024-04-25T11:38:12.528 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_mapext": 0, 2024-04-25T11:38:12.528 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_sparse_read": 0, 2024-04-25T11:38:12.528 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_clonerange": 0, 2024-04-25T11:38:12.528 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_getxattr": 0, 2024-04-25T11:38:12.528 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_setxattr": 20, 2024-04-25T11:38:12.528 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_cmpxattr": 0, 2024-04-25T11:38:12.528 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_rmxattr": 0, 2024-04-25T11:38:12.528 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_resetxattrs": 0, 2024-04-25T11:38:12.528 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_call": 0, 2024-04-25T11:38:12.528 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_watch": 0, 2024-04-25T11:38:12.528 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_notify": 0, 2024-04-25T11:38:12.529 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T11:38:12.529 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_pgls": 0, 2024-04-25T11:38:12.529 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_pgls_filter": 0, 2024-04-25T11:38:12.529 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_other": 0, 2024-04-25T11:38:12.529 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_active": 0, 2024-04-25T11:38:12.529 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_send": 0, 2024-04-25T11:38:12.529 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_resend": 0, 2024-04-25T11:38:12.529 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_ping": 0, 2024-04-25T11:38:12.529 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_active": 0, 2024-04-25T11:38:12.529 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_send": 0, 2024-04-25T11:38:12.529 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_resend": 0, 2024-04-25T11:38:12.529 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_active": 0, 2024-04-25T11:38:12.529 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_send": 0, 2024-04-25T11:38:12.529 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_resend": 0, 2024-04-25T11:38:12.529 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_active": 0, 2024-04-25T11:38:12.530 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_send": 0, 2024-04-25T11:38:12.530 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_resend": 0, 2024-04-25T11:38:12.530 INFO:teuthology.orchestra.run.smithi152.stdout: "command_active": 0, 2024-04-25T11:38:12.530 INFO:teuthology.orchestra.run.smithi152.stdout: "command_send": 0, 2024-04-25T11:38:12.530 INFO:teuthology.orchestra.run.smithi152.stdout: "command_resend": 0, 2024-04-25T11:38:12.530 INFO:teuthology.orchestra.run.smithi152.stdout: "map_epoch": 84, 2024-04-25T11:38:12.530 INFO:teuthology.orchestra.run.smithi152.stdout: "map_full": 0, 2024-04-25T11:38:12.530 INFO:teuthology.orchestra.run.smithi152.stdout: "map_inc": 4, 2024-04-25T11:38:12.530 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_sessions": 8, 2024-04-25T11:38:12.530 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_session_open": 8, 2024-04-25T11:38:12.530 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_session_close": 0, 2024-04-25T11:38:12.530 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_laggy": 0, 2024-04-25T11:38:12.530 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_wr": 16, 2024-04-25T11:38:12.531 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_rd": 0, 2024-04-25T11:38:12.531 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_del": 1 2024-04-25T11:38:12.531 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.531 INFO:teuthology.orchestra.run.smithi152.stdout: "oft": { 2024-04-25T11:38:12.531 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_objs": 1, 2024-04-25T11:38:12.531 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_kv_pairs": 0, 2024-04-25T11:38:12.531 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_updates": 0, 2024-04-25T11:38:12.531 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_removes": 0 2024-04-25T11:38:12.531 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.531 INFO:teuthology.orchestra.run.smithi152.stdout: "osd-slow-ops": { 2024-04-25T11:38:12.531 INFO:teuthology.orchestra.run.smithi152.stdout: "slow_ops_count": 0 2024-04-25T11:38:12.531 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.531 INFO:teuthology.orchestra.run.smithi152.stdout: "purge_queue": { 2024-04-25T11:38:12.531 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_ops": 0, 2024-04-25T11:38:12.531 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T11:38:12.532 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing": 0, 2024-04-25T11:38:12.532 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_high_water": 0, 2024-04-25T11:38:12.532 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executed_ops": 0, 2024-04-25T11:38:12.532 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executed": 0, 2024-04-25T11:38:12.532 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_item_in_journal": 0 2024-04-25T11:38:12.532 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.532 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T11:38:12.532 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-04-25T11:38:12.532 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 104857600, 2024-04-25T11:38:12.532 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-04-25T11:38:12.532 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 679, 2024-04-25T11:38:12.532 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 880540, 2024-04-25T11:38:12.532 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-04-25T11:38:12.532 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 679, 2024-04-25T11:38:12.532 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-04-25T11:38:12.533 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-04-25T11:38:12.533 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 679, 2024-04-25T11:38:12.533 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 880540, 2024-04-25T11:38:12.533 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-04-25T11:38:12.533 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:12.533 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:12.533 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:12.533 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T11:38:12.533 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.533 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-objecter_bytes": { 2024-04-25T11:38:12.533 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-04-25T11:38:12.533 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 104857600, 2024-04-25T11:38:12.533 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-04-25T11:38:12.533 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-04-25T11:38:12.533 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-04-25T11:38:12.534 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-04-25T11:38:12.534 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-04-25T11:38:12.534 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 41, 2024-04-25T11:38:12.534 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 12009, 2024-04-25T11:38:12.534 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 41, 2024-04-25T11:38:12.534 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 12009, 2024-04-25T11:38:12.534 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-04-25T11:38:12.534 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:12.534 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:12.534 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:12.534 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T11:38:12.534 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.535 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-objecter_ops": { 2024-04-25T11:38:12.535 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-04-25T11:38:12.535 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 1024, 2024-04-25T11:38:12.535 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-04-25T11:38:12.535 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-04-25T11:38:12.535 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-04-25T11:38:12.535 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-04-25T11:38:12.535 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-04-25T11:38:12.535 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 41, 2024-04-25T11:38:12.535 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 41, 2024-04-25T11:38:12.535 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 41, 2024-04-25T11:38:12.535 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 41, 2024-04-25T11:38:12.536 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-04-25T11:38:12.536 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:12.536 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:12.536 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:12.536 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T11:38:12.536 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.536 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-write_buf_throttle": { 2024-04-25T11:38:12.536 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-04-25T11:38:12.536 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 3758096384, 2024-04-25T11:38:12.536 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-04-25T11:38:12.536 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-04-25T11:38:12.536 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-04-25T11:38:12.537 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-04-25T11:38:12.537 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-04-25T11:38:12.537 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-04-25T11:38:12.537 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-04-25T11:38:12.537 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 0, 2024-04-25T11:38:12.537 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 0, 2024-04-25T11:38:12.537 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-04-25T11:38:12.537 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:12.537 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:12.537 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:12.537 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T11:38:12.537 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:12.537 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-write_buf_throttle-0x560045a72680": { 2024-04-25T11:38:12.538 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-04-25T11:38:12.538 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 3758096384, 2024-04-25T11:38:12.538 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-04-25T11:38:12.538 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 5, 2024-04-25T11:38:12.538 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 1613, 2024-04-25T11:38:12.538 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-04-25T11:38:12.538 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 5, 2024-04-25T11:38:12.538 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-04-25T11:38:12.538 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-04-25T11:38:12.538 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 4, 2024-04-25T11:38:12.538 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 1613, 2024-04-25T11:38:12.539 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-04-25T11:38:12.539 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:12.539 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:12.539 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:12.539 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T11:38:12.539 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T11:38:12.539 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-25T11:38:13.307 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-04-25T11:38:13.308 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-04-25T11:38:13.308 DEBUG:tasks.check_counter:Getting stats from l 2024-04-25T11:38:13.308 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-25T11:38:13.549 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T11:38:13.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:13 smithi146 ceph-mon[32821]: pgmap v724: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:38:13.672 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:13 smithi152 ceph-mon[32701]: pgmap v724: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:38:13.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:13 smithi069 ceph-mon[29188]: pgmap v724: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:38:15.282 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-04-25T11:38:15.282 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T11:38:15.282 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 1699, 2024-04-25T11:38:15.282 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 1701, 2024-04-25T11:38:15.282 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 509810, 2024-04-25T11:38:15.282 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 540283, 2024-04-25T11:38:15.282 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 6, 2024-04-25T11:38:15.283 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 2, 2024-04-25T11:38:15.283 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.194620603, 2024-04-25T11:38:15.283 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.069243024, 2024-04-25T11:38:15.283 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.049254132, 2024-04-25T11:38:15.283 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.057737374, 2024-04-25T11:38:15.283 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:38:15.283 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 1699, 2024-04-25T11:38:15.283 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.074930638, 2024-04-25T11:38:15.283 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000044102 2024-04-25T11:38:15.283 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.283 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:38:15.283 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:15.283 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:15.283 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:15.284 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.284 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T11:38:15.284 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T11:38:15.284 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.284 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T11:38:15.284 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 766, 2024-04-25T11:38:15.284 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 951, 2024-04-25T11:38:15.284 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 189002, 2024-04-25T11:38:15.284 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 636874, 2024-04-25T11:38:15.284 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 5, 2024-04-25T11:38:15.284 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 4, 2024-04-25T11:38:15.284 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.093746685, 2024-04-25T11:38:15.284 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.038166401, 2024-04-25T11:38:15.284 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.018657734, 2024-04-25T11:38:15.285 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.027297347, 2024-04-25T11:38:15.285 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:38:15.285 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 949, 2024-04-25T11:38:15.285 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.053337676, 2024-04-25T11:38:15.285 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000056204 2024-04-25T11:38:15.285 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.285 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:38:15.285 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:15.285 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:15.285 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:15.285 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.285 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T11:38:15.285 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 393536 2024-04-25T11:38:15.285 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.285 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T11:38:15.286 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 268, 2024-04-25T11:38:15.286 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 257, 2024-04-25T11:38:15.286 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 857340, 2024-04-25T11:38:15.286 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 153102, 2024-04-25T11:38:15.286 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 3, 2024-04-25T11:38:15.286 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 2, 2024-04-25T11:38:15.286 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.049619270, 2024-04-25T11:38:15.286 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.017413750, 2024-04-25T11:38:15.286 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.029164445, 2024-04-25T11:38:15.286 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.006646364, 2024-04-25T11:38:15.286 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:38:15.286 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 257, 2024-04-25T11:38:15.286 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.017368904, 2024-04-25T11:38:15.286 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000067583 2024-04-25T11:38:15.287 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.287 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:38:15.287 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:15.287 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:15.287 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:15.287 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.287 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 857072, 2024-04-25T11:38:15.287 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 152784 2024-04-25T11:38:15.287 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.287 INFO:teuthology.orchestra.run.smithi152.stdout: "cct": { 2024-04-25T11:38:15.287 INFO:teuthology.orchestra.run.smithi152.stdout: "total_workers": 1, 2024-04-25T11:38:15.287 INFO:teuthology.orchestra.run.smithi152.stdout: "unhealthy_workers": 0 2024-04-25T11:38:15.287 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.287 INFO:teuthology.orchestra.run.smithi152.stdout: "finisher-MDSRank": { 2024-04-25T11:38:15.287 INFO:teuthology.orchestra.run.smithi152.stdout: "queue_len": 0, 2024-04-25T11:38:15.288 INFO:teuthology.orchestra.run.smithi152.stdout: "complete_latency": { 2024-04-25T11:38:15.288 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 4066, 2024-04-25T11:38:15.288 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.127138602, 2024-04-25T11:38:15.288 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000031268 2024-04-25T11:38:15.288 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T11:38:15.288 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.288 INFO:teuthology.orchestra.run.smithi152.stdout: "finisher-PurgeQueue": { 2024-04-25T11:38:15.288 INFO:teuthology.orchestra.run.smithi152.stdout: "queue_len": 0, 2024-04-25T11:38:15.288 INFO:teuthology.orchestra.run.smithi152.stdout: "complete_latency": { 2024-04-25T11:38:15.288 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:15.288 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:15.288 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:15.288 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T11:38:15.288 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.289 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": { 2024-04-25T11:38:15.289 INFO:teuthology.orchestra.run.smithi152.stdout: "request": 0, 2024-04-25T11:38:15.289 INFO:teuthology.orchestra.run.smithi152.stdout: "reply": 0, 2024-04-25T11:38:15.289 INFO:teuthology.orchestra.run.smithi152.stdout: "reply_latency": { 2024-04-25T11:38:15.289 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:15.289 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:15.289 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:15.289 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.289 INFO:teuthology.orchestra.run.smithi152.stdout: "slow_reply": 0, 2024-04-25T11:38:15.289 INFO:teuthology.orchestra.run.smithi152.stdout: "forward": 0, 2024-04-25T11:38:15.289 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_fetch_complete": 0, 2024-04-25T11:38:15.289 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_fetch_keys": 0, 2024-04-25T11:38:15.289 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_commit": 0, 2024-04-25T11:38:15.289 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_split": 0, 2024-04-25T11:38:15.290 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_merge": 0, 2024-04-25T11:38:15.290 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes": 0, 2024-04-25T11:38:15.290 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_top": 0, 2024-04-25T11:38:15.290 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_bottom": 0, 2024-04-25T11:38:15.290 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_pin_tail": 0, 2024-04-25T11:38:15.290 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_pinned": 0, 2024-04-25T11:38:15.290 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_expired": 0, 2024-04-25T11:38:15.290 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_with_caps": 0, 2024-04-25T11:38:15.290 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": 0, 2024-04-25T11:38:15.290 INFO:teuthology.orchestra.run.smithi152.stdout: "subtrees": 1, 2024-04-25T11:38:15.290 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse": 0, 2024-04-25T11:38:15.290 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_hit": 0, 2024-04-25T11:38:15.290 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_forward": 0, 2024-04-25T11:38:15.290 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_discover": 0, 2024-04-25T11:38:15.291 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_dir_fetch": 0, 2024-04-25T11:38:15.291 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_remote_ino": 0, 2024-04-25T11:38:15.291 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_lock": 0, 2024-04-25T11:38:15.291 INFO:teuthology.orchestra.run.smithi152.stdout: "load_cent": 0, 2024-04-25T11:38:15.291 INFO:teuthology.orchestra.run.smithi152.stdout: "q": 0, 2024-04-25T11:38:15.291 INFO:teuthology.orchestra.run.smithi152.stdout: "exported": 0, 2024-04-25T11:38:15.291 INFO:teuthology.orchestra.run.smithi152.stdout: "exported_inodes": 0, 2024-04-25T11:38:15.291 INFO:teuthology.orchestra.run.smithi152.stdout: "imported": 0, 2024-04-25T11:38:15.291 INFO:teuthology.orchestra.run.smithi152.stdout: "imported_inodes": 0, 2024-04-25T11:38:15.291 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_dir_fetch": 0, 2024-04-25T11:38:15.291 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_backtrace_fetch": 0, 2024-04-25T11:38:15.291 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_peer_discover": 0, 2024-04-25T11:38:15.291 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rfiles": 0, 2024-04-25T11:38:15.291 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rbytes": 0, 2024-04-25T11:38:15.292 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rsnaps": 0, 2024-04-25T11:38:15.292 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T11:38:15.292 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_set_tag": 0, 2024-04-25T11:38:15.292 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T11:38:15.292 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_inotable_repaired": 0, 2024-04-25T11:38:15.292 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dir_inodes": 0, 2024-04-25T11:38:15.292 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T11:38:15.292 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T11:38:15.292 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_file_inodes": 0, 2024-04-25T11:38:15.292 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_inode_file_caps": 0, 2024-04-25T11:38:15.292 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T11:38:15.292 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_grant": 0, 2024-04-25T11:38:15.292 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T11:38:15.292 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T11:38:15.292 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T11:38:15.293 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_caps": 0, 2024-04-25T11:38:15.293 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_caps_dirty": 0, 2024-04-25T11:38:15.293 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_cap_release": 0, 2024-04-25T11:38:15.293 INFO:teuthology.orchestra.run.smithi152.stdout: "process_request_cap_release": 0 2024-04-25T11:38:15.293 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.293 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_cache": { 2024-04-25T11:38:15.293 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_update": 0, 2024-04-25T11:38:15.293 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_update_receipt": 0, 2024-04-25T11:38:15.293 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_try_discover": 0, 2024-04-25T11:38:15.293 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_send_discover": 0, 2024-04-25T11:38:15.293 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_handle_discover": 0, 2024-04-25T11:38:15.293 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays": 0, 2024-04-25T11:38:15.293 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays_delayed": 0, 2024-04-25T11:38:15.293 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays_enqueuing": 0, 2024-04-25T11:38:15.294 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_created": 0, 2024-04-25T11:38:15.294 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_enqueued": 0, 2024-04-25T11:38:15.294 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_reintegrated": 0, 2024-04-25T11:38:15.294 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_migrated": 0, 2024-04-25T11:38:15.294 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_processing": 0, 2024-04-25T11:38:15.294 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_enqueued": 0, 2024-04-25T11:38:15.294 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_prioritized": 0, 2024-04-25T11:38:15.294 INFO:teuthology.orchestra.run.smithi152.stdout: "recovery_started": 0, 2024-04-25T11:38:15.294 INFO:teuthology.orchestra.run.smithi152.stdout: "recovery_completed": 0, 2024-04-25T11:38:15.294 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_quiesce_path": 0, 2024-04-25T11:38:15.294 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_quiesce_inode": 0, 2024-04-25T11:38:15.294 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T11:38:15.294 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_exportdir": 0, 2024-04-25T11:38:15.294 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_flush": 0, 2024-04-25T11:38:15.294 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_fragmentdir": 0, 2024-04-25T11:38:15.295 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_fragstats": 0, 2024-04-25T11:38:15.295 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_inodestats": 0 2024-04-25T11:38:15.295 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.295 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_log": { 2024-04-25T11:38:15.295 INFO:teuthology.orchestra.run.smithi152.stdout: "evlrg": 0, 2024-04-25T11:38:15.295 INFO:teuthology.orchestra.run.smithi152.stdout: "evadd": 0, 2024-04-25T11:38:15.295 INFO:teuthology.orchestra.run.smithi152.stdout: "evex": 0, 2024-04-25T11:38:15.295 INFO:teuthology.orchestra.run.smithi152.stdout: "evtrm": 0, 2024-04-25T11:38:15.295 INFO:teuthology.orchestra.run.smithi152.stdout: "ev": 7, 2024-04-25T11:38:15.295 INFO:teuthology.orchestra.run.smithi152.stdout: "evexg": 0, 2024-04-25T11:38:15.295 INFO:teuthology.orchestra.run.smithi152.stdout: "evexd": 0, 2024-04-25T11:38:15.295 INFO:teuthology.orchestra.run.smithi152.stdout: "segadd": 0, 2024-04-25T11:38:15.295 INFO:teuthology.orchestra.run.smithi152.stdout: "segex": 0, 2024-04-25T11:38:15.295 INFO:teuthology.orchestra.run.smithi152.stdout: "segtrm": 0, 2024-04-25T11:38:15.296 INFO:teuthology.orchestra.run.smithi152.stdout: "seg": 2, 2024-04-25T11:38:15.296 INFO:teuthology.orchestra.run.smithi152.stdout: "segmjr": 2, 2024-04-25T11:38:15.296 INFO:teuthology.orchestra.run.smithi152.stdout: "segexg": 0, 2024-04-25T11:38:15.296 INFO:teuthology.orchestra.run.smithi152.stdout: "segexd": 0, 2024-04-25T11:38:15.296 INFO:teuthology.orchestra.run.smithi152.stdout: "expos": 4194304, 2024-04-25T11:38:15.296 INFO:teuthology.orchestra.run.smithi152.stdout: "wrpos": 4196449, 2024-04-25T11:38:15.296 INFO:teuthology.orchestra.run.smithi152.stdout: "rdpos": 4196308, 2024-04-25T11:38:15.296 INFO:teuthology.orchestra.run.smithi152.stdout: "jlat": { 2024-04-25T11:38:15.297 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:15.297 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:15.297 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:15.297 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.297 INFO:teuthology.orchestra.run.smithi152.stdout: "replayed": 7 2024-04-25T11:38:15.297 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.297 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_mem": { 2024-04-25T11:38:15.297 INFO:teuthology.orchestra.run.smithi152.stdout: "ino": 3, 2024-04-25T11:38:15.297 INFO:teuthology.orchestra.run.smithi152.stdout: "ino+": 3, 2024-04-25T11:38:15.297 INFO:teuthology.orchestra.run.smithi152.stdout: "ino-": 0, 2024-04-25T11:38:15.297 INFO:teuthology.orchestra.run.smithi152.stdout: "dir": 1, 2024-04-25T11:38:15.297 INFO:teuthology.orchestra.run.smithi152.stdout: "dir+": 1, 2024-04-25T11:38:15.297 INFO:teuthology.orchestra.run.smithi152.stdout: "dir-": 0, 2024-04-25T11:38:15.298 INFO:teuthology.orchestra.run.smithi152.stdout: "dn": 0, 2024-04-25T11:38:15.298 INFO:teuthology.orchestra.run.smithi152.stdout: "dn+": 0, 2024-04-25T11:38:15.298 INFO:teuthology.orchestra.run.smithi152.stdout: "dn-": 0, 2024-04-25T11:38:15.298 INFO:teuthology.orchestra.run.smithi152.stdout: "cap": 0, 2024-04-25T11:38:15.298 INFO:teuthology.orchestra.run.smithi152.stdout: "cap+": 0, 2024-04-25T11:38:15.298 INFO:teuthology.orchestra.run.smithi152.stdout: "cap-": 0, 2024-04-25T11:38:15.298 INFO:teuthology.orchestra.run.smithi152.stdout: "rss": 118740, 2024-04-25T11:38:15.298 INFO:teuthology.orchestra.run.smithi152.stdout: "heap": 215440 2024-04-25T11:38:15.298 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.298 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_server": { 2024-04-25T11:38:15.298 INFO:teuthology.orchestra.run.smithi152.stdout: "dispatch_client_request": 0, 2024-04-25T11:38:15.298 INFO:teuthology.orchestra.run.smithi152.stdout: "dispatch_server_request": 0, 2024-04-25T11:38:15.298 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_request": 0, 2024-04-25T11:38:15.299 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_session": 0, 2024-04-25T11:38:15.299 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_peer_request": 0, 2024-04-25T11:38:15.299 INFO:teuthology.orchestra.run.smithi152.stdout: "req_create_latency": { 2024-04-25T11:38:15.299 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:15.299 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:15.299 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:15.299 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.299 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getattr_latency": { 2024-04-25T11:38:15.299 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:15.299 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:15.299 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:15.299 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.299 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getfilelock_latency": { 2024-04-25T11:38:15.299 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:15.300 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:15.300 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:15.300 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.300 INFO:teuthology.orchestra.run.smithi152.stdout: "req_link_latency": { 2024-04-25T11:38:15.300 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:15.300 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:15.300 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:15.300 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.300 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookup_latency": { 2024-04-25T11:38:15.300 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:15.300 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:15.300 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:15.300 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.300 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookuphash_latency": { 2024-04-25T11:38:15.301 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:15.301 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:15.301 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:15.301 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.301 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupino_latency": { 2024-04-25T11:38:15.301 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:15.301 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:15.301 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:15.301 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.301 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupname_latency": { 2024-04-25T11:38:15.301 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:15.301 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:15.301 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:15.301 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.302 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupparent_latency": { 2024-04-25T11:38:15.302 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:15.302 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:15.302 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:15.302 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.302 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupsnap_latency": { 2024-04-25T11:38:15.302 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:15.302 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:15.302 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:15.302 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.302 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lssnap_latency": { 2024-04-25T11:38:15.302 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:15.302 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:15.302 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:15.302 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.303 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mkdir_latency": { 2024-04-25T11:38:15.303 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:15.303 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:15.303 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:15.303 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.303 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mknod_latency": { 2024-04-25T11:38:15.304 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:15.304 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:15.304 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:15.304 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.304 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mksnap_latency": { 2024-04-25T11:38:15.304 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:15.304 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:15.304 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:15.304 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.304 INFO:teuthology.orchestra.run.smithi152.stdout: "req_open_latency": { 2024-04-25T11:38:15.304 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:15.305 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:15.305 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:15.305 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.305 INFO:teuthology.orchestra.run.smithi152.stdout: "req_readdir_latency": { 2024-04-25T11:38:15.305 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:15.305 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:15.305 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:15.305 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.305 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rename_latency": { 2024-04-25T11:38:15.305 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:15.305 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:15.305 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:15.305 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.305 INFO:teuthology.orchestra.run.smithi152.stdout: "req_renamesnap_latency": { 2024-04-25T11:38:15.306 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:15.306 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:15.306 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:15.306 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.306 INFO:teuthology.orchestra.run.smithi152.stdout: "req_snapdiff_latency": { 2024-04-25T11:38:15.306 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:15.306 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:15.306 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:15.306 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.306 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmdir_latency": { 2024-04-25T11:38:15.306 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:15.306 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:15.306 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:15.306 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.307 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmsnap_latency": { 2024-04-25T11:38:15.307 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:15.307 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:15.307 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:15.307 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.307 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmxattr_latency": { 2024-04-25T11:38:15.307 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:15.307 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:15.307 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:15.307 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.307 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setattr_latency": { 2024-04-25T11:38:15.307 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:15.307 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:15.308 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:15.308 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.308 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setdirlayout_latency": { 2024-04-25T11:38:15.308 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:15.308 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:15.308 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:15.308 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.308 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setfilelock_latency": { 2024-04-25T11:38:15.308 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:15.308 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:15.308 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:15.308 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.308 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setlayout_latency": { 2024-04-25T11:38:15.308 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:15.308 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:15.309 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:15.309 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.309 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setxattr_latency": { 2024-04-25T11:38:15.309 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:15.309 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:15.309 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:15.309 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.309 INFO:teuthology.orchestra.run.smithi152.stdout: "req_symlink_latency": { 2024-04-25T11:38:15.309 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:15.309 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:15.309 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:15.309 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.309 INFO:teuthology.orchestra.run.smithi152.stdout: "req_unlink_latency": { 2024-04-25T11:38:15.309 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:15.309 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:15.310 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:15.310 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.310 INFO:teuthology.orchestra.run.smithi152.stdout: "cap_revoke_eviction": 0, 2024-04-25T11:38:15.310 INFO:teuthology.orchestra.run.smithi152.stdout: "cap_acquisition_throttle": 0, 2024-04-25T11:38:15.310 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getvxattr_latency": { 2024-04-25T11:38:15.310 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:15.310 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:15.310 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:15.310 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T11:38:15.310 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.310 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_sessions": { 2024-04-25T11:38:15.310 INFO:teuthology.orchestra.run.smithi152.stdout: "session_count": 1, 2024-04-25T11:38:15.310 INFO:teuthology.orchestra.run.smithi152.stdout: "session_add": 3, 2024-04-25T11:38:15.310 INFO:teuthology.orchestra.run.smithi152.stdout: "session_remove": 2, 2024-04-25T11:38:15.310 INFO:teuthology.orchestra.run.smithi152.stdout: "sessions_open": 2, 2024-04-25T11:38:15.311 INFO:teuthology.orchestra.run.smithi152.stdout: "sessions_stale": 0, 2024-04-25T11:38:15.311 INFO:teuthology.orchestra.run.smithi152.stdout: "total_load": 0, 2024-04-25T11:38:15.311 INFO:teuthology.orchestra.run.smithi152.stdout: "average_load": 0, 2024-04-25T11:38:15.311 INFO:teuthology.orchestra.run.smithi152.stdout: "avg_session_uptime": 2979, 2024-04-25T11:38:15.311 INFO:teuthology.orchestra.run.smithi152.stdout: "mdthresh_evicted": 0 2024-04-25T11:38:15.311 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.311 INFO:teuthology.orchestra.run.smithi152.stdout: "mempool": { 2024-04-25T11:38:15.311 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_bytes": 0, 2024-04-25T11:38:15.311 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_items": 0, 2024-04-25T11:38:15.311 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T11:38:15.311 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_items": 0, 2024-04-25T11:38:15.311 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T11:38:15.311 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_items": 0, 2024-04-25T11:38:15.311 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T11:38:15.312 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T11:38:15.312 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T11:38:15.312 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T11:38:15.312 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T11:38:15.312 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_items": 0, 2024-04-25T11:38:15.312 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T11:38:15.312 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T11:38:15.312 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_bytes": 0, 2024-04-25T11:38:15.312 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_items": 0, 2024-04-25T11:38:15.312 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_bytes": 0, 2024-04-25T11:38:15.312 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_items": 0, 2024-04-25T11:38:15.312 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T11:38:15.312 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T11:38:15.312 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T11:38:15.312 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T11:38:15.312 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T11:38:15.313 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_items": 0, 2024-04-25T11:38:15.313 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_bytes": 0, 2024-04-25T11:38:15.313 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_items": 0, 2024-04-25T11:38:15.313 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T11:38:15.313 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T11:38:15.313 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_bytes": 0, 2024-04-25T11:38:15.313 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_items": 0, 2024-04-25T11:38:15.313 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_bytes": 0, 2024-04-25T11:38:15.313 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_items": 0, 2024-04-25T11:38:15.313 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T11:38:15.313 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_items": 0, 2024-04-25T11:38:15.313 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T11:38:15.313 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_items": 0, 2024-04-25T11:38:15.313 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_bytes": 189423, 2024-04-25T11:38:15.313 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_items": 51, 2024-04-25T11:38:15.314 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_bytes": 0, 2024-04-25T11:38:15.314 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_items": 0, 2024-04-25T11:38:15.314 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_bytes": 0, 2024-04-25T11:38:15.314 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_items": 0, 2024-04-25T11:38:15.314 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_bytes": 0, 2024-04-25T11:38:15.314 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_items": 0, 2024-04-25T11:38:15.314 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_bytes": 0, 2024-04-25T11:38:15.314 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_items": 0, 2024-04-25T11:38:15.314 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_bytes": 9088, 2024-04-25T11:38:15.314 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_items": 175, 2024-04-25T11:38:15.314 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T11:38:15.314 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_items": 0, 2024-04-25T11:38:15.314 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_bytes": 0, 2024-04-25T11:38:15.314 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_items": 0, 2024-04-25T11:38:15.314 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_bytes": 8512, 2024-04-25T11:38:15.315 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_items": 13, 2024-04-25T11:38:15.315 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_bytes": 0, 2024-04-25T11:38:15.315 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_items": 0, 2024-04-25T11:38:15.315 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_bytes": 0, 2024-04-25T11:38:15.315 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_items": 0 2024-04-25T11:38:15.315 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.315 INFO:teuthology.orchestra.run.smithi152.stdout: "objecter": { 2024-04-25T11:38:15.315 INFO:teuthology.orchestra.run.smithi152.stdout: "op_active": 0, 2024-04-25T11:38:15.315 INFO:teuthology.orchestra.run.smithi152.stdout: "op_laggy": 0, 2024-04-25T11:38:15.315 INFO:teuthology.orchestra.run.smithi152.stdout: "op_send": 2465, 2024-04-25T11:38:15.315 INFO:teuthology.orchestra.run.smithi152.stdout: "op_send_bytes": 21, 2024-04-25T11:38:15.315 INFO:teuthology.orchestra.run.smithi152.stdout: "op_resend": 3, 2024-04-25T11:38:15.315 INFO:teuthology.orchestra.run.smithi152.stdout: "op_reply": 2465, 2024-04-25T11:38:15.315 INFO:teuthology.orchestra.run.smithi152.stdout: "op_latency": { 2024-04-25T11:38:15.315 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 2465, 2024-04-25T11:38:15.316 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 1.200984336, 2024-04-25T11:38:15.316 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000487214 2024-04-25T11:38:15.316 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.316 INFO:teuthology.orchestra.run.smithi152.stdout: "op_inflight": 0, 2024-04-25T11:38:15.316 INFO:teuthology.orchestra.run.smithi152.stdout: "oplen_avg": { 2024-04-25T11:38:15.316 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 2465, 2024-04-25T11:38:15.316 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 2466 2024-04-25T11:38:15.316 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.316 INFO:teuthology.orchestra.run.smithi152.stdout: "op": 2465, 2024-04-25T11:38:15.316 INFO:teuthology.orchestra.run.smithi152.stdout: "op_r": 2465, 2024-04-25T11:38:15.316 INFO:teuthology.orchestra.run.smithi152.stdout: "op_w": 0, 2024-04-25T11:38:15.316 INFO:teuthology.orchestra.run.smithi152.stdout: "op_rmw": 0, 2024-04-25T11:38:15.316 INFO:teuthology.orchestra.run.smithi152.stdout: "op_pg": 0, 2024-04-25T11:38:15.316 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_stat": 1608, 2024-04-25T11:38:15.316 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_create": 0, 2024-04-25T11:38:15.316 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_read": 855, 2024-04-25T11:38:15.317 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_write": 0, 2024-04-25T11:38:15.317 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_writefull": 0, 2024-04-25T11:38:15.317 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_writesame": 0, 2024-04-25T11:38:15.317 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_append": 0, 2024-04-25T11:38:15.317 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_zero": 0, 2024-04-25T11:38:15.317 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_truncate": 0, 2024-04-25T11:38:15.317 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_delete": 0, 2024-04-25T11:38:15.317 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_mapext": 0, 2024-04-25T11:38:15.317 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_sparse_read": 0, 2024-04-25T11:38:15.317 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_clonerange": 0, 2024-04-25T11:38:15.317 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_getxattr": 1, 2024-04-25T11:38:15.317 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_setxattr": 0, 2024-04-25T11:38:15.317 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_cmpxattr": 0, 2024-04-25T11:38:15.317 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_rmxattr": 0, 2024-04-25T11:38:15.317 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_resetxattrs": 0, 2024-04-25T11:38:15.317 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_call": 0, 2024-04-25T11:38:15.318 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_watch": 0, 2024-04-25T11:38:15.318 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_notify": 0, 2024-04-25T11:38:15.318 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T11:38:15.318 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_pgls": 0, 2024-04-25T11:38:15.318 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_pgls_filter": 0, 2024-04-25T11:38:15.318 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_other": 0, 2024-04-25T11:38:15.318 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_active": 0, 2024-04-25T11:38:15.318 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_send": 0, 2024-04-25T11:38:15.318 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_resend": 0, 2024-04-25T11:38:15.318 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_ping": 0, 2024-04-25T11:38:15.318 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_active": 0, 2024-04-25T11:38:15.318 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_send": 0, 2024-04-25T11:38:15.318 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_resend": 0, 2024-04-25T11:38:15.318 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_active": 0, 2024-04-25T11:38:15.319 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_send": 0, 2024-04-25T11:38:15.319 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_resend": 0, 2024-04-25T11:38:15.319 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_active": 0, 2024-04-25T11:38:15.319 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_send": 0, 2024-04-25T11:38:15.319 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_resend": 0, 2024-04-25T11:38:15.319 INFO:teuthology.orchestra.run.smithi152.stdout: "command_active": 0, 2024-04-25T11:38:15.319 INFO:teuthology.orchestra.run.smithi152.stdout: "command_send": 0, 2024-04-25T11:38:15.319 INFO:teuthology.orchestra.run.smithi152.stdout: "command_resend": 0, 2024-04-25T11:38:15.319 INFO:teuthology.orchestra.run.smithi152.stdout: "map_epoch": 84, 2024-04-25T11:38:15.319 INFO:teuthology.orchestra.run.smithi152.stdout: "map_full": 0, 2024-04-25T11:38:15.319 INFO:teuthology.orchestra.run.smithi152.stdout: "map_inc": 4, 2024-04-25T11:38:15.319 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_sessions": 5, 2024-04-25T11:38:15.319 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_session_open": 5, 2024-04-25T11:38:15.320 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_session_close": 0, 2024-04-25T11:38:15.320 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_laggy": 0, 2024-04-25T11:38:15.320 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_wr": 0, 2024-04-25T11:38:15.320 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_rd": 2, 2024-04-25T11:38:15.320 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_del": 0 2024-04-25T11:38:15.320 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.320 INFO:teuthology.orchestra.run.smithi152.stdout: "oft": { 2024-04-25T11:38:15.320 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_objs": 0, 2024-04-25T11:38:15.320 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_kv_pairs": 0, 2024-04-25T11:38:15.320 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_updates": 0, 2024-04-25T11:38:15.320 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_removes": 0 2024-04-25T11:38:15.320 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.320 INFO:teuthology.orchestra.run.smithi152.stdout: "osd-slow-ops": { 2024-04-25T11:38:15.320 INFO:teuthology.orchestra.run.smithi152.stdout: "slow_ops_count": 0 2024-04-25T11:38:15.321 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.321 INFO:teuthology.orchestra.run.smithi152.stdout: "purge_queue": { 2024-04-25T11:38:15.321 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_ops": 0, 2024-04-25T11:38:15.321 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T11:38:15.321 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing": 0, 2024-04-25T11:38:15.321 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_high_water": 0, 2024-04-25T11:38:15.321 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executed_ops": 0, 2024-04-25T11:38:15.321 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executed": 0, 2024-04-25T11:38:15.321 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_item_in_journal": 0 2024-04-25T11:38:15.321 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.321 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T11:38:15.321 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-04-25T11:38:15.321 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 104857600, 2024-04-25T11:38:15.321 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-04-25T11:38:15.321 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 2729, 2024-04-25T11:38:15.322 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 1294105, 2024-04-25T11:38:15.322 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-04-25T11:38:15.322 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 2729, 2024-04-25T11:38:15.322 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-04-25T11:38:15.322 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-04-25T11:38:15.322 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 2729, 2024-04-25T11:38:15.322 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 1294105, 2024-04-25T11:38:15.322 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-04-25T11:38:15.322 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:15.322 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:15.322 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:15.322 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T11:38:15.322 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.322 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-objecter_bytes": { 2024-04-25T11:38:15.322 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-04-25T11:38:15.323 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 104857600, 2024-04-25T11:38:15.323 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-04-25T11:38:15.323 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-04-25T11:38:15.323 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-04-25T11:38:15.323 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-04-25T11:38:15.323 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-04-25T11:38:15.323 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 2465, 2024-04-25T11:38:15.323 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 2150, 2024-04-25T11:38:15.323 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 2465, 2024-04-25T11:38:15.323 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 2150, 2024-04-25T11:38:15.323 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-04-25T11:38:15.323 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:15.323 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:15.323 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:15.323 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T11:38:15.324 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.324 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-objecter_ops": { 2024-04-25T11:38:15.324 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-04-25T11:38:15.324 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 1024, 2024-04-25T11:38:15.324 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-04-25T11:38:15.324 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-04-25T11:38:15.324 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-04-25T11:38:15.324 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-04-25T11:38:15.324 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-04-25T11:38:15.324 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 2465, 2024-04-25T11:38:15.324 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 2465, 2024-04-25T11:38:15.324 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 2465, 2024-04-25T11:38:15.324 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 2465, 2024-04-25T11:38:15.324 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-04-25T11:38:15.324 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:15.324 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:15.325 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:15.325 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T11:38:15.325 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.325 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-write_buf_throttle": { 2024-04-25T11:38:15.325 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-04-25T11:38:15.325 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 3758096384, 2024-04-25T11:38:15.325 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-04-25T11:38:15.325 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-04-25T11:38:15.325 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-04-25T11:38:15.325 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-04-25T11:38:15.325 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-04-25T11:38:15.325 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-04-25T11:38:15.325 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-04-25T11:38:15.325 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 0, 2024-04-25T11:38:15.326 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 0, 2024-04-25T11:38:15.326 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-04-25T11:38:15.326 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:15.326 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:15.326 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:15.326 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T11:38:15.326 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-04-25T11:38:15.326 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-write_buf_throttle-0x55c8384148f0": { 2024-04-25T11:38:15.326 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-04-25T11:38:15.326 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 3758096384, 2024-04-25T11:38:15.326 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-04-25T11:38:15.326 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-04-25T11:38:15.326 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-04-25T11:38:15.326 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-04-25T11:38:15.327 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-04-25T11:38:15.327 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-04-25T11:38:15.327 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-04-25T11:38:15.327 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 0, 2024-04-25T11:38:15.327 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 0, 2024-04-25T11:38:15.327 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-04-25T11:38:15.327 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-04-25T11:38:15.327 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-04-25T11:38:15.327 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-04-25T11:38:15.327 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T11:38:15.327 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-04-25T11:38:15.327 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-04-25T11:38:15.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:15 smithi146 ceph-mon[32821]: pgmap v725: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T11:38:15.681 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:15 smithi152 ceph-mon[32701]: pgmap v725: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T11:38:15.686 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:15 smithi069 ceph-mon[29188]: pgmap v725: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-25T11:38:16.027 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-04-25T11:38:16.027 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-04-25T11:38:16.028 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-25T11:38:16.038 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-25T11:38:16.038 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T11:38:16.038 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:38:16.060 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T11:38:16.060 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:38:16.119 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:38:16.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.294+0000 7fd7da6bd640 1 -- 172.21.15.69:0/3088914243 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7d4104450 msgr2=0x7fd7d410ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:16.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.294+0000 7fd7da6bd640 1 --2- 172.21.15.69:0/3088914243 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7d4104450 0x7fd7d410ab90 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc009ca0 tx=0x7fd7cc02a8c0 comp rx=0 tx=0).stop 2024-04-25T11:38:16.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.294+0000 7fd7da6bd640 1 -- 172.21.15.69:0/3088914243 shutdown_connections 2024-04-25T11:38:16.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.294+0000 7fd7da6bd640 1 --2- 172.21.15.69:0/3088914243 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7d4104450 0x7fd7d410ab90 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:16.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.294+0000 7fd7da6bd640 1 --2- 172.21.15.69:0/3088914243 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd7d4103910 0x7fd7d4103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:16.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.294+0000 7fd7da6bd640 1 --2- 172.21.15.69:0/3088914243 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd7d4102f30 0x7fd7d4103340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:16.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.294+0000 7fd7da6bd640 1 -- 172.21.15.69:0/3088914243 >> 172.21.15.69:0/3088914243 conn(0x7fd7d40fd330 msgr2=0x7fd7d40ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:16.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.294+0000 7fd7da6bd640 1 -- 172.21.15.69:0/3088914243 shutdown_connections 2024-04-25T11:38:16.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.294+0000 7fd7da6bd640 1 -- 172.21.15.69:0/3088914243 wait complete. 2024-04-25T11:38:16.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.294+0000 7fd7da6bd640 1 Processor -- start 2024-04-25T11:38:16.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.294+0000 7fd7da6bd640 1 -- start start 2024-04-25T11:38:16.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.295+0000 7fd7da6bd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7d4102f30 0x7fd7d4198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:16.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.295+0000 7fd7da6bd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd7d4103910 0x7fd7d4199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:16.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.295+0000 7fd7da6bd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd7d4104450 0x7fd7d41a04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:16.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.295+0000 7fd7da6bd640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d410e400 con 0x7fd7d4102f30 2024-04-25T11:38:16.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.295+0000 7fd7da6bd640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d410e280 con 0x7fd7d4104450 2024-04-25T11:38:16.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.295+0000 7fd7da6bd640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d410e580 con 0x7fd7d4103910 2024-04-25T11:38:16.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.295+0000 7fd7d3fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7d4102f30 0x7fd7d4198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:16.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.295+0000 7fd7d3fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7d4102f30 0x7fd7d4198fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42018/0 (socket says 172.21.15.69:42018) 2024-04-25T11:38:16.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.295+0000 7fd7d3fff640 1 -- 172.21.15.69:0/2703228665 learned_addr learned my addr 172.21.15.69:0/2703228665 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:38:16.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.295+0000 7fd7d8c5a640 1 --2- 172.21.15.69:0/2703228665 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd7d4104450 0x7fd7d41a04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:16.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.295+0000 7fd7d37fe640 1 --2- 172.21.15.69:0/2703228665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd7d4103910 0x7fd7d4199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:16.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.295+0000 7fd7d3fff640 1 -- 172.21.15.69:0/2703228665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd7d4103910 msgr2=0x7fd7d4199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:16.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.295+0000 7fd7d3fff640 1 --2- 172.21.15.69:0/2703228665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd7d4103910 0x7fd7d4199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:16.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.295+0000 7fd7d3fff640 1 -- 172.21.15.69:0/2703228665 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd7d4104450 msgr2=0x7fd7d41a04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:16.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.295+0000 7fd7d3fff640 1 --2- 172.21.15.69:0/2703228665 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd7d4104450 0x7fd7d41a04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:16.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.295+0000 7fd7d3fff640 1 -- 172.21.15.69:0/2703228665 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7d41a0a60 con 0x7fd7d4102f30 2024-04-25T11:38:16.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.295+0000 7fd7d37fe640 1 --2- 172.21.15.69:0/2703228665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd7d4103910 0x7fd7d4199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:38:16.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.295+0000 7fd7d8c5a640 1 --2- 172.21.15.69:0/2703228665 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd7d4104450 0x7fd7d41a04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:38:16.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.296+0000 7fd7d3fff640 1 --2- 172.21.15.69:0/2703228665 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7d4102f30 0x7fd7d4198fd0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd7c400c8c0 tx=0x7fd7c400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:16.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.296+0000 7fd7d17fa640 1 -- 172.21.15.69:0/2703228665 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4007bb0 con 0x7fd7d4102f30 2024-04-25T11:38:16.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.296+0000 7fd7d17fa640 1 -- 172.21.15.69:0/2703228665 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd7c4007d50 con 0x7fd7d4102f30 2024-04-25T11:38:16.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.296+0000 7fd7da6bd640 1 -- 172.21.15.69:0/2703228665 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7d41a0d50 con 0x7fd7d4102f30 2024-04-25T11:38:16.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.296+0000 7fd7d17fa640 1 -- 172.21.15.69:0/2703228665 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c40186a0 con 0x7fd7d4102f30 2024-04-25T11:38:16.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.296+0000 7fd7da6bd640 1 -- 172.21.15.69:0/2703228665 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7d41a1210 con 0x7fd7d4102f30 2024-04-25T11:38:16.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.297+0000 7fd7da6bd640 1 -- 172.21.15.69:0/2703228665 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7d41098e0 con 0x7fd7d4102f30 2024-04-25T11:38:16.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.298+0000 7fd7d17fa640 1 -- 172.21.15.69:0/2703228665 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fd7c400fa00 con 0x7fd7d4102f30 2024-04-25T11:38:16.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.299+0000 7fd7d17fa640 1 --2- 172.21.15.69:0/2703228665 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fd7bc0afb70 0x7fd7bc0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:16.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.299+0000 7fd7d37fe640 1 --2- 172.21.15.69:0/2703228665 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fd7bc0afb70 0x7fd7bc0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:16.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.299+0000 7fd7d17fa640 1 -- 172.21.15.69:0/2703228665 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7fd7c40df390 con 0x7fd7d4102f30 2024-04-25T11:38:16.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.300+0000 7fd7d37fe640 1 --2- 172.21.15.69:0/2703228665 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fd7bc0afb70 0x7fd7bc0b2080 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8005fd0 tx=0x7fd7c8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:16.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.303+0000 7fd7d17fa640 1 -- 172.21.15.69:0/2703228665 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd7c4014030 con 0x7fd7d4102f30 2024-04-25T11:38:16.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.486+0000 7fd7da6bd640 1 -- 172.21.15.69:0/2703228665 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd7d402dd20 con 0x7fd7d4102f30 2024-04-25T11:38:16.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.488+0000 7fd7d17fa640 1 -- 172.21.15.69:0/2703228665 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+24862 (secure 0 0 0) 0x7fd7c40a7cb0 con 0x7fd7d4102f30 2024-04-25T11:38:16.476 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:38:16.476 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":84,"fsid":"f7ed73dc-02f4-11ef-bc93-c7b262605968","created":"2024-04-25T11:15:37.965281+0000","modified":"2024-04-25T11:25:33.888184+0000","last_up_change":"2024-04-25T11:22:50.691869+0000","last_in_change":"2024-04-25T11:22:30.271959+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:18:57.781263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:23:38.685110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:23:39.943803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T11:23:43.650521+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6a0d179-4998-4754-bf82-2ea3dac54e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6803","nonce":3756025235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6805","nonce":3756025235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6809","nonce":3756025235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6807","nonce":3756025235}]},"public_addr":"172.21.15.69:6803/3756025235","cluster_addr":"172.21.15.69:6805/3756025235","heartbeat_back_addr":"172.21.15.69:6809/3756025235","heartbeat_front_addr":"172.21.15.69:6807/3756025235","state":["exists","up"]},{"osd":1,"uuid":"c0c66930-4233-493f-b2bb-99c473994b59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6801","nonce":3166305213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6803","nonce":3166305213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6807","nonce":3166305213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6805","nonce":3166305213}]},"public_addr":"172.21.15.146:6801/3166305213","cluster_addr":"172.21.15.146:6803/3166305213","heartbeat_back_addr":"172.21.15.146:6807/3166305213","heartbeat_front_addr":"172.21.15.146:6805/3166305213","state":["exists","up"]},{"osd":2,"uuid":"d4ef735b-9786-4cb9-83ef-7c3f0691a729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6801","nonce":1474319175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6803","nonce":1474319175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6807","nonce":1474319175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6805","nonce":1474319175}]},"public_addr":"172.21.15.152:6801/1474319175","cluster_addr":"172.21.15.152:6803/1474319175","heartbeat_back_addr":"172.21.15.152:6807/1474319175","heartbeat_front_addr":"172.21.15.152:6805/1474319175","state":["exists","up"]},{"osd":3,"uuid":"0ab85fb8-5bbb-4ffc-bddf-cae7644a64ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6811","nonce":1809071296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6813","nonce":1809071296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6817","nonce":1809071296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6815","nonce":1809071296}]},"public_addr":"172.21.15.69:6811/1809071296","cluster_addr":"172.21.15.69:6813/1809071296","heartbeat_back_addr":"172.21.15.69:6817/1809071296","heartbeat_front_addr":"172.21.15.69:6815/1809071296","state":["exists","up"]},{"osd":4,"uuid":"273e0592-bbbc-4188-8eeb-46a3879cf632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6809","nonce":1898520972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6811","nonce":1898520972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6815","nonce":1898520972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6813","nonce":1898520972}]},"public_addr":"172.21.15.146:6809/1898520972","cluster_addr":"172.21.15.146:6811/1898520972","heartbeat_back_addr":"172.21.15.146:6815/1898520972","heartbeat_front_addr":"172.21.15.146:6813/1898520972","state":["exists","up"]},{"osd":5,"uuid":"bbeae945-9fa2-48f9-a1d6-8bf18dcaef19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6809","nonce":1996555108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6811","nonce":1996555108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6815","nonce":1996555108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6813","nonce":1996555108}]},"public_addr":"172.21.15.152:6809/1996555108","cluster_addr":"172.21.15.152:6811/1996555108","heartbeat_back_addr":"172.21.15.152:6815/1996555108","heartbeat_front_addr":"172.21.15.152:6813/1996555108","state":["exists","up"]},{"osd":6,"uuid":"dbb61253-30bc-418e-bd75-118c80bbf931","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6819","nonce":491215762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6821","nonce":491215762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6825","nonce":491215762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6823","nonce":491215762}]},"public_addr":"172.21.15.69:6819/491215762","cluster_addr":"172.21.15.69:6821/491215762","heartbeat_back_addr":"172.21.15.69:6825/491215762","heartbeat_front_addr":"172.21.15.69:6823/491215762","state":["exists","up"]},{"osd":7,"uuid":"75f23854-e283-4e3e-94cf-7b4079d83876","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6817","nonce":2330294690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6819","nonce":2330294690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6823","nonce":2330294690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6821","nonce":2330294690}]},"public_addr":"172.21.15.146:6817/2330294690","cluster_addr":"172.21.15.146:6819/2330294690","heartbeat_back_addr":"172.21.15.146:6823/2330294690","heartbeat_front_addr":"172.21.15.146:6821/2330294690","state":["exists","up"]},{"osd":8,"uuid":"1aa1060c-0c14-4abd-be56-5ebfeb78759c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6817","nonce":950669381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6819","nonce":950669381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6823","nonce":950669381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6821","nonce":950669381}]},"public_addr":"172.21.15.152:6817/950669381","cluster_addr":"172.21.15.152:6819/950669381","heartbeat_back_addr":"172.21.15.152:6823/950669381","heartbeat_front_addr":"172.21.15.152:6821/950669381","state":["exists","up"]},{"osd":9,"uuid":"3f4bf800-4090-4138-9bb3-d001f13f450c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6827","nonce":2414186758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6829","nonce":2414186758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6833","nonce":2414186758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6831","nonce":2414186758}]},"public_addr":"172.21.15.69:6827/2414186758","cluster_addr":"172.21.15.69:6829/2414186758","heartbeat_back_addr":"172.21.15.69:6833/2414186758","heartbeat_front_addr":"172.21.15.69:6831/2414186758","state":["exists","up"]},{"osd":10,"uuid":"19e967e5-51ce-4506-80c9-0a95441baf99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6825","nonce":707256400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6827","nonce":707256400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6831","nonce":707256400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6829","nonce":707256400}]},"public_addr":"172.21.15.146:6825/707256400","cluster_addr":"172.21.15.146:6827/707256400","heartbeat_back_addr":"172.21.15.146:6831/707256400","heartbeat_front_addr":"172.21.15.146:6829/707256400","state":["exists","up"]},{"osd":11,"uuid":"7788c2de-6930-4808-be0f-f1fd1ae0cb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6825","nonce":2934336833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6827","nonce":2934336833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6831","nonce":2934336833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6829","nonce":2934336833}]},"public_addr":"172.21.15.152:6825/2934336833","cluster_addr":"172.21.15.152:6827/2934336833","heartbeat_back_addr":"172.21.15.152:6831/2934336833","heartbeat_front_addr":"172.21.15.152:6829/2934336833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:07.497448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:31.227494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:55.027978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:20.713437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:46.214704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:20:11.199983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:20:37.177070+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:02.951240+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:28.963680+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:56.095807+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:22:22.221744+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:22:48.929334+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1b","mappings":[{"from":5,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/1372727546":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6801/1097124842":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6800/1508198215":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3103465990":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3540132960":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3899912190":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6800/1097124842":"2024-04-26T11:16:31.688159+0000","172.21.15.69:0/1199907202":"2024-04-26T11:16:04.860792+0000","172.21.15.69:6801/1508198215":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/2879102529":"2024-04-26T11:16:31.688159+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:38:16.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.490+0000 7fd7da6bd640 1 -- 172.21.15.69:0/2703228665 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fd7bc0afb70 msgr2=0x7fd7bc0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:16.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.490+0000 7fd7da6bd640 1 --2- 172.21.15.69:0/2703228665 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fd7bc0afb70 0x7fd7bc0b2080 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8005fd0 tx=0x7fd7c8005850 comp rx=0 tx=0).stop 2024-04-25T11:38:16.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.490+0000 7fd7da6bd640 1 -- 172.21.15.69:0/2703228665 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7d4102f30 msgr2=0x7fd7d4198fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:16.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.490+0000 7fd7da6bd640 1 --2- 172.21.15.69:0/2703228665 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7d4102f30 0x7fd7d4198fd0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd7c400c8c0 tx=0x7fd7c400cd80 comp rx=0 tx=0).stop 2024-04-25T11:38:16.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.490+0000 7fd7da6bd640 1 -- 172.21.15.69:0/2703228665 shutdown_connections 2024-04-25T11:38:16.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.490+0000 7fd7da6bd640 1 --2- 172.21.15.69:0/2703228665 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fd7bc0afb70 0x7fd7bc0b2080 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:16.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.490+0000 7fd7da6bd640 1 --2- 172.21.15.69:0/2703228665 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd7d4104450 0x7fd7d41a04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:16.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.490+0000 7fd7da6bd640 1 --2- 172.21.15.69:0/2703228665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd7d4103910 0x7fd7d4199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:16.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.490+0000 7fd7da6bd640 1 --2- 172.21.15.69:0/2703228665 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7d4102f30 0x7fd7d4198fd0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:16.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.490+0000 7fd7da6bd640 1 -- 172.21.15.69:0/2703228665 >> 172.21.15.69:0/2703228665 conn(0x7fd7d40fd330 msgr2=0x7fd7d4106ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:16.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.490+0000 7fd7da6bd640 1 -- 172.21.15.69:0/2703228665 shutdown_connections 2024-04-25T11:38:16.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.490+0000 7fd7da6bd640 1 -- 172.21.15.69:0/2703228665 wait complete. 2024-04-25T11:38:16.489 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:18:57.781263+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:23:38.685110+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:23:39.943803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T11:23:43.650521+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T11:38:16.490 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:38:16.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.613+0000 7f8ba3706640 1 -- 172.21.15.69:0/3426096 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b9c103950 msgr2=0x7f8b9c103de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:16.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.613+0000 7f8ba3706640 1 --2- 172.21.15.69:0/3426096 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b9c103950 0x7f8b9c103de0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8b98009900 tx=0x7f8b9802a410 comp rx=0 tx=0).stop 2024-04-25T11:38:16.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.613+0000 7f8ba3706640 1 -- 172.21.15.69:0/3426096 shutdown_connections 2024-04-25T11:38:16.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.613+0000 7f8ba3706640 1 --2- 172.21.15.69:0/3426096 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b9c104470 0x7f8b9c10abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:16.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.613+0000 7f8ba3706640 1 --2- 172.21.15.69:0/3426096 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b9c103950 0x7f8b9c103de0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:16.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.613+0000 7f8ba3706640 1 --2- 172.21.15.69:0/3426096 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b9c102f70 0x7f8b9c103380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:16.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.613+0000 7f8ba3706640 1 -- 172.21.15.69:0/3426096 >> 172.21.15.69:0/3426096 conn(0x7f8b9c0fd350 msgr2=0x7f8b9c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:16.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.613+0000 7f8ba3706640 1 -- 172.21.15.69:0/3426096 shutdown_connections 2024-04-25T11:38:16.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.613+0000 7f8ba3706640 1 -- 172.21.15.69:0/3426096 wait complete. 2024-04-25T11:38:16.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.613+0000 7f8ba3706640 1 Processor -- start 2024-04-25T11:38:16.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.614+0000 7f8ba3706640 1 -- start start 2024-04-25T11:38:16.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.614+0000 7f8ba3706640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b9c102f70 0x7f8b9c199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:16.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.614+0000 7f8ba3706640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b9c103950 0x7f8b9c199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:16.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.614+0000 7f8ba3706640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b9c104470 0x7f8b9c1a04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:16.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.614+0000 7f8ba3706640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9c10e3d0 con 0x7f8b9c102f70 2024-04-25T11:38:16.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.614+0000 7f8ba3706640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9c10e250 con 0x7f8b9c103950 2024-04-25T11:38:16.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.614+0000 7f8ba3706640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9c10e550 con 0x7f8b9c104470 2024-04-25T11:38:16.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.614+0000 7f8ba14a2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b9c102f70 0x7f8b9c199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:16.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.614+0000 7f8ba14a2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b9c102f70 0x7f8b9c199010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42024/0 (socket says 172.21.15.69:42024) 2024-04-25T11:38:16.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.614+0000 7f8ba14a2640 1 -- 172.21.15.69:0/1703978041 learned_addr learned my addr 172.21.15.69:0/1703978041 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:38:16.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.614+0000 7f8ba0ca1640 1 --2- 172.21.15.69:0/1703978041 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b9c103950 0x7f8b9c199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:16.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.614+0000 7f8ba14a2640 1 -- 172.21.15.69:0/1703978041 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b9c104470 msgr2=0x7f8b9c1a04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:16.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.614+0000 7f8ba14a2640 1 --2- 172.21.15.69:0/1703978041 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b9c104470 0x7f8b9c1a04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:16.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.614+0000 7f8ba14a2640 1 -- 172.21.15.69:0/1703978041 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b9c103950 msgr2=0x7f8b9c199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:16.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.614+0000 7f8ba14a2640 1 --2- 172.21.15.69:0/1703978041 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b9c103950 0x7f8b9c199550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:16.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.614+0000 7f8ba14a2640 1 -- 172.21.15.69:0/1703978041 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b9c1a0b50 con 0x7f8b9c102f70 2024-04-25T11:38:16.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.614+0000 7f8ba0ca1640 1 --2- 172.21.15.69:0/1703978041 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b9c103950 0x7f8b9c199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:16.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.614+0000 7f8ba14a2640 1 --2- 172.21.15.69:0/1703978041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b9c102f70 0x7f8b9c199010 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c005fd0 tx=0x7f8b8c009e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:16.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.614+0000 7f8b927fc640 1 -- 172.21.15.69:0/1703978041 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c00d020 con 0x7f8b9c102f70 2024-04-25T11:38:16.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.614+0000 7f8b927fc640 1 -- 172.21.15.69:0/1703978041 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8b8c00d1c0 con 0x7f8b9c102f70 2024-04-25T11:38:16.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.614+0000 7f8ba3706640 1 -- 172.21.15.69:0/1703978041 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b9c101800 con 0x7f8b9c102f70 2024-04-25T11:38:16.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.614+0000 7f8b927fc640 1 -- 172.21.15.69:0/1703978041 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c003850 con 0x7f8b9c102f70 2024-04-25T11:38:16.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.615+0000 7f8ba3706640 1 -- 172.21.15.69:0/1703978041 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b9c101d50 con 0x7f8b9c102f70 2024-04-25T11:38:16.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.615+0000 7f8ba3706640 1 -- 172.21.15.69:0/1703978041 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b9c06a160 con 0x7f8b9c102f70 2024-04-25T11:38:16.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.617+0000 7f8b927fc640 1 -- 172.21.15.69:0/1703978041 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f8b8c01f020 con 0x7f8b9c102f70 2024-04-25T11:38:16.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.618+0000 7f8b927fc640 1 --2- 172.21.15.69:0/1703978041 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f8b880afaf0 0x7f8b880b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:16.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.618+0000 7f8ba0ca1640 1 --2- 172.21.15.69:0/1703978041 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f8b880afaf0 0x7f8b880b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:16.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.618+0000 7f8b927fc640 1 -- 172.21.15.69:0/1703978041 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f8b8c0e0ac0 con 0x7f8b9c102f70 2024-04-25T11:38:16.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.618+0000 7f8ba0ca1640 1 --2- 172.21.15.69:0/1703978041 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f8b880afaf0 0x7f8b880b2000 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8b98005f70 tx=0x7f8b98005720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:16.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.621+0000 7f8b927fc640 1 -- 172.21.15.69:0/1703978041 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8b8c026070 con 0x7f8b9c102f70 2024-04-25T11:38:16.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.796+0000 7f8ba3706640 1 -- 172.21.15.69:0/1703978041 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8b9c107610 con 0x7f8b9c102f70 2024-04-25T11:38:16.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.796+0000 7f8b927fc640 1 -- 172.21.15.69:0/1703978041 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7f8b8c0a92d0 con 0x7f8b9c102f70 2024-04-25T11:38:16.784 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-04-25T11:38:16.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.798+0000 7f8ba3706640 1 -- 172.21.15.69:0/1703978041 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f8b880afaf0 msgr2=0x7f8b880b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:16.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.798+0000 7f8ba3706640 1 --2- 172.21.15.69:0/1703978041 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f8b880afaf0 0x7f8b880b2000 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8b98005f70 tx=0x7f8b98005720 comp rx=0 tx=0).stop 2024-04-25T11:38:16.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.798+0000 7f8ba3706640 1 -- 172.21.15.69:0/1703978041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b9c102f70 msgr2=0x7f8b9c199010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:16.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.798+0000 7f8ba3706640 1 --2- 172.21.15.69:0/1703978041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b9c102f70 0x7f8b9c199010 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c005fd0 tx=0x7f8b8c009e00 comp rx=0 tx=0).stop 2024-04-25T11:38:16.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.798+0000 7f8ba3706640 1 -- 172.21.15.69:0/1703978041 shutdown_connections 2024-04-25T11:38:16.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.798+0000 7f8ba3706640 1 --2- 172.21.15.69:0/1703978041 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f8b880afaf0 0x7f8b880b2000 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:16.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.798+0000 7f8ba3706640 1 --2- 172.21.15.69:0/1703978041 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b9c104470 0x7f8b9c1a04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:16.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.798+0000 7f8ba3706640 1 --2- 172.21.15.69:0/1703978041 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b9c103950 0x7f8b9c199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:16.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.798+0000 7f8ba3706640 1 --2- 172.21.15.69:0/1703978041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b9c102f70 0x7f8b9c199010 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:16.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.798+0000 7f8ba3706640 1 -- 172.21.15.69:0/1703978041 >> 172.21.15.69:0/1703978041 conn(0x7f8b9c0fd350 msgr2=0x7f8b9c104ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:16.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.798+0000 7f8ba3706640 1 -- 172.21.15.69:0/1703978041 shutdown_connections 2024-04-25T11:38:16.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.798+0000 7f8ba3706640 1 -- 172.21.15.69:0/1703978041 wait complete. 2024-04-25T11:38:16.797 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:38:16.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.921+0000 7f2ccc35f640 1 -- 172.21.15.69:0/3829084173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2cc4103c20 msgr2=0x7f2cc4104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:16.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.921+0000 7f2ccc35f640 1 --2- 172.21.15.69:0/3829084173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2cc4103c20 0x7f2cc4104030 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4009900 tx=0x7f2cb402a410 comp rx=0 tx=0).stop 2024-04-25T11:38:16.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.921+0000 7f2ccc35f640 1 -- 172.21.15.69:0/3829084173 shutdown_connections 2024-04-25T11:38:16.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.921+0000 7f2ccc35f640 1 --2- 172.21.15.69:0/3829084173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2cc4108920 0x7f2cc410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:16.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.921+0000 7f2ccc35f640 1 --2- 172.21.15.69:0/3829084173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2cc4103c20 0x7f2cc4104030 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:16.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.921+0000 7f2ccc35f640 1 --2- 172.21.15.69:0/3829084173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2cc41029e0 0x7f2cc41036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:16.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.921+0000 7f2ccc35f640 1 -- 172.21.15.69:0/3829084173 >> 172.21.15.69:0/3829084173 conn(0x7f2cc40fdbd0 msgr2=0x7f2cc4100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:16.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.921+0000 7f2ccc35f640 1 -- 172.21.15.69:0/3829084173 shutdown_connections 2024-04-25T11:38:16.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.921+0000 7f2ccc35f640 1 -- 172.21.15.69:0/3829084173 wait complete. 2024-04-25T11:38:16.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.922+0000 7f2ccc35f640 1 Processor -- start 2024-04-25T11:38:16.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.922+0000 7f2ccc35f640 1 -- start start 2024-04-25T11:38:16.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.922+0000 7f2ccc35f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2cc41029e0 0x7f2cc4199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:16.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.922+0000 7f2ccc35f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2cc4103c20 0x7f2cc41998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:16.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.922+0000 7f2ccc35f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2cc4108920 0x7f2cc41a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:16.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.922+0000 7f2ccc35f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc410e3b0 con 0x7f2cc4108920 2024-04-25T11:38:16.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.922+0000 7f2ccc35f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc410e230 con 0x7f2cc4103c20 2024-04-25T11:38:16.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.922+0000 7f2ccc35f640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc410e530 con 0x7f2cc41029e0 2024-04-25T11:38:16.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.922+0000 7f2cca8fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2cc4108920 0x7f2cc41a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:16.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.922+0000 7f2cca8fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2cc4108920 0x7f2cc41a0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42046/0 (socket says 172.21.15.69:42046) 2024-04-25T11:38:16.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.922+0000 7f2cca8fc640 1 -- 172.21.15.69:0/1541754580 learned_addr learned my addr 172.21.15.69:0/1541754580 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:38:16.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.923+0000 7f2cca0fb640 1 --2- 172.21.15.69:0/1541754580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2cc41029e0 0x7f2cc4199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:16.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.923+0000 7f2cc98fa640 1 --2- 172.21.15.69:0/1541754580 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2cc4103c20 0x7f2cc41998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:16.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.923+0000 7f2cca8fc640 1 -- 172.21.15.69:0/1541754580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2cc41029e0 msgr2=0x7f2cc4199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:16.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.923+0000 7f2cca8fc640 1 --2- 172.21.15.69:0/1541754580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2cc41029e0 0x7f2cc4199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:16.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.923+0000 7f2cca8fc640 1 -- 172.21.15.69:0/1541754580 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2cc4103c20 msgr2=0x7f2cc41998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:16.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.923+0000 7f2cca8fc640 1 --2- 172.21.15.69:0/1541754580 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2cc4103c20 0x7f2cc41998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:16.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.923+0000 7f2cca8fc640 1 -- 172.21.15.69:0/1541754580 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cc41a0ea0 con 0x7f2cc4108920 2024-04-25T11:38:16.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.923+0000 7f2cca8fc640 1 --2- 172.21.15.69:0/1541754580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2cc4108920 0x7f2cc41a0830 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc00eb10 tx=0x7f2cbc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:16.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.923+0000 7f2cbb7fe640 1 -- 172.21.15.69:0/1541754580 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc004b20 con 0x7f2cc4108920 2024-04-25T11:38:16.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.923+0000 7f2ccc35f640 1 -- 172.21.15.69:0/1541754580 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cc41a1190 con 0x7f2cc4108920 2024-04-25T11:38:16.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.923+0000 7f2cbb7fe640 1 -- 172.21.15.69:0/1541754580 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2cbc004cc0 con 0x7f2cc4108920 2024-04-25T11:38:16.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.923+0000 7f2cbb7fe640 1 -- 172.21.15.69:0/1541754580 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc010690 con 0x7f2cc4108920 2024-04-25T11:38:16.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.923+0000 7f2ccc35f640 1 -- 172.21.15.69:0/1541754580 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cc41a16e0 con 0x7f2cc4108920 2024-04-25T11:38:16.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.924+0000 7f2ccc35f640 1 -- 172.21.15.69:0/1541754580 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cc4107730 con 0x7f2cc4108920 2024-04-25T11:38:16.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.926+0000 7f2cbb7fe640 1 -- 172.21.15.69:0/1541754580 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f2cbc005000 con 0x7f2cc4108920 2024-04-25T11:38:16.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.926+0000 7f2cbb7fe640 1 --2- 172.21.15.69:0/1541754580 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f2cb00afaf0 0x7f2cb00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:16.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.926+0000 7f2cca0fb640 1 --2- 172.21.15.69:0/1541754580 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f2cb00afaf0 0x7f2cb00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:16.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.926+0000 7f2cbb7fe640 1 -- 172.21.15.69:0/1541754580 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f2cbc0df8d0 con 0x7f2cc4108920 2024-04-25T11:38:16.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.927+0000 7f2cca0fb640 1 --2- 172.21.15.69:0/1541754580 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f2cb00afaf0 0x7f2cb00b2000 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2cc00058c0 tx=0x7f2cc000a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:16.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:16.930+0000 7f2cbb7fe640 1 -- 172.21.15.69:0/1541754580 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2cbc053ff0 con 0x7f2cc4108920 2024-04-25T11:38:17.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.106+0000 7f2ccc35f640 1 -- 172.21.15.69:0/1541754580 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2cc402dce0 con 0x7f2cc4108920 2024-04-25T11:38:17.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.106+0000 7f2cbb7fe640 1 -- 172.21.15.69:0/1541754580 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) ==== 104+0+11 (secure 0 0 0) 0x7f2cbc0a80e0 con 0x7f2cc4108920 2024-04-25T11:38:17.094 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-25T11:38:17.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.108+0000 7f2ccc35f640 1 -- 172.21.15.69:0/1541754580 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f2cb00afaf0 msgr2=0x7f2cb00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:17.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.108+0000 7f2ccc35f640 1 --2- 172.21.15.69:0/1541754580 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f2cb00afaf0 0x7f2cb00b2000 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2cc00058c0 tx=0x7f2cc000a4c0 comp rx=0 tx=0).stop 2024-04-25T11:38:17.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.108+0000 7f2ccc35f640 1 -- 172.21.15.69:0/1541754580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2cc4108920 msgr2=0x7f2cc41a0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:17.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.108+0000 7f2ccc35f640 1 --2- 172.21.15.69:0/1541754580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2cc4108920 0x7f2cc41a0830 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc00eb10 tx=0x7f2cbc00c2d0 comp rx=0 tx=0).stop 2024-04-25T11:38:17.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.108+0000 7f2ccc35f640 1 -- 172.21.15.69:0/1541754580 shutdown_connections 2024-04-25T11:38:17.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.108+0000 7f2ccc35f640 1 --2- 172.21.15.69:0/1541754580 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f2cb00afaf0 0x7f2cb00b2000 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:17.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.108+0000 7f2ccc35f640 1 --2- 172.21.15.69:0/1541754580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2cc4108920 0x7f2cc41a0830 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:17.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.108+0000 7f2ccc35f640 1 --2- 172.21.15.69:0/1541754580 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2cc4103c20 0x7f2cc41998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:17.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.108+0000 7f2ccc35f640 1 --2- 172.21.15.69:0/1541754580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2cc41029e0 0x7f2cc4199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:17.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.108+0000 7f2ccc35f640 1 -- 172.21.15.69:0/1541754580 >> 172.21.15.69:0/1541754580 conn(0x7f2cc40fdbd0 msgr2=0x7f2cc4104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:17.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.108+0000 7f2ccc35f640 1 -- 172.21.15.69:0/1541754580 shutdown_connections 2024-04-25T11:38:17.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.108+0000 7f2ccc35f640 1 -- 172.21.15.69:0/1541754580 wait complete. 2024-04-25T11:38:17.107 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:38:17.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.231+0000 7fd3d935f640 1 -- 172.21.15.69:0/989343507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3d4106f40 msgr2=0x7fd3d4109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:17.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.231+0000 7fd3d935f640 1 --2- 172.21.15.69:0/989343507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3d4106f40 0x7fd3d4109360 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc009900 tx=0x7fd3bc02a410 comp rx=0 tx=0).stop 2024-04-25T11:38:17.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.231+0000 7fd3d935f640 1 -- 172.21.15.69:0/989343507 shutdown_connections 2024-04-25T11:38:17.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.231+0000 7fd3d935f640 1 --2- 172.21.15.69:0/989343507 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3d41098a0 0x7fd3d410bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:17.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.231+0000 7fd3d935f640 1 --2- 172.21.15.69:0/989343507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3d4106f40 0x7fd3d4109360 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:17.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.231+0000 7fd3d935f640 1 --2- 172.21.15.69:0/989343507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3d41045e0 0x7fd3d4106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:17.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.231+0000 7fd3d935f640 1 -- 172.21.15.69:0/989343507 >> 172.21.15.69:0/989343507 conn(0x7fd3d40fdbc0 msgr2=0x7fd3d4100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:17.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.231+0000 7fd3d935f640 1 -- 172.21.15.69:0/989343507 shutdown_connections 2024-04-25T11:38:17.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.231+0000 7fd3d935f640 1 -- 172.21.15.69:0/989343507 wait complete. 2024-04-25T11:38:17.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.231+0000 7fd3d935f640 1 Processor -- start 2024-04-25T11:38:17.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.231+0000 7fd3d935f640 1 -- start start 2024-04-25T11:38:17.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.232+0000 7fd3d935f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3d41045e0 0x7fd3d419a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:17.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.232+0000 7fd3d935f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3d4106f40 0x7fd3d419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:17.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.232+0000 7fd3d935f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3d41098a0 0x7fd3d41a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:17.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.232+0000 7fd3d935f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d410f540 con 0x7fd3d41045e0 2024-04-25T11:38:17.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.232+0000 7fd3d935f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d410f3c0 con 0x7fd3d41098a0 2024-04-25T11:38:17.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.232+0000 7fd3d935f640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d410f6c0 con 0x7fd3d4106f40 2024-04-25T11:38:17.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.232+0000 7fd3d2ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3d41045e0 0x7fd3d419a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:17.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.232+0000 7fd3d2ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3d41045e0 0x7fd3d419a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42074/0 (socket says 172.21.15.69:42074) 2024-04-25T11:38:17.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.232+0000 7fd3d2ffd640 1 -- 172.21.15.69:0/1607143969 learned_addr learned my addr 172.21.15.69:0/1607143969 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:38:17.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.232+0000 7fd3d37fe640 1 --2- 172.21.15.69:0/1607143969 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3d41098a0 0x7fd3d41a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:17.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.232+0000 7fd3d2ffd640 1 -- 172.21.15.69:0/1607143969 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3d4106f40 msgr2=0x7fd3d419a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:17.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.232+0000 7fd3cbfff640 1 --2- 172.21.15.69:0/1607143969 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3d4106f40 0x7fd3d419a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:17.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.232+0000 7fd3d2ffd640 1 --2- 172.21.15.69:0/1607143969 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3d4106f40 0x7fd3d419a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:17.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.232+0000 7fd3d2ffd640 1 -- 172.21.15.69:0/1607143969 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3d41098a0 msgr2=0x7fd3d41a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:17.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.232+0000 7fd3d2ffd640 1 --2- 172.21.15.69:0/1607143969 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3d41098a0 0x7fd3d41a16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:17.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.232+0000 7fd3d2ffd640 1 -- 172.21.15.69:0/1607143969 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3d41a1d20 con 0x7fd3d41045e0 2024-04-25T11:38:17.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.232+0000 7fd3d37fe640 1 --2- 172.21.15.69:0/1607143969 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3d41098a0 0x7fd3d41a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:38:17.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.232+0000 7fd3d2ffd640 1 --2- 172.21.15.69:0/1607143969 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3d41045e0 0x7fd3d419a1e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fd3c400b450 tx=0x7fd3c400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:17.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.232+0000 7fd3d0ff9640 1 -- 172.21.15.69:0/1607143969 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c4015020 con 0x7fd3d41045e0 2024-04-25T11:38:17.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.232+0000 7fd3d0ff9640 1 -- 172.21.15.69:0/1607143969 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd3c4011d20 con 0x7fd3d41045e0 2024-04-25T11:38:17.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.232+0000 7fd3d935f640 1 -- 172.21.15.69:0/1607143969 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3d41a1fb0 con 0x7fd3d41045e0 2024-04-25T11:38:17.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.232+0000 7fd3d0ff9640 1 -- 172.21.15.69:0/1607143969 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c4010e10 con 0x7fd3d41045e0 2024-04-25T11:38:17.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.232+0000 7fd3d935f640 1 -- 172.21.15.69:0/1607143969 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3d41a24f0 con 0x7fd3d41045e0 2024-04-25T11:38:17.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.233+0000 7fd3d935f640 1 -- 172.21.15.69:0/1607143969 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3d40ff6e0 con 0x7fd3d41045e0 2024-04-25T11:38:17.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.235+0000 7fd3d0ff9640 1 -- 172.21.15.69:0/1607143969 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fd3c4018020 con 0x7fd3d41045e0 2024-04-25T11:38:17.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.235+0000 7fd3d0ff9640 1 --2- 172.21.15.69:0/1607143969 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fd3c00afaf0 0x7fd3c00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:17.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.236+0000 7fd3cbfff640 1 --2- 172.21.15.69:0/1607143969 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fd3c00afaf0 0x7fd3c00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:17.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.236+0000 7fd3d0ff9640 1 -- 172.21.15.69:0/1607143969 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7fd3c40def40 con 0x7fd3d41045e0 2024-04-25T11:38:17.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.236+0000 7fd3cbfff640 1 --2- 172.21.15.69:0/1607143969 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fd3c00afaf0 0x7fd3c00b2000 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc031460 tx=0x7fd3bc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:17.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.239+0000 7fd3d0ff9640 1 -- 172.21.15.69:0/1607143969 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd3c401f020 con 0x7fd3d41045e0 2024-04-25T11:38:17.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.417+0000 7fd3d935f640 1 -- 172.21.15.69:0/1607143969 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd3d40637a0 con 0x7fd3d41045e0 2024-04-25T11:38:17.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.417+0000 7fd3d0ff9640 1 -- 172.21.15.69:0/1607143969 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) ==== 100+0+11 (secure 0 0 0) 0x7fd3c400d040 con 0x7fd3d41045e0 2024-04-25T11:38:17.405 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-04-25T11:38:17.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.419+0000 7fd3d935f640 1 -- 172.21.15.69:0/1607143969 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fd3c00afaf0 msgr2=0x7fd3c00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:17.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.419+0000 7fd3d935f640 1 --2- 172.21.15.69:0/1607143969 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fd3c00afaf0 0x7fd3c00b2000 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc031460 tx=0x7fd3bc000bc0 comp rx=0 tx=0).stop 2024-04-25T11:38:17.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.419+0000 7fd3d935f640 1 -- 172.21.15.69:0/1607143969 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3d41045e0 msgr2=0x7fd3d419a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:17.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.419+0000 7fd3d935f640 1 --2- 172.21.15.69:0/1607143969 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3d41045e0 0x7fd3d419a1e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fd3c400b450 tx=0x7fd3c400b910 comp rx=0 tx=0).stop 2024-04-25T11:38:17.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.419+0000 7fd3d935f640 1 -- 172.21.15.69:0/1607143969 shutdown_connections 2024-04-25T11:38:17.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.419+0000 7fd3d935f640 1 --2- 172.21.15.69:0/1607143969 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fd3c00afaf0 0x7fd3c00b2000 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:17.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.419+0000 7fd3d935f640 1 --2- 172.21.15.69:0/1607143969 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3d41098a0 0x7fd3d41a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:17.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.419+0000 7fd3d935f640 1 --2- 172.21.15.69:0/1607143969 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3d4106f40 0x7fd3d419a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:17.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.419+0000 7fd3d935f640 1 --2- 172.21.15.69:0/1607143969 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3d41045e0 0x7fd3d419a1e0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:17.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.419+0000 7fd3d935f640 1 -- 172.21.15.69:0/1607143969 >> 172.21.15.69:0/1607143969 conn(0x7fd3d40fdbc0 msgr2=0x7fd3d4106640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:17.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.419+0000 7fd3d935f640 1 -- 172.21.15.69:0/1607143969 shutdown_connections 2024-04-25T11:38:17.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.419+0000 7fd3d935f640 1 -- 172.21.15.69:0/1607143969 wait complete. 2024-04-25T11:38:17.416 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:17 smithi069 ceph-mon[29188]: pgmap v726: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T11:38:17.417 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:17 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2703228665' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:38:17.417 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:17 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/1703978041' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:38:17.417 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:17 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/1541754580' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:38:17.418 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T11:38:17.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.541+0000 7f731cc4c640 1 -- 172.21.15.69:0/1039468114 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73181029e0 msgr2=0x7f73181036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:17.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.541+0000 7f731cc4c640 1 --2- 172.21.15.69:0/1039468114 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73181029e0 0x7f73181036e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f7308009900 tx=0x7f730802a410 comp rx=0 tx=0).stop 2024-04-25T11:38:17.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.541+0000 7f731cc4c640 1 -- 172.21.15.69:0/1039468114 shutdown_connections 2024-04-25T11:38:17.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.541+0000 7f731cc4c640 1 --2- 172.21.15.69:0/1039468114 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7318108920 0x7f731810ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:17.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.541+0000 7f731cc4c640 1 --2- 172.21.15.69:0/1039468114 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7318103c20 0x7f7318104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:17.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.541+0000 7f731cc4c640 1 --2- 172.21.15.69:0/1039468114 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73181029e0 0x7f73181036e0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:17.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.541+0000 7f731cc4c640 1 -- 172.21.15.69:0/1039468114 >> 172.21.15.69:0/1039468114 conn(0x7f73180fdbd0 msgr2=0x7f7318100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:17.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.541+0000 7f731cc4c640 1 -- 172.21.15.69:0/1039468114 shutdown_connections 2024-04-25T11:38:17.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.541+0000 7f731cc4c640 1 -- 172.21.15.69:0/1039468114 wait complete. 2024-04-25T11:38:17.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.542+0000 7f731cc4c640 1 Processor -- start 2024-04-25T11:38:17.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.542+0000 7f731cc4c640 1 -- start start 2024-04-25T11:38:17.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.542+0000 7f731cc4c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73181029e0 0x7f731806a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:17.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.542+0000 7f731cc4c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7318103c20 0x7f731819f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:17.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.542+0000 7f731cc4c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7318108920 0x7f73181a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:17.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.542+0000 7f731cc4c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f731810e390 con 0x7f73181029e0 2024-04-25T11:38:17.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.542+0000 7f731cc4c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f731810e210 con 0x7f7318103c20 2024-04-25T11:38:17.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.542+0000 7f731cc4c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f731810e510 con 0x7f7318108920 2024-04-25T11:38:17.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.542+0000 7f731659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73181029e0 0x7f731806a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:17.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.542+0000 7f731659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73181029e0 0x7f731806a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42096/0 (socket says 172.21.15.69:42096) 2024-04-25T11:38:17.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.542+0000 7f731659c640 1 -- 172.21.15.69:0/1410470692 learned_addr learned my addr 172.21.15.69:0/1410470692 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:38:17.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.542+0000 7f730ffff640 1 --2- 172.21.15.69:0/1410470692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7318103c20 0x7f731819f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:17.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.542+0000 7f731659c640 1 -- 172.21.15.69:0/1410470692 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7318108920 msgr2=0x7f73181a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:17.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.542+0000 7f731659c640 1 --2- 172.21.15.69:0/1410470692 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7318108920 0x7f73181a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:17.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.542+0000 7f731659c640 1 -- 172.21.15.69:0/1410470692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7318103c20 msgr2=0x7f731819f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:17.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.542+0000 7f731659c640 1 --2- 172.21.15.69:0/1410470692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7318103c20 0x7f731819f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:17.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.542+0000 7f731659c640 1 -- 172.21.15.69:0/1410470692 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73181a1f80 con 0x7f73181029e0 2024-04-25T11:38:17.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.542+0000 7f731659c640 1 --2- 172.21.15.69:0/1410470692 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73181029e0 0x7f731806a900 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7308005760 tx=0x7f7308004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:17.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.543+0000 7f730f7fe640 1 -- 172.21.15.69:0/1410470692 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730803f030 con 0x7f73181029e0 2024-04-25T11:38:17.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.543+0000 7f730f7fe640 1 -- 172.21.15.69:0/1410470692 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7308003a80 con 0x7f73181029e0 2024-04-25T11:38:17.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.543+0000 7f731cc4c640 1 -- 172.21.15.69:0/1410470692 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73181a2210 con 0x7f73181029e0 2024-04-25T11:38:17.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.543+0000 7f730f7fe640 1 -- 172.21.15.69:0/1410470692 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7308035070 con 0x7f73181029e0 2024-04-25T11:38:17.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.543+0000 7f731cc4c640 1 -- 172.21.15.69:0/1410470692 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73181a2700 con 0x7f73181029e0 2024-04-25T11:38:17.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.544+0000 7f731cc4c640 1 -- 172.21.15.69:0/1410470692 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7318107730 con 0x7f73181029e0 2024-04-25T11:38:17.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.545+0000 7f730f7fe640 1 -- 172.21.15.69:0/1410470692 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f7308042030 con 0x7f73181029e0 2024-04-25T11:38:17.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.546+0000 7f730f7fe640 1 --2- 172.21.15.69:0/1410470692 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f73040afaf0 0x7f73040b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:17.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.546+0000 7f730ffff640 1 --2- 172.21.15.69:0/1410470692 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f73040afaf0 0x7f73040b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:17.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.546+0000 7f730f7fe640 1 -- 172.21.15.69:0/1410470692 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f73080ffbd0 con 0x7f73181029e0 2024-04-25T11:38:17.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.547+0000 7f730ffff640 1 --2- 172.21.15.69:0/1410470692 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f73040afaf0 0x7f73040b2000 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7300005d00 tx=0x7f7300005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:17.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.550+0000 7f730f7fe640 1 -- 172.21.15.69:0/1410470692 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f73080c83e0 con 0x7f73181029e0 2024-04-25T11:38:17.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:17 smithi146 ceph-mon[32821]: pgmap v726: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T11:38:17.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:17 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/2703228665' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:38:17.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:17 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/1703978041' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:38:17.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:17 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/1541754580' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:38:17.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.731+0000 7f731cc4c640 1 -- 172.21.15.69:0/1410470692 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f731802dd20 con 0x7f73181029e0 2024-04-25T11:38:17.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.732+0000 7f730f7fe640 1 -- 172.21.15.69:0/1410470692 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v84) ==== 103+0+11 (secure 0 0 0) 0x7f73080c7ce0 con 0x7f73181029e0 2024-04-25T11:38:17.719 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-25T11:38:17.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.734+0000 7f731cc4c640 1 -- 172.21.15.69:0/1410470692 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f73040afaf0 msgr2=0x7f73040b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:17.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.734+0000 7f731cc4c640 1 --2- 172.21.15.69:0/1410470692 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f73040afaf0 0x7f73040b2000 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7300005d00 tx=0x7f7300005c70 comp rx=0 tx=0).stop 2024-04-25T11:38:17.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.734+0000 7f731cc4c640 1 -- 172.21.15.69:0/1410470692 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73181029e0 msgr2=0x7f731806a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:17.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.734+0000 7f731cc4c640 1 --2- 172.21.15.69:0/1410470692 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73181029e0 0x7f731806a900 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7308005760 tx=0x7f7308004270 comp rx=0 tx=0).stop 2024-04-25T11:38:17.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.734+0000 7f731cc4c640 1 -- 172.21.15.69:0/1410470692 shutdown_connections 2024-04-25T11:38:17.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.734+0000 7f731cc4c640 1 --2- 172.21.15.69:0/1410470692 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f73040afaf0 0x7f73040b2000 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:17.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.734+0000 7f731cc4c640 1 --2- 172.21.15.69:0/1410470692 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7318108920 0x7f73181a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:17.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.734+0000 7f731cc4c640 1 --2- 172.21.15.69:0/1410470692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7318103c20 0x7f731819f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:17.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.734+0000 7f731cc4c640 1 --2- 172.21.15.69:0/1410470692 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73181029e0 0x7f731806a900 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:17.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.734+0000 7f731cc4c640 1 -- 172.21.15.69:0/1410470692 >> 172.21.15.69:0/1410470692 conn(0x7f73180fdbd0 msgr2=0x7f7318104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:17.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.734+0000 7f731cc4c640 1 -- 172.21.15.69:0/1410470692 shutdown_connections 2024-04-25T11:38:17.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.734+0000 7f731cc4c640 1 -- 172.21.15.69:0/1410470692 wait complete. 2024-04-25T11:38:17.724 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:17 smithi152 ceph-mon[32701]: pgmap v726: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T11:38:17.724 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:17 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/2703228665' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:38:17.724 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:17 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/1703978041' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:38:17.724 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:17 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/1541754580' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:38:17.733 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:38:17.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.854+0000 7fd3f0534640 1 -- 172.21.15.69:0/3044406756 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3e8106f30 msgr2=0x7fd3e8109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:17.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.854+0000 7fd3f0534640 1 --2- 172.21.15.69:0/3044406756 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3e8106f30 0x7fd3e8109350 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc009900 tx=0x7fd3dc02a410 comp rx=0 tx=0).stop 2024-04-25T11:38:17.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.854+0000 7fd3f0534640 1 -- 172.21.15.69:0/3044406756 shutdown_connections 2024-04-25T11:38:17.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.854+0000 7fd3f0534640 1 --2- 172.21.15.69:0/3044406756 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3e8109890 0x7fd3e810bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:17.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.854+0000 7fd3f0534640 1 --2- 172.21.15.69:0/3044406756 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3e8106f30 0x7fd3e8109350 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:17.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.854+0000 7fd3f0534640 1 --2- 172.21.15.69:0/3044406756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3e81045d0 0x7fd3e81069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:17.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.854+0000 7fd3f0534640 1 -- 172.21.15.69:0/3044406756 >> 172.21.15.69:0/3044406756 conn(0x7fd3e80fdbd0 msgr2=0x7fd3e8100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:17.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.854+0000 7fd3f0534640 1 -- 172.21.15.69:0/3044406756 shutdown_connections 2024-04-25T11:38:17.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.854+0000 7fd3f0534640 1 -- 172.21.15.69:0/3044406756 wait complete. 2024-04-25T11:38:17.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.855+0000 7fd3f0534640 1 Processor -- start 2024-04-25T11:38:17.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.855+0000 7fd3f0534640 1 -- start start 2024-04-25T11:38:17.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.855+0000 7fd3f0534640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3e81045d0 0x7fd3e819a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:17.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.855+0000 7fd3f0534640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3e8106f30 0x7fd3e819a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:17.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.855+0000 7fd3f0534640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3e8109890 0x7fd3e81a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:17.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.855+0000 7fd3f0534640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e810f4c0 con 0x7fd3e8109890 2024-04-25T11:38:17.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.855+0000 7fd3f0534640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e810f340 con 0x7fd3e81045d0 2024-04-25T11:38:17.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.855+0000 7fd3f0534640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e810f640 con 0x7fd3e8106f30 2024-04-25T11:38:17.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.855+0000 7fd3eead1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3e8109890 0x7fd3e81a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:17.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.855+0000 7fd3eead1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3e8109890 0x7fd3e81a1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42118/0 (socket says 172.21.15.69:42118) 2024-04-25T11:38:17.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.855+0000 7fd3eead1640 1 -- 172.21.15.69:0/438221288 learned_addr learned my addr 172.21.15.69:0/438221288 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:38:17.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.855+0000 7fd3ee2d0640 1 --2- 172.21.15.69:0/438221288 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3e81045d0 0x7fd3e819a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:17.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.855+0000 7fd3edacf640 1 --2- 172.21.15.69:0/438221288 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3e8106f30 0x7fd3e819a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:17.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.855+0000 7fd3eead1640 1 -- 172.21.15.69:0/438221288 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3e8106f30 msgr2=0x7fd3e819a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:17.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.855+0000 7fd3eead1640 1 --2- 172.21.15.69:0/438221288 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3e8106f30 0x7fd3e819a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:17.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.855+0000 7fd3eead1640 1 -- 172.21.15.69:0/438221288 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3e81045d0 msgr2=0x7fd3e819a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:17.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.855+0000 7fd3eead1640 1 --2- 172.21.15.69:0/438221288 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3e81045d0 0x7fd3e819a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:17.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.855+0000 7fd3eead1640 1 -- 172.21.15.69:0/438221288 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3e81a1d00 con 0x7fd3e8109890 2024-04-25T11:38:17.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.856+0000 7fd3eead1640 1 --2- 172.21.15.69:0/438221288 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3e8109890 0x7fd3e81a1690 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fd3d800c970 tx=0x7fd3d800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:17.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.856+0000 7fd3e77fe640 1 -- 172.21.15.69:0/438221288 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d8004230 con 0x7fd3e8109890 2024-04-25T11:38:17.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.856+0000 7fd3e77fe640 1 -- 172.21.15.69:0/438221288 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd3d8007e40 con 0x7fd3e8109890 2024-04-25T11:38:17.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.856+0000 7fd3f0534640 1 -- 172.21.15.69:0/438221288 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3e81a1ff0 con 0x7fd3e8109890 2024-04-25T11:38:17.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.856+0000 7fd3e77fe640 1 -- 172.21.15.69:0/438221288 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d8003da0 con 0x7fd3e8109890 2024-04-25T11:38:17.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.856+0000 7fd3f0534640 1 -- 172.21.15.69:0/438221288 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3e81a24b0 con 0x7fd3e8109890 2024-04-25T11:38:17.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.857+0000 7fd3f0534640 1 -- 172.21.15.69:0/438221288 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3e80ff6f0 con 0x7fd3e8109890 2024-04-25T11:38:17.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.858+0000 7fd3e77fe640 1 -- 172.21.15.69:0/438221288 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fd3d80079b0 con 0x7fd3e8109890 2024-04-25T11:38:17.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.859+0000 7fd3e77fe640 1 --2- 172.21.15.69:0/438221288 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fd3d00afb70 0x7fd3d00b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:17.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.859+0000 7fd3ee2d0640 1 --2- 172.21.15.69:0/438221288 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fd3d00afb70 0x7fd3d00b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:17.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.859+0000 7fd3e77fe640 1 -- 172.21.15.69:0/438221288 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7fd3d80df0f0 con 0x7fd3e8109890 2024-04-25T11:38:17.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.859+0000 7fd3ee2d0640 1 --2- 172.21.15.69:0/438221288 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fd3d00afb70 0x7fd3d00b2080 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd3e0005c40 tx=0x7fd3e0005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:17.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:17.863+0000 7fd3e77fe640 1 -- 172.21.15.69:0/438221288 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd3d8014070 con 0x7fd3e8109890 2024-04-25T11:38:18.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.053+0000 7fd3f0534640 1 -- 172.21.15.69:0/438221288 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd3e802db50 con 0x7fd3e8109890 2024-04-25T11:38:18.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.054+0000 7fd3e77fe640 1 -- 172.21.15.69:0/438221288 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v84) ==== 81+0+592 (secure 0 0 0) 0x7fd3d80a7980 con 0x7fd3e8109890 2024-04-25T11:38:18.041 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/1372727546 2024-04-26T11:16:31.688159+0000 2024-04-25T11:38:18.041 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6801/1097124842 2024-04-26T11:16:31.688159+0000 2024-04-25T11:38:18.042 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6800/1508198215 2024-04-26T11:16:04.860792+0000 2024-04-25T11:38:18.042 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/3103465990 2024-04-26T11:16:04.860792+0000 2024-04-25T11:38:18.042 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/3540132960 2024-04-26T11:16:04.860792+0000 2024-04-25T11:38:18.042 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/3899912190 2024-04-26T11:16:31.688159+0000 2024-04-25T11:38:18.042 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6800/1097124842 2024-04-26T11:16:31.688159+0000 2024-04-25T11:38:18.042 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/1199907202 2024-04-26T11:16:04.860792+0000 2024-04-25T11:38:18.042 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6801/1508198215 2024-04-26T11:16:04.860792+0000 2024-04-25T11:38:18.042 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/2879102529 2024-04-26T11:16:31.688159+0000 2024-04-25T11:38:18.042 INFO:teuthology.orchestra.run.smithi069.stderr:listed 10 entries 2024-04-25T11:38:18.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.056+0000 7fd3f0534640 1 -- 172.21.15.69:0/438221288 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fd3d00afb70 msgr2=0x7fd3d00b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:18.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.056+0000 7fd3f0534640 1 --2- 172.21.15.69:0/438221288 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fd3d00afb70 0x7fd3d00b2080 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd3e0005c40 tx=0x7fd3e0005bd0 comp rx=0 tx=0).stop 2024-04-25T11:38:18.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.056+0000 7fd3f0534640 1 -- 172.21.15.69:0/438221288 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3e8109890 msgr2=0x7fd3e81a1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:18.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.056+0000 7fd3f0534640 1 --2- 172.21.15.69:0/438221288 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3e8109890 0x7fd3e81a1690 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fd3d800c970 tx=0x7fd3d800ce30 comp rx=0 tx=0).stop 2024-04-25T11:38:18.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.056+0000 7fd3f0534640 1 -- 172.21.15.69:0/438221288 shutdown_connections 2024-04-25T11:38:18.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.056+0000 7fd3f0534640 1 --2- 172.21.15.69:0/438221288 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fd3d00afb70 0x7fd3d00b2080 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:18.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.056+0000 7fd3f0534640 1 --2- 172.21.15.69:0/438221288 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3e8109890 0x7fd3e81a1690 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:18.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.056+0000 7fd3f0534640 1 --2- 172.21.15.69:0/438221288 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3e8106f30 0x7fd3e819a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:18.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.056+0000 7fd3f0534640 1 --2- 172.21.15.69:0/438221288 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd3e81045d0 0x7fd3e819a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:18.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.056+0000 7fd3f0534640 1 -- 172.21.15.69:0/438221288 >> 172.21.15.69:0/438221288 conn(0x7fd3e80fdbd0 msgr2=0x7fd3e8106670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:18.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.056+0000 7fd3f0534640 1 -- 172.21.15.69:0/438221288 shutdown_connections 2024-04-25T11:38:18.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.056+0000 7fd3f0534640 1 -- 172.21.15.69:0/438221288 wait complete. 2024-04-25T11:38:18.055 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T11:38:18.055 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:38:18.070 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:38:18.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.231+0000 7f7555a0c640 1 -- 172.21.15.69:0/2576711662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75501029e0 msgr2=0x7f75501036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:18.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.231+0000 7f7555a0c640 1 --2- 172.21.15.69:0/2576711662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75501029e0 0x7f75501036e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f7540009ca0 tx=0x7f754002a8c0 comp rx=0 tx=0).stop 2024-04-25T11:38:18.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.231+0000 7f7555a0c640 1 -- 172.21.15.69:0/2576711662 shutdown_connections 2024-04-25T11:38:18.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.231+0000 7f7555a0c640 1 --2- 172.21.15.69:0/2576711662 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7550108920 0x7f755010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:18.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.231+0000 7f7555a0c640 1 --2- 172.21.15.69:0/2576711662 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7550103c20 0x7f7550104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:18.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.231+0000 7f7555a0c640 1 --2- 172.21.15.69:0/2576711662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75501029e0 0x7f75501036e0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:18.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.231+0000 7f7555a0c640 1 -- 172.21.15.69:0/2576711662 >> 172.21.15.69:0/2576711662 conn(0x7f75500fdbf0 msgr2=0x7f7550100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:18.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.231+0000 7f7555a0c640 1 -- 172.21.15.69:0/2576711662 shutdown_connections 2024-04-25T11:38:18.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.231+0000 7f7555a0c640 1 -- 172.21.15.69:0/2576711662 wait complete. 2024-04-25T11:38:18.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.232+0000 7f7555a0c640 1 Processor -- start 2024-04-25T11:38:18.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.232+0000 7f7555a0c640 1 -- start start 2024-04-25T11:38:18.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.232+0000 7f7555a0c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75501029e0 0x7f75501990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:18.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.232+0000 7f7555a0c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7550103c20 0x7f7550199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:18.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.232+0000 7f7555a0c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7550108920 0x7f75501a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:18.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.232+0000 7f7555a0c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f755010e3c0 con 0x7f75501029e0 2024-04-25T11:38:18.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.232+0000 7f7555a0c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f755010e240 con 0x7f7550108920 2024-04-25T11:38:18.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.232+0000 7f7555a0c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f755010e540 con 0x7f7550103c20 2024-04-25T11:38:18.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.232+0000 7f754effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75501029e0 0x7f75501990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:18.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.232+0000 7f754effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75501029e0 0x7f75501990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42140/0 (socket says 172.21.15.69:42140) 2024-04-25T11:38:18.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.232+0000 7f754effd640 1 -- 172.21.15.69:0/28253011 learned_addr learned my addr 172.21.15.69:0/28253011 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:38:18.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.233+0000 7f754f7fe640 1 --2- 172.21.15.69:0/28253011 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7550108920 0x7f75501a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:18.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.233+0000 7f754e7fc640 1 --2- 172.21.15.69:0/28253011 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7550103c20 0x7f7550199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:18.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.233+0000 7f754effd640 1 -- 172.21.15.69:0/28253011 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7550103c20 msgr2=0x7f7550199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:18.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.233+0000 7f754effd640 1 --2- 172.21.15.69:0/28253011 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7550103c20 0x7f7550199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:18.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.233+0000 7f754effd640 1 -- 172.21.15.69:0/28253011 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7550108920 msgr2=0x7f75501a05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:18.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.233+0000 7f754effd640 1 --2- 172.21.15.69:0/28253011 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7550108920 0x7f75501a05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:18.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.233+0000 7f754effd640 1 -- 172.21.15.69:0/28253011 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75501a0c20 con 0x7f75501029e0 2024-04-25T11:38:18.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.233+0000 7f754e7fc640 1 --2- 172.21.15.69:0/28253011 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7550103c20 0x7f7550199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:38:18.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.233+0000 7f754effd640 1 --2- 172.21.15.69:0/28253011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75501029e0 0x7f75501990e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7540005e30 tx=0x7f75400039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:18.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.233+0000 7f7554a0a640 1 -- 172.21.15.69:0/28253011 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7540003de0 con 0x7f75501029e0 2024-04-25T11:38:18.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.233+0000 7f7554a0a640 1 -- 172.21.15.69:0/28253011 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f75400044d0 con 0x7f75501029e0 2024-04-25T11:38:18.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.233+0000 7f7555a0c640 1 -- 172.21.15.69:0/28253011 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75501a0eb0 con 0x7f75501029e0 2024-04-25T11:38:18.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.233+0000 7f7554a0a640 1 -- 172.21.15.69:0/28253011 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7540030ca0 con 0x7f75501029e0 2024-04-25T11:38:18.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.233+0000 7f7555a0c640 1 -- 172.21.15.69:0/28253011 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75501a1310 con 0x7f75501029e0 2024-04-25T11:38:18.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.234+0000 7f7555a0c640 1 -- 172.21.15.69:0/28253011 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7550107730 con 0x7f75501029e0 2024-04-25T11:38:18.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.236+0000 7f7554a0a640 1 -- 172.21.15.69:0/28253011 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f7540004680 con 0x7f75501029e0 2024-04-25T11:38:18.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.236+0000 7f7554a0a640 1 --2- 172.21.15.69:0/28253011 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f75380afaf0 0x7f75380b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:18.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.236+0000 7f754e7fc640 1 --2- 172.21.15.69:0/28253011 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f75380afaf0 0x7f75380b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:18.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.236+0000 7f7554a0a640 1 -- 172.21.15.69:0/28253011 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f7540100670 con 0x7f75501029e0 2024-04-25T11:38:18.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.237+0000 7f754e7fc640 1 --2- 172.21.15.69:0/28253011 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f75380afaf0 0x7f75380b2000 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7544005fd0 tx=0x7f754400b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:18.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.240+0000 7f7554a0a640 1 -- 172.21.15.69:0/28253011 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f75400c8f10 con 0x7f75501029e0 2024-04-25T11:38:18.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.422+0000 7f7555a0c640 1 -- 172.21.15.69:0/28253011 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f75500637a0 con 0x7f75501029e0 2024-04-25T11:38:18.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.424+0000 7f7554a0a640 1 -- 172.21.15.69:0/28253011 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+24862 (secure 0 0 0) 0x7f75400c8810 con 0x7f75501029e0 2024-04-25T11:38:18.412 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:38:18.412 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":84,"fsid":"f7ed73dc-02f4-11ef-bc93-c7b262605968","created":"2024-04-25T11:15:37.965281+0000","modified":"2024-04-25T11:25:33.888184+0000","last_up_change":"2024-04-25T11:22:50.691869+0000","last_in_change":"2024-04-25T11:22:30.271959+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:18:57.781263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:23:38.685110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:23:39.943803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T11:23:43.650521+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6a0d179-4998-4754-bf82-2ea3dac54e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6803","nonce":3756025235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6805","nonce":3756025235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6809","nonce":3756025235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6807","nonce":3756025235}]},"public_addr":"172.21.15.69:6803/3756025235","cluster_addr":"172.21.15.69:6805/3756025235","heartbeat_back_addr":"172.21.15.69:6809/3756025235","heartbeat_front_addr":"172.21.15.69:6807/3756025235","state":["exists","up"]},{"osd":1,"uuid":"c0c66930-4233-493f-b2bb-99c473994b59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6801","nonce":3166305213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6803","nonce":3166305213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6807","nonce":3166305213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6805","nonce":3166305213}]},"public_addr":"172.21.15.146:6801/3166305213","cluster_addr":"172.21.15.146:6803/3166305213","heartbeat_back_addr":"172.21.15.146:6807/3166305213","heartbeat_front_addr":"172.21.15.146:6805/3166305213","state":["exists","up"]},{"osd":2,"uuid":"d4ef735b-9786-4cb9-83ef-7c3f0691a729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6801","nonce":1474319175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6803","nonce":1474319175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6807","nonce":1474319175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6805","nonce":1474319175}]},"public_addr":"172.21.15.152:6801/1474319175","cluster_addr":"172.21.15.152:6803/1474319175","heartbeat_back_addr":"172.21.15.152:6807/1474319175","heartbeat_front_addr":"172.21.15.152:6805/1474319175","state":["exists","up"]},{"osd":3,"uuid":"0ab85fb8-5bbb-4ffc-bddf-cae7644a64ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6811","nonce":1809071296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6813","nonce":1809071296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6817","nonce":1809071296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6815","nonce":1809071296}]},"public_addr":"172.21.15.69:6811/1809071296","cluster_addr":"172.21.15.69:6813/1809071296","heartbeat_back_addr":"172.21.15.69:6817/1809071296","heartbeat_front_addr":"172.21.15.69:6815/1809071296","state":["exists","up"]},{"osd":4,"uuid":"273e0592-bbbc-4188-8eeb-46a3879cf632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6809","nonce":1898520972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6811","nonce":1898520972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6815","nonce":1898520972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6813","nonce":1898520972}]},"public_addr":"172.21.15.146:6809/1898520972","cluster_addr":"172.21.15.146:6811/1898520972","heartbeat_back_addr":"172.21.15.146:6815/1898520972","heartbeat_front_addr":"172.21.15.146:6813/1898520972","state":["exists","up"]},{"osd":5,"uuid":"bbeae945-9fa2-48f9-a1d6-8bf18dcaef19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6809","nonce":1996555108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6811","nonce":1996555108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6815","nonce":1996555108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6813","nonce":1996555108}]},"public_addr":"172.21.15.152:6809/1996555108","cluster_addr":"172.21.15.152:6811/1996555108","heartbeat_back_addr":"172.21.15.152:6815/1996555108","heartbeat_front_addr":"172.21.15.152:6813/1996555108","state":["exists","up"]},{"osd":6,"uuid":"dbb61253-30bc-418e-bd75-118c80bbf931","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6819","nonce":491215762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6821","nonce":491215762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6825","nonce":491215762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6823","nonce":491215762}]},"public_addr":"172.21.15.69:6819/491215762","cluster_addr":"172.21.15.69:6821/491215762","heartbeat_back_addr":"172.21.15.69:6825/491215762","heartbeat_front_addr":"172.21.15.69:6823/491215762","state":["exists","up"]},{"osd":7,"uuid":"75f23854-e283-4e3e-94cf-7b4079d83876","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6817","nonce":2330294690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6819","nonce":2330294690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6823","nonce":2330294690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6821","nonce":2330294690}]},"public_addr":"172.21.15.146:6817/2330294690","cluster_addr":"172.21.15.146:6819/2330294690","heartbeat_back_addr":"172.21.15.146:6823/2330294690","heartbeat_front_addr":"172.21.15.146:6821/2330294690","state":["exists","up"]},{"osd":8,"uuid":"1aa1060c-0c14-4abd-be56-5ebfeb78759c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6817","nonce":950669381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6819","nonce":950669381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6823","nonce":950669381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6821","nonce":950669381}]},"public_addr":"172.21.15.152:6817/950669381","cluster_addr":"172.21.15.152:6819/950669381","heartbeat_back_addr":"172.21.15.152:6823/950669381","heartbeat_front_addr":"172.21.15.152:6821/950669381","state":["exists","up"]},{"osd":9,"uuid":"3f4bf800-4090-4138-9bb3-d001f13f450c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6827","nonce":2414186758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6829","nonce":2414186758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6833","nonce":2414186758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6831","nonce":2414186758}]},"public_addr":"172.21.15.69:6827/2414186758","cluster_addr":"172.21.15.69:6829/2414186758","heartbeat_back_addr":"172.21.15.69:6833/2414186758","heartbeat_front_addr":"172.21.15.69:6831/2414186758","state":["exists","up"]},{"osd":10,"uuid":"19e967e5-51ce-4506-80c9-0a95441baf99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6825","nonce":707256400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6827","nonce":707256400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6831","nonce":707256400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6829","nonce":707256400}]},"public_addr":"172.21.15.146:6825/707256400","cluster_addr":"172.21.15.146:6827/707256400","heartbeat_back_addr":"172.21.15.146:6831/707256400","heartbeat_front_addr":"172.21.15.146:6829/707256400","state":["exists","up"]},{"osd":11,"uuid":"7788c2de-6930-4808-be0f-f1fd1ae0cb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6825","nonce":2934336833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6827","nonce":2934336833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6831","nonce":2934336833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6829","nonce":2934336833}]},"public_addr":"172.21.15.152:6825/2934336833","cluster_addr":"172.21.15.152:6827/2934336833","heartbeat_back_addr":"172.21.15.152:6831/2934336833","heartbeat_front_addr":"172.21.15.152:6829/2934336833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:07.497448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:31.227494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:55.027978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:20.713437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:46.214704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:20:11.199983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:20:37.177070+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:02.951240+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:28.963680+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:56.095807+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:22:22.221744+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:22:48.929334+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1b","mappings":[{"from":5,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/1372727546":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6801/1097124842":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6800/1508198215":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3103465990":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3540132960":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3899912190":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6800/1097124842":"2024-04-26T11:16:31.688159+0000","172.21.15.69:0/1199907202":"2024-04-26T11:16:04.860792+0000","172.21.15.69:6801/1508198215":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/2879102529":"2024-04-26T11:16:31.688159+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:38:18.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.426+0000 7f7555a0c640 1 -- 172.21.15.69:0/28253011 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f75380afaf0 msgr2=0x7f75380b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:18.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.426+0000 7f7555a0c640 1 --2- 172.21.15.69:0/28253011 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f75380afaf0 0x7f75380b2000 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7544005fd0 tx=0x7f754400b2c0 comp rx=0 tx=0).stop 2024-04-25T11:38:18.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.426+0000 7f7555a0c640 1 -- 172.21.15.69:0/28253011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75501029e0 msgr2=0x7f75501990e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:18.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.426+0000 7f7555a0c640 1 --2- 172.21.15.69:0/28253011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75501029e0 0x7f75501990e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7540005e30 tx=0x7f75400039f0 comp rx=0 tx=0).stop 2024-04-25T11:38:18.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.426+0000 7f7555a0c640 1 -- 172.21.15.69:0/28253011 shutdown_connections 2024-04-25T11:38:18.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.426+0000 7f7555a0c640 1 --2- 172.21.15.69:0/28253011 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f75380afaf0 0x7f75380b2000 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:18.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.426+0000 7f7555a0c640 1 --2- 172.21.15.69:0/28253011 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7550108920 0x7f75501a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:18.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.426+0000 7f7555a0c640 1 --2- 172.21.15.69:0/28253011 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7550103c20 0x7f7550199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:18.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.426+0000 7f7555a0c640 1 --2- 172.21.15.69:0/28253011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75501029e0 0x7f75501990e0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:18.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.426+0000 7f7555a0c640 1 -- 172.21.15.69:0/28253011 >> 172.21.15.69:0/28253011 conn(0x7f75500fdbf0 msgr2=0x7f7550104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:18.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.426+0000 7f7555a0c640 1 -- 172.21.15.69:0/28253011 shutdown_connections 2024-04-25T11:38:18.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.426+0000 7f7555a0c640 1 -- 172.21.15.69:0/28253011 wait complete. 2024-04-25T11:38:18.424 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:18 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/1607143969' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:38:18.424 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:18 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/1410470692' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T11:38:18.424 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:18 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/438221288' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T11:38:18.425 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:18:57.781263+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:23:38.685110+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:23:39.943803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T11:23:43.650521+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T11:38:18.426 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:38:18.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.549+0000 7f9d87c6e640 1 -- 172.21.15.69:0/1236061615 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d80102a00 msgr2=0x7f9d80103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:18.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.549+0000 7f9d87c6e640 1 --2- 172.21.15.69:0/1236061615 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d80102a00 0x7f9d80103700 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f9d78009ca0 tx=0x7f9d7802a8c0 comp rx=0 tx=0).stop 2024-04-25T11:38:18.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.549+0000 7f9d87c6e640 1 -- 172.21.15.69:0/1236061615 shutdown_connections 2024-04-25T11:38:18.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.549+0000 7f9d87c6e640 1 --2- 172.21.15.69:0/1236061615 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d80108940 0x7f9d8010ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:18.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.549+0000 7f9d87c6e640 1 --2- 172.21.15.69:0/1236061615 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d80103c40 0x7f9d80104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:18.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.549+0000 7f9d87c6e640 1 --2- 172.21.15.69:0/1236061615 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d80102a00 0x7f9d80103700 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:18.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.549+0000 7f9d87c6e640 1 -- 172.21.15.69:0/1236061615 >> 172.21.15.69:0/1236061615 conn(0x7f9d800fdbf0 msgr2=0x7f9d801000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:18.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.549+0000 7f9d87c6e640 1 -- 172.21.15.69:0/1236061615 shutdown_connections 2024-04-25T11:38:18.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.549+0000 7f9d87c6e640 1 -- 172.21.15.69:0/1236061615 wait complete. 2024-04-25T11:38:18.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.550+0000 7f9d87c6e640 1 Processor -- start 2024-04-25T11:38:18.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.550+0000 7f9d87c6e640 1 -- start start 2024-04-25T11:38:18.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.550+0000 7f9d87c6e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d80102a00 0x7f9d80075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:18.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.550+0000 7f9d87c6e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d80103c40 0x7f9d8019eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:18.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.550+0000 7f9d87c6e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d80108940 0x7f9d801a1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:18.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.550+0000 7f9d87c6e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8010e3e0 con 0x7f9d80108940 2024-04-25T11:38:18.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.550+0000 7f9d87c6e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8010e260 con 0x7f9d80103c40 2024-04-25T11:38:18.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.550+0000 7f9d87c6e640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8010e560 con 0x7f9d80102a00 2024-04-25T11:38:18.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.550+0000 7f9d8620b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d80108940 0x7f9d801a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:18.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.550+0000 7f9d8620b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d80108940 0x7f9d801a1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42156/0 (socket says 172.21.15.69:42156) 2024-04-25T11:38:18.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.550+0000 7f9d8620b640 1 -- 172.21.15.69:0/3266125680 learned_addr learned my addr 172.21.15.69:0/3266125680 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:38:18.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.550+0000 7f9d85a0a640 1 --2- 172.21.15.69:0/3266125680 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d80102a00 0x7f9d80075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:18.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.550+0000 7f9d85209640 1 --2- 172.21.15.69:0/3266125680 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d80103c40 0x7f9d8019eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:18.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.550+0000 7f9d8620b640 1 -- 172.21.15.69:0/3266125680 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d80102a00 msgr2=0x7f9d80075860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:18.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.550+0000 7f9d8620b640 1 --2- 172.21.15.69:0/3266125680 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d80102a00 0x7f9d80075860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:18.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.550+0000 7f9d8620b640 1 -- 172.21.15.69:0/3266125680 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d80103c40 msgr2=0x7f9d8019eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:18.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.550+0000 7f9d8620b640 1 --2- 172.21.15.69:0/3266125680 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d80103c40 0x7f9d8019eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:18.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.550+0000 7f9d8620b640 1 -- 172.21.15.69:0/3266125680 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d801a1be0 con 0x7f9d80108940 2024-04-25T11:38:18.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.550+0000 7f9d85209640 1 --2- 172.21.15.69:0/3266125680 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d80103c40 0x7f9d8019eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:18.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.551+0000 7f9d8620b640 1 --2- 172.21.15.69:0/3266125680 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d80108940 0x7f9d801a1450 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f9d7000c970 tx=0x7f9d7000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:18.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.551+0000 7f9d7effd640 1 -- 172.21.15.69:0/3266125680 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d70007bb0 con 0x7f9d80108940 2024-04-25T11:38:18.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.551+0000 7f9d7effd640 1 -- 172.21.15.69:0/3266125680 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d70007d50 con 0x7f9d80108940 2024-04-25T11:38:18.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.551+0000 7f9d7effd640 1 -- 172.21.15.69:0/3266125680 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d70018660 con 0x7f9d80108940 2024-04-25T11:38:18.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.551+0000 7f9d87c6e640 1 -- 172.21.15.69:0/3266125680 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d801a1ed0 con 0x7f9d80108940 2024-04-25T11:38:18.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.551+0000 7f9d87c6e640 1 -- 172.21.15.69:0/3266125680 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d801a22e0 con 0x7f9d80108940 2024-04-25T11:38:18.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.552+0000 7f9d87c6e640 1 -- 172.21.15.69:0/3266125680 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d80107750 con 0x7f9d80108940 2024-04-25T11:38:18.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.553+0000 7f9d7effd640 1 -- 172.21.15.69:0/3266125680 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f9d7000f970 con 0x7f9d80108940 2024-04-25T11:38:18.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.554+0000 7f9d7effd640 1 --2- 172.21.15.69:0/3266125680 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f9d680afaf0 0x7f9d680b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:18.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.554+0000 7f9d85a0a640 1 --2- 172.21.15.69:0/3266125680 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f9d680afaf0 0x7f9d680b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:18.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.554+0000 7f9d7effd640 1 -- 172.21.15.69:0/3266125680 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f9d700df270 con 0x7f9d80108940 2024-04-25T11:38:18.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.554+0000 7f9d85a0a640 1 --2- 172.21.15.69:0/3266125680 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f9d680afaf0 0x7f9d680b2000 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9d78006010 tx=0x7f9d780318e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:18.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.557+0000 7f9d7effd640 1 -- 172.21.15.69:0/3266125680 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9d70053ff0 con 0x7f9d80108940 2024-04-25T11:38:18.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:18 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/1607143969' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:38:18.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:18 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/1410470692' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T11:38:18.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:18 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/438221288' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T11:38:18.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.735+0000 7f9d87c6e640 1 -- 172.21.15.69:0/3266125680 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9d8002ddb0 con 0x7f9d80108940 2024-04-25T11:38:18.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.736+0000 7f9d7effd640 1 -- 172.21.15.69:0/3266125680 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7f9d70014030 con 0x7f9d80108940 2024-04-25T11:38:18.723 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-04-25T11:38:18.724 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:18 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/1607143969' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:38:18.724 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:18 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/1410470692' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T11:38:18.724 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:18 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/438221288' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T11:38:18.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.738+0000 7f9d87c6e640 1 -- 172.21.15.69:0/3266125680 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f9d680afaf0 msgr2=0x7f9d680b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:18.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.738+0000 7f9d87c6e640 1 --2- 172.21.15.69:0/3266125680 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f9d680afaf0 0x7f9d680b2000 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9d78006010 tx=0x7f9d780318e0 comp rx=0 tx=0).stop 2024-04-25T11:38:18.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.738+0000 7f9d87c6e640 1 -- 172.21.15.69:0/3266125680 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d80108940 msgr2=0x7f9d801a1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:18.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.738+0000 7f9d87c6e640 1 --2- 172.21.15.69:0/3266125680 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d80108940 0x7f9d801a1450 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f9d7000c970 tx=0x7f9d7000ce30 comp rx=0 tx=0).stop 2024-04-25T11:38:18.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.738+0000 7f9d87c6e640 1 -- 172.21.15.69:0/3266125680 shutdown_connections 2024-04-25T11:38:18.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.738+0000 7f9d87c6e640 1 --2- 172.21.15.69:0/3266125680 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f9d680afaf0 0x7f9d680b2000 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:18.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.738+0000 7f9d87c6e640 1 --2- 172.21.15.69:0/3266125680 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d80108940 0x7f9d801a1450 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:18.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.738+0000 7f9d87c6e640 1 --2- 172.21.15.69:0/3266125680 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9d80103c40 0x7f9d8019eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:18.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.738+0000 7f9d87c6e640 1 --2- 172.21.15.69:0/3266125680 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d80102a00 0x7f9d80075860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:18.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.738+0000 7f9d87c6e640 1 -- 172.21.15.69:0/3266125680 >> 172.21.15.69:0/3266125680 conn(0x7f9d800fdbf0 msgr2=0x7f9d80104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:18.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.738+0000 7f9d87c6e640 1 -- 172.21.15.69:0/3266125680 shutdown_connections 2024-04-25T11:38:18.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.738+0000 7f9d87c6e640 1 -- 172.21.15.69:0/3266125680 wait complete. 2024-04-25T11:38:18.737 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:38:18.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.860+0000 7f80da96c640 1 -- 172.21.15.69:0/2409260225 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80d41050a0 msgr2=0x7f80d41054b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:18.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.860+0000 7f80da96c640 1 --2- 172.21.15.69:0/2409260225 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80d41050a0 0x7f80d41054b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f80c4009900 tx=0x7f80c402a410 comp rx=0 tx=0).stop 2024-04-25T11:38:18.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.860+0000 7f80da96c640 1 -- 172.21.15.69:0/2409260225 shutdown_connections 2024-04-25T11:38:18.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.860+0000 7f80da96c640 1 --2- 172.21.15.69:0/2409260225 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f80d4102250 0x7f80d410bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:18.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.860+0000 7f80da96c640 1 --2- 172.21.15.69:0/2409260225 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80d4101750 0x7f80d4101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:18.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.860+0000 7f80da96c640 1 --2- 172.21.15.69:0/2409260225 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80d41050a0 0x7f80d41054b0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:18.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.860+0000 7f80da96c640 1 -- 172.21.15.69:0/2409260225 >> 172.21.15.69:0/2409260225 conn(0x7f80d40fd350 msgr2=0x7f80d40ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:18.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.860+0000 7f80da96c640 1 -- 172.21.15.69:0/2409260225 shutdown_connections 2024-04-25T11:38:18.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.860+0000 7f80da96c640 1 -- 172.21.15.69:0/2409260225 wait complete. 2024-04-25T11:38:18.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.860+0000 7f80da96c640 1 Processor -- start 2024-04-25T11:38:18.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.860+0000 7f80da96c640 1 -- start start 2024-04-25T11:38:18.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.861+0000 7f80da96c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f80d4101750 0x7f80d419a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:18.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.861+0000 7f80da96c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80d4102250 0x7f80d419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:18.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.861+0000 7f80da96c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80d41050a0 0x7f80d41a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:18.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.861+0000 7f80da96c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f80d410f520 con 0x7f80d4102250 2024-04-25T11:38:18.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.861+0000 7f80da96c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f80d410f3a0 con 0x7f80d41050a0 2024-04-25T11:38:18.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.861+0000 7f80da96c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f80d410f6a0 con 0x7f80d4101750 2024-04-25T11:38:18.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.861+0000 7f80d37fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80d4102250 0x7f80d419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:18.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.861+0000 7f80d37fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80d4102250 0x7f80d419a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42172/0 (socket says 172.21.15.69:42172) 2024-04-25T11:38:18.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.861+0000 7f80d37fe640 1 -- 172.21.15.69:0/2638663445 learned_addr learned my addr 172.21.15.69:0/2638663445 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:38:18.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.861+0000 7f80d8f09640 1 --2- 172.21.15.69:0/2638663445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80d41050a0 0x7f80d41a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:18.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.861+0000 7f80d3fff640 1 --2- 172.21.15.69:0/2638663445 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f80d4101750 0x7f80d419a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:18.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.861+0000 7f80d37fe640 1 -- 172.21.15.69:0/2638663445 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f80d4101750 msgr2=0x7f80d419a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:18.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.861+0000 7f80d37fe640 1 --2- 172.21.15.69:0/2638663445 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f80d4101750 0x7f80d419a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:18.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.861+0000 7f80d37fe640 1 -- 172.21.15.69:0/2638663445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80d41050a0 msgr2=0x7f80d41a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:18.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.861+0000 7f80d37fe640 1 --2- 172.21.15.69:0/2638663445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80d41050a0 0x7f80d41a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:18.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.861+0000 7f80d37fe640 1 -- 172.21.15.69:0/2638663445 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80d41a1c50 con 0x7f80d4102250 2024-04-25T11:38:18.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.861+0000 7f80d3fff640 1 --2- 172.21.15.69:0/2638663445 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f80d4101750 0x7f80d419a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:38:18.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.861+0000 7f80d37fe640 1 --2- 172.21.15.69:0/2638663445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80d4102250 0x7f80d419a700 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f80c800c920 tx=0x7f80c800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:18.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.861+0000 7f80d17fa640 1 -- 172.21.15.69:0/2638663445 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c8007c70 con 0x7f80d4102250 2024-04-25T11:38:18.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.861+0000 7f80d17fa640 1 -- 172.21.15.69:0/2638663445 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80c8007e10 con 0x7f80d4102250 2024-04-25T11:38:18.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.861+0000 7f80da96c640 1 -- 172.21.15.69:0/2638663445 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80d41a1f40 con 0x7f80d4102250 2024-04-25T11:38:18.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.861+0000 7f80d17fa640 1 -- 172.21.15.69:0/2638663445 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c8018710 con 0x7f80d4102250 2024-04-25T11:38:18.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.861+0000 7f80da96c640 1 -- 172.21.15.69:0/2638663445 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80d41a2420 con 0x7f80d4102250 2024-04-25T11:38:18.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.862+0000 7f80da96c640 1 -- 172.21.15.69:0/2638663445 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80d4108a30 con 0x7f80d4102250 2024-04-25T11:38:18.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.864+0000 7f80d17fa640 1 -- 172.21.15.69:0/2638663445 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f80c800f450 con 0x7f80d4102250 2024-04-25T11:38:18.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.865+0000 7f80d17fa640 1 --2- 172.21.15.69:0/2638663445 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f80bc0afaf0 0x7f80bc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:18.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.865+0000 7f80d3fff640 1 --2- 172.21.15.69:0/2638663445 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f80bc0afaf0 0x7f80bc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:18.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.865+0000 7f80d17fa640 1 -- 172.21.15.69:0/2638663445 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f80c80df3f0 con 0x7f80d4102250 2024-04-25T11:38:18.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.865+0000 7f80d3fff640 1 --2- 172.21.15.69:0/2638663445 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f80bc0afaf0 0x7f80bc0b2000 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f80c40057c0 tx=0x7f80c4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:18.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:18.868+0000 7f80d17fa640 1 -- 172.21.15.69:0/2638663445 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f80c8014030 con 0x7f80d4102250 2024-04-25T11:38:19.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.046+0000 7f80da96c640 1 -- 172.21.15.69:0/2638663445 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f80d40637d0 con 0x7f80d4102250 2024-04-25T11:38:19.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.047+0000 7f80d17fa640 1 -- 172.21.15.69:0/2638663445 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) ==== 104+0+11 (secure 0 0 0) 0x7f80c80a7c90 con 0x7f80d4102250 2024-04-25T11:38:19.034 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-25T11:38:19.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.049+0000 7f80da96c640 1 -- 172.21.15.69:0/2638663445 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f80bc0afaf0 msgr2=0x7f80bc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:19.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.049+0000 7f80da96c640 1 --2- 172.21.15.69:0/2638663445 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f80bc0afaf0 0x7f80bc0b2000 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f80c40057c0 tx=0x7f80c4000bc0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.049+0000 7f80da96c640 1 -- 172.21.15.69:0/2638663445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80d4102250 msgr2=0x7f80d419a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:19.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.049+0000 7f80da96c640 1 --2- 172.21.15.69:0/2638663445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80d4102250 0x7f80d419a700 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f80c800c920 tx=0x7f80c800cde0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.049+0000 7f80da96c640 1 -- 172.21.15.69:0/2638663445 shutdown_connections 2024-04-25T11:38:19.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.049+0000 7f80da96c640 1 --2- 172.21.15.69:0/2638663445 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f80bc0afaf0 0x7f80bc0b2000 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.049+0000 7f80da96c640 1 --2- 172.21.15.69:0/2638663445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f80d41050a0 0x7f80d41a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.049+0000 7f80da96c640 1 --2- 172.21.15.69:0/2638663445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f80d4102250 0x7f80d419a700 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.049+0000 7f80da96c640 1 --2- 172.21.15.69:0/2638663445 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f80d4101750 0x7f80d419a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.049+0000 7f80da96c640 1 -- 172.21.15.69:0/2638663445 >> 172.21.15.69:0/2638663445 conn(0x7f80d40fd350 msgr2=0x7f80d4105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:19.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.049+0000 7f80da96c640 1 -- 172.21.15.69:0/2638663445 shutdown_connections 2024-04-25T11:38:19.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.049+0000 7f80da96c640 1 -- 172.21.15.69:0/2638663445 wait complete. 2024-04-25T11:38:19.047 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:38:19.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.170+0000 7f7b6517a640 1 -- 172.21.15.69:0/2402170175 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b60103c30 msgr2=0x7f7b60104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:19.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.170+0000 7f7b6517a640 1 --2- 172.21.15.69:0/2402170175 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b60103c30 0x7f7b60104040 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f7b54009900 tx=0x7f7b5402a410 comp rx=0 tx=0).stop 2024-04-25T11:38:19.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.170+0000 7f7b6517a640 1 -- 172.21.15.69:0/2402170175 shutdown_connections 2024-04-25T11:38:19.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.170+0000 7f7b6517a640 1 --2- 172.21.15.69:0/2402170175 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b60108930 0x7f7b6010ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.170+0000 7f7b6517a640 1 --2- 172.21.15.69:0/2402170175 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b60103c30 0x7f7b60104040 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.170+0000 7f7b6517a640 1 --2- 172.21.15.69:0/2402170175 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b601029f0 0x7f7b601036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.170+0000 7f7b6517a640 1 -- 172.21.15.69:0/2402170175 >> 172.21.15.69:0/2402170175 conn(0x7f7b600fdc00 msgr2=0x7f7b60100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:19.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.171+0000 7f7b6517a640 1 -- 172.21.15.69:0/2402170175 shutdown_connections 2024-04-25T11:38:19.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.171+0000 7f7b6517a640 1 -- 172.21.15.69:0/2402170175 wait complete. 2024-04-25T11:38:19.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.171+0000 7f7b6517a640 1 Processor -- start 2024-04-25T11:38:19.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.171+0000 7f7b6517a640 1 -- start start 2024-04-25T11:38:19.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.172+0000 7f7b6517a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b601029f0 0x7f7b6019a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:19.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.172+0000 7f7b6517a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b60103c30 0x7f7b6019a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:19.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.172+0000 7f7b6517a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b60108930 0x7f7b601a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:19.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.172+0000 7f7b6517a640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6010e410 con 0x7f7b60108930 2024-04-25T11:38:19.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.172+0000 7f7b6517a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6010e290 con 0x7f7b60103c30 2024-04-25T11:38:19.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.172+0000 7f7b6517a640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6010e590 con 0x7f7b601029f0 2024-04-25T11:38:19.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.172+0000 7f7b5f59e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b60108930 0x7f7b601a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:19.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.172+0000 7f7b5f59e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b60108930 0x7f7b601a1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42182/0 (socket says 172.21.15.69:42182) 2024-04-25T11:38:19.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.172+0000 7f7b5f59e640 1 -- 172.21.15.69:0/1153120795 learned_addr learned my addr 172.21.15.69:0/1153120795 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:38:19.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.172+0000 7f7b5e59c640 1 --2- 172.21.15.69:0/1153120795 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b60103c30 0x7f7b6019a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:19.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.172+0000 7f7b5ed9d640 1 --2- 172.21.15.69:0/1153120795 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b601029f0 0x7f7b6019a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:19.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.172+0000 7f7b5f59e640 1 -- 172.21.15.69:0/1153120795 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b601029f0 msgr2=0x7f7b6019a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:19.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.172+0000 7f7b5f59e640 1 --2- 172.21.15.69:0/1153120795 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b601029f0 0x7f7b6019a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.172+0000 7f7b5f59e640 1 -- 172.21.15.69:0/1153120795 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b60103c30 msgr2=0x7f7b6019a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:19.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.172+0000 7f7b5f59e640 1 --2- 172.21.15.69:0/1153120795 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b60103c30 0x7f7b6019a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.172+0000 7f7b5f59e640 1 -- 172.21.15.69:0/1153120795 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b601a1cd0 con 0x7f7b60108930 2024-04-25T11:38:19.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.172+0000 7f7b5e59c640 1 --2- 172.21.15.69:0/1153120795 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b60103c30 0x7f7b6019a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:38:19.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.172+0000 7f7b5f59e640 1 --2- 172.21.15.69:0/1153120795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b60108930 0x7f7b601a1660 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7b58009fd0 tx=0x7f7b5800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:19.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.172+0000 7f7b47fff640 1 -- 172.21.15.69:0/1153120795 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b58017070 con 0x7f7b60108930 2024-04-25T11:38:19.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.173+0000 7f7b6517a640 1 -- 172.21.15.69:0/1153120795 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b601a1fc0 con 0x7f7b60108930 2024-04-25T11:38:19.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.173+0000 7f7b47fff640 1 -- 172.21.15.69:0/1153120795 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b58003e00 con 0x7f7b60108930 2024-04-25T11:38:19.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.173+0000 7f7b47fff640 1 -- 172.21.15.69:0/1153120795 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b58010460 con 0x7f7b60108930 2024-04-25T11:38:19.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.173+0000 7f7b6517a640 1 -- 172.21.15.69:0/1153120795 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b601a2420 con 0x7f7b60108930 2024-04-25T11:38:19.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.174+0000 7f7b6517a640 1 -- 172.21.15.69:0/1153120795 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b601a2680 con 0x7f7b60108930 2024-04-25T11:38:19.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.175+0000 7f7b47fff640 1 -- 172.21.15.69:0/1153120795 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f7b58003970 con 0x7f7b60108930 2024-04-25T11:38:19.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.175+0000 7f7b47fff640 1 --2- 172.21.15.69:0/1153120795 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f7b480afaf0 0x7f7b480b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:19.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.176+0000 7f7b5ed9d640 1 --2- 172.21.15.69:0/1153120795 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f7b480afaf0 0x7f7b480b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:19.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.176+0000 7f7b47fff640 1 -- 172.21.15.69:0/1153120795 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f7b580df220 con 0x7f7b60108930 2024-04-25T11:38:19.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.176+0000 7f7b5ed9d640 1 --2- 172.21.15.69:0/1153120795 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f7b480afaf0 0x7f7b480b2000 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7b601035f0 tx=0x7f7b50005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:19.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.180+0000 7f7b47fff640 1 -- 172.21.15.69:0/1153120795 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7b580a7a30 con 0x7f7b60108930 2024-04-25T11:38:19.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.359+0000 7f7b6517a640 1 -- 172.21.15.69:0/1153120795 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7b601a2850 con 0x7f7b60108930 2024-04-25T11:38:19.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.359+0000 7f7b47fff640 1 -- 172.21.15.69:0/1153120795 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) ==== 100+0+11 (secure 0 0 0) 0x7f7b580a7330 con 0x7f7b60108930 2024-04-25T11:38:19.347 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-04-25T11:38:19.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.361+0000 7f7b6517a640 1 -- 172.21.15.69:0/1153120795 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f7b480afaf0 msgr2=0x7f7b480b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:19.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.361+0000 7f7b6517a640 1 --2- 172.21.15.69:0/1153120795 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f7b480afaf0 0x7f7b480b2000 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7b601035f0 tx=0x7f7b50005f90 comp rx=0 tx=0).stop 2024-04-25T11:38:19.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.361+0000 7f7b6517a640 1 -- 172.21.15.69:0/1153120795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b60108930 msgr2=0x7f7b601a1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:19.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.361+0000 7f7b6517a640 1 --2- 172.21.15.69:0/1153120795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b60108930 0x7f7b601a1660 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7b58009fd0 tx=0x7f7b5800b260 comp rx=0 tx=0).stop 2024-04-25T11:38:19.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.361+0000 7f7b6517a640 1 -- 172.21.15.69:0/1153120795 shutdown_connections 2024-04-25T11:38:19.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.361+0000 7f7b6517a640 1 --2- 172.21.15.69:0/1153120795 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f7b480afaf0 0x7f7b480b2000 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.361+0000 7f7b6517a640 1 --2- 172.21.15.69:0/1153120795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b60108930 0x7f7b601a1660 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.361+0000 7f7b6517a640 1 --2- 172.21.15.69:0/1153120795 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7b60103c30 0x7f7b6019a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.361+0000 7f7b6517a640 1 --2- 172.21.15.69:0/1153120795 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b601029f0 0x7f7b6019a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.361+0000 7f7b6517a640 1 -- 172.21.15.69:0/1153120795 >> 172.21.15.69:0/1153120795 conn(0x7f7b600fdc00 msgr2=0x7f7b60109ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:19.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.361+0000 7f7b6517a640 1 -- 172.21.15.69:0/1153120795 shutdown_connections 2024-04-25T11:38:19.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.361+0000 7f7b6517a640 1 -- 172.21.15.69:0/1153120795 wait complete. 2024-04-25T11:38:19.360 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T11:38:19.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.483+0000 7f0243b7a640 1 -- 172.21.15.69:0/2877395910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f023c1029e0 msgr2=0x7f023c1036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:19.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.483+0000 7f0243b7a640 1 --2- 172.21.15.69:0/2877395910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f023c1029e0 0x7f023c1036e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0238009900 tx=0x7f023802a410 comp rx=0 tx=0).stop 2024-04-25T11:38:19.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.483+0000 7f0243b7a640 1 -- 172.21.15.69:0/2877395910 shutdown_connections 2024-04-25T11:38:19.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.483+0000 7f0243b7a640 1 --2- 172.21.15.69:0/2877395910 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f023c108920 0x7f023c10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.483+0000 7f0243b7a640 1 --2- 172.21.15.69:0/2877395910 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f023c103c20 0x7f023c104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.483+0000 7f0243b7a640 1 --2- 172.21.15.69:0/2877395910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f023c1029e0 0x7f023c1036e0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.483+0000 7f0243b7a640 1 -- 172.21.15.69:0/2877395910 >> 172.21.15.69:0/2877395910 conn(0x7f023c0fdbd0 msgr2=0x7f023c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:19.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.483+0000 7f0243b7a640 1 -- 172.21.15.69:0/2877395910 shutdown_connections 2024-04-25T11:38:19.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.483+0000 7f0243b7a640 1 -- 172.21.15.69:0/2877395910 wait complete. 2024-04-25T11:38:19.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.483+0000 7f0243b7a640 1 Processor -- start 2024-04-25T11:38:19.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.483+0000 7f0243b7a640 1 -- start start 2024-04-25T11:38:19.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.484+0000 7f0243b7a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f023c1029e0 0x7f023c19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:19.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.484+0000 7f0243b7a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f023c103c20 0x7f023c19a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:19.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.484+0000 7f0243b7a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f023c108920 0x7f023c1a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:19.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.484+0000 7f0243b7a640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f023c10e370 con 0x7f023c1029e0 2024-04-25T11:38:19.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.484+0000 7f0243b7a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f023c10e1f0 con 0x7f023c103c20 2024-04-25T11:38:19.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.484+0000 7f0243b7a640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f023c10e4f0 con 0x7f023c108920 2024-04-25T11:38:19.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.484+0000 7f0241916640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f023c1029e0 0x7f023c19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:19.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.484+0000 7f0241916640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f023c1029e0 0x7f023c19a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42210/0 (socket says 172.21.15.69:42210) 2024-04-25T11:38:19.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.484+0000 7f0241916640 1 -- 172.21.15.69:0/4123740137 learned_addr learned my addr 172.21.15.69:0/4123740137 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:38:19.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.484+0000 7f0241115640 1 --2- 172.21.15.69:0/4123740137 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f023c103c20 0x7f023c19a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:19.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.484+0000 7f0242117640 1 --2- 172.21.15.69:0/4123740137 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f023c108920 0x7f023c1a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:19.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.484+0000 7f0241916640 1 -- 172.21.15.69:0/4123740137 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f023c108920 msgr2=0x7f023c1a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:19.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.484+0000 7f0241916640 1 --2- 172.21.15.69:0/4123740137 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f023c108920 0x7f023c1a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.484+0000 7f0241916640 1 -- 172.21.15.69:0/4123740137 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f023c103c20 msgr2=0x7f023c19a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:19.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.484+0000 7f0241916640 1 --2- 172.21.15.69:0/4123740137 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f023c103c20 0x7f023c19a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.484+0000 7f0241916640 1 -- 172.21.15.69:0/4123740137 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f023c1a1fb0 con 0x7f023c1029e0 2024-04-25T11:38:19.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.484+0000 7f0242117640 1 --2- 172.21.15.69:0/4123740137 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f023c108920 0x7f023c1a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:19.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.484+0000 7f0241115640 1 --2- 172.21.15.69:0/4123740137 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f023c103c20 0x7f023c19a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:19.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.484+0000 7f0241916640 1 --2- 172.21.15.69:0/4123740137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f023c1029e0 0x7f023c19a470 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f0238005760 tx=0x7f0238004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:19.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.484+0000 7f0232ffd640 1 -- 172.21.15.69:0/4123740137 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f023803f030 con 0x7f023c1029e0 2024-04-25T11:38:19.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.484+0000 7f0232ffd640 1 -- 172.21.15.69:0/4123740137 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0238004440 con 0x7f023c1029e0 2024-04-25T11:38:19.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.484+0000 7f0232ffd640 1 -- 172.21.15.69:0/4123740137 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0238035070 con 0x7f023c1029e0 2024-04-25T11:38:19.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.484+0000 7f0243b7a640 1 -- 172.21.15.69:0/4123740137 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f023c1a2240 con 0x7f023c1029e0 2024-04-25T11:38:19.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.484+0000 7f0243b7a640 1 -- 172.21.15.69:0/4123740137 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f023c1a26e0 con 0x7f023c1029e0 2024-04-25T11:38:19.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.486+0000 7f0243b7a640 1 -- 172.21.15.69:0/4123740137 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f023c06c2a0 con 0x7f023c1029e0 2024-04-25T11:38:19.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.487+0000 7f0232ffd640 1 -- 172.21.15.69:0/4123740137 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f0238042030 con 0x7f023c1029e0 2024-04-25T11:38:19.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.488+0000 7f0232ffd640 1 --2- 172.21.15.69:0/4123740137 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f02280afb70 0x7f02280b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:19.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.488+0000 7f0241115640 1 --2- 172.21.15.69:0/4123740137 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f02280afb70 0x7f02280b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:19.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.488+0000 7f0232ffd640 1 -- 172.21.15.69:0/4123740137 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f02380ff850 con 0x7f023c1029e0 2024-04-25T11:38:19.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.488+0000 7f0241115640 1 --2- 172.21.15.69:0/4123740137 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f02280afb70 0x7f02280b2080 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f023c19b510 tx=0x7f022c005e90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:19.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.492+0000 7f0232ffd640 1 -- 172.21.15.69:0/4123740137 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f02380c8060 con 0x7f023c1029e0 2024-04-25T11:38:19.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:19 smithi146 ceph-mon[32821]: pgmap v727: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:38:19.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:19 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/28253011' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:38:19.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:19 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/3266125680' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:38:19.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:19 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/2638663445' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:38:19.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.667+0000 7f0243b7a640 1 -- 172.21.15.69:0/4123740137 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f023c1a2940 con 0x7f023c1029e0 2024-04-25T11:38:19.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.667+0000 7f0232ffd640 1 -- 172.21.15.69:0/4123740137 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v84) ==== 103+0+11 (secure 0 0 0) 0x7f02380d00e0 con 0x7f023c1029e0 2024-04-25T11:38:19.655 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-04-25T11:38:19.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.669+0000 7f0243b7a640 1 -- 172.21.15.69:0/4123740137 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f02280afb70 msgr2=0x7f02280b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:19.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.669+0000 7f0243b7a640 1 --2- 172.21.15.69:0/4123740137 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f02280afb70 0x7f02280b2080 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f023c19b510 tx=0x7f022c005e90 comp rx=0 tx=0).stop 2024-04-25T11:38:19.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.669+0000 7f0243b7a640 1 -- 172.21.15.69:0/4123740137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f023c1029e0 msgr2=0x7f023c19a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:19.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.669+0000 7f0243b7a640 1 --2- 172.21.15.69:0/4123740137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f023c1029e0 0x7f023c19a470 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f0238005760 tx=0x7f0238004040 comp rx=0 tx=0).stop 2024-04-25T11:38:19.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.669+0000 7f0243b7a640 1 -- 172.21.15.69:0/4123740137 shutdown_connections 2024-04-25T11:38:19.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.669+0000 7f0243b7a640 1 --2- 172.21.15.69:0/4123740137 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f02280afb70 0x7f02280b2080 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.669+0000 7f0243b7a640 1 --2- 172.21.15.69:0/4123740137 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f023c108920 0x7f023c1a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.669+0000 7f0243b7a640 1 --2- 172.21.15.69:0/4123740137 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f023c103c20 0x7f023c19a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.669+0000 7f0243b7a640 1 --2- 172.21.15.69:0/4123740137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f023c1029e0 0x7f023c19a470 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.669+0000 7f0243b7a640 1 -- 172.21.15.69:0/4123740137 >> 172.21.15.69:0/4123740137 conn(0x7f023c0fdbd0 msgr2=0x7f023c069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:19.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.669+0000 7f0243b7a640 1 -- 172.21.15.69:0/4123740137 shutdown_connections 2024-04-25T11:38:19.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.669+0000 7f0243b7a640 1 -- 172.21.15.69:0/4123740137 wait complete. 2024-04-25T11:38:19.667 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:19 smithi069 ceph-mon[29188]: pgmap v727: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:38:19.667 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:19 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/28253011' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:38:19.667 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:19 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/3266125680' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:38:19.667 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:19 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2638663445' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:38:19.669 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:38:19.724 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:19 smithi152 ceph-mon[32701]: pgmap v727: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-25T11:38:19.724 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:19 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/28253011' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:38:19.724 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:19 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/3266125680' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:38:19.724 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:19 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/2638663445' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:38:19.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.790+0000 7f83249f8640 1 -- 172.21.15.69:0/2563264314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8320104480 msgr2=0x7f832010abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:19.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.790+0000 7f83249f8640 1 --2- 172.21.15.69:0/2563264314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8320104480 0x7f832010abc0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f8318009ca0 tx=0x7f831802a8c0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.790+0000 7f83249f8640 1 -- 172.21.15.69:0/2563264314 shutdown_connections 2024-04-25T11:38:19.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.790+0000 7f83249f8640 1 --2- 172.21.15.69:0/2563264314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8320104480 0x7f832010abc0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.790+0000 7f83249f8640 1 --2- 172.21.15.69:0/2563264314 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8320103940 0x7f8320103df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.790+0000 7f83249f8640 1 --2- 172.21.15.69:0/2563264314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8320102f60 0x7f8320103370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.790+0000 7f83249f8640 1 -- 172.21.15.69:0/2563264314 >> 172.21.15.69:0/2563264314 conn(0x7f83200fd380 msgr2=0x7f83200ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:19.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.790+0000 7f83249f8640 1 -- 172.21.15.69:0/2563264314 shutdown_connections 2024-04-25T11:38:19.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.790+0000 7f83249f8640 1 -- 172.21.15.69:0/2563264314 wait complete. 2024-04-25T11:38:19.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.791+0000 7f83249f8640 1 Processor -- start 2024-04-25T11:38:19.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.791+0000 7f83249f8640 1 -- start start 2024-04-25T11:38:19.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.791+0000 7f83249f8640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8320102f60 0x7f832019a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:19.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.791+0000 7f83249f8640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8320103940 0x7f832019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:19.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.791+0000 7f83249f8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8320104480 0x7f83201a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:19.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.791+0000 7f83249f8640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f832010f550 con 0x7f8320103940 2024-04-25T11:38:19.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.791+0000 7f83249f8640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f832010f3d0 con 0x7f8320104480 2024-04-25T11:38:19.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.791+0000 7f83249f8640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f832010f6d0 con 0x7f8320102f60 2024-04-25T11:38:19.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.791+0000 7f831dd9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8320103940 0x7f832019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:19.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.791+0000 7f831dd9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8320103940 0x7f832019a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42228/0 (socket says 172.21.15.69:42228) 2024-04-25T11:38:19.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.791+0000 7f831dd9b640 1 -- 172.21.15.69:0/3305712365 learned_addr learned my addr 172.21.15.69:0/3305712365 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:38:19.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.792+0000 7f831ed9d640 1 --2- 172.21.15.69:0/3305712365 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8320104480 0x7f83201a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:19.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.792+0000 7f831e59c640 1 --2- 172.21.15.69:0/3305712365 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8320102f60 0x7f832019a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:19.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.792+0000 7f831dd9b640 1 -- 172.21.15.69:0/3305712365 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8320102f60 msgr2=0x7f832019a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:19.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.792+0000 7f831dd9b640 1 --2- 172.21.15.69:0/3305712365 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8320102f60 0x7f832019a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.792+0000 7f831dd9b640 1 -- 172.21.15.69:0/3305712365 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8320104480 msgr2=0x7f83201a15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:19.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.792+0000 7f831dd9b640 1 --2- 172.21.15.69:0/3305712365 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8320104480 0x7f83201a15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.792+0000 7f831dd9b640 1 -- 172.21.15.69:0/3305712365 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83201a1c10 con 0x7f8320103940 2024-04-25T11:38:19.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.792+0000 7f831dd9b640 1 --2- 172.21.15.69:0/3305712365 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8320103940 0x7f832019a610 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f831400d950 tx=0x7f831400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:19.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.792+0000 7f830f7fe640 1 -- 172.21.15.69:0/3305712365 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8314017070 con 0x7f8320103940 2024-04-25T11:38:19.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.792+0000 7f830f7fe640 1 -- 172.21.15.69:0/3305712365 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8314009c30 con 0x7f8320103940 2024-04-25T11:38:19.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.792+0000 7f83249f8640 1 -- 172.21.15.69:0/3305712365 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83201a1f00 con 0x7f8320103940 2024-04-25T11:38:19.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.792+0000 7f830f7fe640 1 -- 172.21.15.69:0/3305712365 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831400f620 con 0x7f8320103940 2024-04-25T11:38:19.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.792+0000 7f83249f8640 1 -- 172.21.15.69:0/3305712365 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83201a2370 con 0x7f8320103940 2024-04-25T11:38:19.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.793+0000 7f83249f8640 1 -- 172.21.15.69:0/3305712365 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8320109910 con 0x7f8320103940 2024-04-25T11:38:19.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.794+0000 7f830f7fe640 1 -- 172.21.15.69:0/3305712365 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f8314010460 con 0x7f8320103940 2024-04-25T11:38:19.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.795+0000 7f830f7fe640 1 --2- 172.21.15.69:0/3305712365 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f83080afaf0 0x7f83080b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:19.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.795+0000 7f831e59c640 1 --2- 172.21.15.69:0/3305712365 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f83080afaf0 0x7f83080b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:19.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.795+0000 7f830f7fe640 1 -- 172.21.15.69:0/3305712365 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f83140dec00 con 0x7f8320103940 2024-04-25T11:38:19.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.796+0000 7f831e59c640 1 --2- 172.21.15.69:0/3305712365 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f83080afaf0 0x7f83080b2000 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f832006b4e0 tx=0x7f8310005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:19.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.799+0000 7f830f7fe640 1 -- 172.21.15.69:0/3305712365 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f83140a7410 con 0x7f8320103940 2024-04-25T11:38:19.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.990+0000 7f83249f8640 1 -- 172.21.15.69:0/3305712365 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f832002dda0 con 0x7f8320103940 2024-04-25T11:38:19.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.990+0000 7f830f7fe640 1 -- 172.21.15.69:0/3305712365 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v84) ==== 81+0+592 (secure 0 0 0) 0x7f8314009de0 con 0x7f8320103940 2024-04-25T11:38:19.978 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/1372727546 2024-04-26T11:16:31.688159+0000 2024-04-25T11:38:19.978 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6801/1097124842 2024-04-26T11:16:31.688159+0000 2024-04-25T11:38:19.978 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6800/1508198215 2024-04-26T11:16:04.860792+0000 2024-04-25T11:38:19.978 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/3103465990 2024-04-26T11:16:04.860792+0000 2024-04-25T11:38:19.978 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/3540132960 2024-04-26T11:16:04.860792+0000 2024-04-25T11:38:19.978 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/3899912190 2024-04-26T11:16:31.688159+0000 2024-04-25T11:38:19.979 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6800/1097124842 2024-04-26T11:16:31.688159+0000 2024-04-25T11:38:19.979 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/1199907202 2024-04-26T11:16:04.860792+0000 2024-04-25T11:38:19.979 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6801/1508198215 2024-04-26T11:16:04.860792+0000 2024-04-25T11:38:19.979 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/2879102529 2024-04-26T11:16:31.688159+0000 2024-04-25T11:38:19.980 INFO:teuthology.orchestra.run.smithi069.stderr:listed 10 entries 2024-04-25T11:38:19.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.992+0000 7f83249f8640 1 -- 172.21.15.69:0/3305712365 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f83080afaf0 msgr2=0x7f83080b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:19.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.992+0000 7f83249f8640 1 --2- 172.21.15.69:0/3305712365 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f83080afaf0 0x7f83080b2000 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f832006b4e0 tx=0x7f8310005f90 comp rx=0 tx=0).stop 2024-04-25T11:38:19.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.992+0000 7f83249f8640 1 -- 172.21.15.69:0/3305712365 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8320103940 msgr2=0x7f832019a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:19.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.992+0000 7f83249f8640 1 --2- 172.21.15.69:0/3305712365 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8320103940 0x7f832019a610 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f831400d950 tx=0x7f831400de10 comp rx=0 tx=0).stop 2024-04-25T11:38:19.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.992+0000 7f83249f8640 1 -- 172.21.15.69:0/3305712365 shutdown_connections 2024-04-25T11:38:19.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.992+0000 7f83249f8640 1 --2- 172.21.15.69:0/3305712365 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f83080afaf0 0x7f83080b2000 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.992+0000 7f83249f8640 1 --2- 172.21.15.69:0/3305712365 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8320104480 0x7f83201a15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.992+0000 7f83249f8640 1 --2- 172.21.15.69:0/3305712365 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8320103940 0x7f832019a610 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.992+0000 7f83249f8640 1 --2- 172.21.15.69:0/3305712365 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8320102f60 0x7f832019a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:19.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.992+0000 7f83249f8640 1 -- 172.21.15.69:0/3305712365 >> 172.21.15.69:0/3305712365 conn(0x7f83200fd380 msgr2=0x7f8320106ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:19.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.992+0000 7f83249f8640 1 -- 172.21.15.69:0/3305712365 shutdown_connections 2024-04-25T11:38:19.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:19.992+0000 7f83249f8640 1 -- 172.21.15.69:0/3305712365 wait complete. 2024-04-25T11:38:19.992 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi069.front.sepia.ceph.com... 2024-04-25T11:38:19.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:38:19.993 DEBUG:teuthology.orchestra.run.smithi069:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T11:38:20.014 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi069.stderr:ceph-fuse[83193]: fuse finished with error 0 and tester_r 0 2024-04-25T11:38:20.016 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:38:20.592 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:20 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/1153120795' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:38:20.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:20 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/4123740137' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T11:38:20.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:20 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/3305712365' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T11:38:20.686 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:20 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/1153120795' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:38:20.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:20 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/4123740137' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T11:38:20.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:20 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/3305712365' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T11:38:20.725 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:20 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/1153120795' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:38:20.726 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:20 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/4123740137' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T11:38:20.726 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:20 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/3305712365' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T11:38:21.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:21 smithi146 ceph-mon[32821]: pgmap v728: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-25T11:38:21.686 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:21 smithi069 ceph-mon[29188]: pgmap v728: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-25T11:38:21.724 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:21 smithi152 ceph-mon[32701]: pgmap v728: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-25T11:38:23.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:23 smithi146 ceph-mon[32821]: pgmap v729: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T11:38:23.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:23 smithi069 ceph-mon[29188]: pgmap v729: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T11:38:23.724 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:23 smithi152 ceph-mon[32701]: pgmap v729: 129 pgs: 129 active+clean; 7.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-25T11:38:25.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:25 smithi146 ceph-mon[32821]: pgmap v730: 129 pgs: 129 active+clean; 7.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 682 B/s wr, 11 op/s 2024-04-25T11:38:25.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:25 smithi069 ceph-mon[29188]: pgmap v730: 129 pgs: 129 active+clean; 7.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 682 B/s wr, 11 op/s 2024-04-25T11:38:25.724 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:25 smithi152 ceph-mon[32701]: pgmap v730: 129 pgs: 129 active+clean; 7.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 682 B/s wr, 11 op/s 2024-04-25T11:38:26.118 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi069.front.sepia.ceph.com 2024-04-25T11:38:26.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:38:26.119 DEBUG:teuthology.orchestra.run.smithi069:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:38:26.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:38:26.138 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list 2024-04-25T11:38:26.196 INFO:teuthology.orchestra.run.smithi069.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T11:38:26.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:38:26.197 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:38:26.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:38:26.267 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link delete ceph-brx 2024-04-25T11:38:26.366 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-25T11:38:26.376 INFO:tasks.cephadm:Teardown begin 2024-04-25T11:38:26.377 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T11:38:26.377 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:38:26.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.502+0000 7f11d9cf9640 1 -- 172.21.15.69:0/1094871263 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11d4103ae0 msgr2=0x7f11d41047e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:26.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.503+0000 7f11d9cf9640 1 --2- 172.21.15.69:0/1094871263 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11d4103ae0 0x7f11d41047e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f11c4009900 tx=0x7f11c402a410 comp rx=0 tx=0).stop 2024-04-25T11:38:26.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.503+0000 7f11d9cf9640 1 -- 172.21.15.69:0/1094871263 shutdown_connections 2024-04-25T11:38:26.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.503+0000 7f11d9cf9640 1 --2- 172.21.15.69:0/1094871263 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11d4109a20 0x7f11d410be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:26.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.503+0000 7f11d9cf9640 1 --2- 172.21.15.69:0/1094871263 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11d4104d20 0x7f11d4105130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:26.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.503+0000 7f11d9cf9640 1 --2- 172.21.15.69:0/1094871263 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11d4103ae0 0x7f11d41047e0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:26.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.503+0000 7f11d9cf9640 1 -- 172.21.15.69:0/1094871263 >> 172.21.15.69:0/1094871263 conn(0x7f11d40fecd0 msgr2=0x7f11d4101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:26.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.503+0000 7f11d9cf9640 1 -- 172.21.15.69:0/1094871263 shutdown_connections 2024-04-25T11:38:26.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.503+0000 7f11d9cf9640 1 -- 172.21.15.69:0/1094871263 wait complete. 2024-04-25T11:38:26.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.503+0000 7f11d9cf9640 1 Processor -- start 2024-04-25T11:38:26.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.503+0000 7f11d9cf9640 1 -- start start 2024-04-25T11:38:26.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.504+0000 7f11d9cf9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11d4103ae0 0x7f11d419a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:26.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.504+0000 7f11d9cf9640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11d4104d20 0x7f11d419a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:26.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.504+0000 7f11d9cf9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11d4109a20 0x7f11d41a15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:26.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.504+0000 7f11d9cf9640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f11d410f470 con 0x7f11d4109a20 2024-04-25T11:38:26.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.504+0000 7f11d9cf9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f11d410f2f0 con 0x7f11d4103ae0 2024-04-25T11:38:26.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.504+0000 7f11d9cf9640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f11d410f5f0 con 0x7f11d4104d20 2024-04-25T11:38:26.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.504+0000 7f11d3fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11d4109a20 0x7f11d41a15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:26.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.504+0000 7f11d3fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11d4109a20 0x7f11d41a15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59444/0 (socket says 172.21.15.69:59444) 2024-04-25T11:38:26.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.504+0000 7f11d3fff640 1 -- 172.21.15.69:0/326333146 learned_addr learned my addr 172.21.15.69:0/326333146 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:38:26.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.504+0000 7f11d37fe640 1 --2- 172.21.15.69:0/326333146 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11d4103ae0 0x7f11d419a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:26.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.504+0000 7f11d2ffd640 1 --2- 172.21.15.69:0/326333146 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11d4104d20 0x7f11d419a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:26.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.504+0000 7f11d3fff640 1 -- 172.21.15.69:0/326333146 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11d4104d20 msgr2=0x7f11d419a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:26.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.504+0000 7f11d3fff640 1 --2- 172.21.15.69:0/326333146 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11d4104d20 0x7f11d419a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:26.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.504+0000 7f11d3fff640 1 -- 172.21.15.69:0/326333146 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11d4103ae0 msgr2=0x7f11d419a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:26.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.504+0000 7f11d3fff640 1 --2- 172.21.15.69:0/326333146 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11d4103ae0 0x7f11d419a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:26.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.504+0000 7f11d3fff640 1 -- 172.21.15.69:0/326333146 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d41a1c60 con 0x7f11d4109a20 2024-04-25T11:38:26.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.504+0000 7f11d2ffd640 1 --2- 172.21.15.69:0/326333146 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11d4104d20 0x7f11d419a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:38:26.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.504+0000 7f11d3fff640 1 --2- 172.21.15.69:0/326333146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11d4109a20 0x7f11d41a15f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f11cc00cad0 tx=0x7f11cc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:26.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.504+0000 7f11d0ff9640 1 -- 172.21.15.69:0/326333146 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11cc00ae20 con 0x7f11d4109a20 2024-04-25T11:38:26.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.504+0000 7f11d0ff9640 1 -- 172.21.15.69:0/326333146 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f11cc005020 con 0x7f11d4109a20 2024-04-25T11:38:26.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.504+0000 7f11d9cf9640 1 -- 172.21.15.69:0/326333146 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11d41a1f50 con 0x7f11d4109a20 2024-04-25T11:38:26.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.504+0000 7f11d0ff9640 1 -- 172.21.15.69:0/326333146 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11cc00f6b0 con 0x7f11d4109a20 2024-04-25T11:38:26.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.505+0000 7f11d9cf9640 1 -- 172.21.15.69:0/326333146 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11d41a23b0 con 0x7f11d4109a20 2024-04-25T11:38:26.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.505+0000 7f11d9cf9640 1 -- 172.21.15.69:0/326333146 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11d4108830 con 0x7f11d4109a20 2024-04-25T11:38:26.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.507+0000 7f11d0ff9640 1 -- 172.21.15.69:0/326333146 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f11cc0051d0 con 0x7f11d4109a20 2024-04-25T11:38:26.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.507+0000 7f11d0ff9640 1 --2- 172.21.15.69:0/326333146 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f11bc0afb70 0x7f11bc0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:26.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.508+0000 7f11d37fe640 1 --2- 172.21.15.69:0/326333146 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f11bc0afb70 0x7f11bc0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:26.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.508+0000 7f11d0ff9640 1 -- 172.21.15.69:0/326333146 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f11cc0df300 con 0x7f11d4109a20 2024-04-25T11:38:26.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.508+0000 7f11d37fe640 1 --2- 172.21.15.69:0/326333146 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f11bc0afb70 0x7f11bc0b2080 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f11c4005760 tx=0x7f11c4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:26.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.511+0000 7f11d0ff9640 1 -- 172.21.15.69:0/326333146 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f11cc014030 con 0x7f11d4109a20 2024-04-25T11:38:26.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.727+0000 7f11d9cf9640 1 -- 172.21.15.69:0/326333146 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11d402db50 con 0x7f11d4109a20 2024-04-25T11:38:26.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.728+0000 7f11d0ff9640 1 -- 172.21.15.69:0/326333146 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11944 (secure 0 0 0) 0x7f11cc0a7b90 con 0x7f11d4109a20 2024-04-25T11:38:26.715 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:38:26.715 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3399424094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3399424094},{"type":"v1","addr":"172.21.15.69:6835","nonce":3399424094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1463217771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1463217771},{"type":"v1","addr":"172.21.15.69:6837","nonce":1463217771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T11:23:40.984930+0000","modified":"2024-04-25T11:26:07.156294+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24485,"mds_2":24475,"mds_3":24458,"mds_4":14625},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6839/966316001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":966316001},{"type":"v1","addr":"172.21.15.69:6839","nonce":966316001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.69:6841/2060225181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":2060225181},{"type":"v1","addr":"172.21.15.69:6841","nonce":2060225181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6833/4197482473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":4197482473},{"type":"v1","addr":"172.21.15.152:6833","nonce":4197482473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/4267885323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4267885323},{"type":"v1","addr":"172.21.15.146:6833","nonce":4267885323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6835/3188843362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3188843362},{"type":"v1","addr":"172.21.15.152:6835","nonce":3188843362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6837/563274982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":563274982},{"type":"v1","addr":"172.21.15.152:6837","nonce":563274982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.146:6835/1876533699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1876533699},{"type":"v1","addr":"172.21.15.146:6835","nonce":1876533699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.152:6839/3008615657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":3008615657},{"type":"v1","addr":"172.21.15.152:6839","nonce":3008615657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.146:6837/2730317134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2730317134},{"type":"v1","addr":"172.21.15.146:6837","nonce":2730317134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6839/2013283448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":2013283448},{"type":"v1","addr":"172.21.15.146:6839","nonce":2013283448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[14625,24458,24475,24485,24515]},"id":1}]} 2024-04-25T11:38:26.715 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 29 2024-04-25T11:38:26.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.730+0000 7f11d9cf9640 1 -- 172.21.15.69:0/326333146 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f11bc0afb70 msgr2=0x7f11bc0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:26.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.730+0000 7f11d9cf9640 1 --2- 172.21.15.69:0/326333146 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f11bc0afb70 0x7f11bc0b2080 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f11c4005760 tx=0x7f11c4000bc0 comp rx=0 tx=0).stop 2024-04-25T11:38:26.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.730+0000 7f11d9cf9640 1 -- 172.21.15.69:0/326333146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11d4109a20 msgr2=0x7f11d41a15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:26.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.730+0000 7f11d9cf9640 1 --2- 172.21.15.69:0/326333146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11d4109a20 0x7f11d41a15f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f11cc00cad0 tx=0x7f11cc00cf90 comp rx=0 tx=0).stop 2024-04-25T11:38:26.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.730+0000 7f11d9cf9640 1 -- 172.21.15.69:0/326333146 shutdown_connections 2024-04-25T11:38:26.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.730+0000 7f11d9cf9640 1 --2- 172.21.15.69:0/326333146 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f11bc0afb70 0x7f11bc0b2080 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:26.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.730+0000 7f11d9cf9640 1 --2- 172.21.15.69:0/326333146 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11d4109a20 0x7f11d41a15f0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:26.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.730+0000 7f11d9cf9640 1 --2- 172.21.15.69:0/326333146 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11d4104d20 0x7f11d419a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:26.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.730+0000 7f11d9cf9640 1 --2- 172.21.15.69:0/326333146 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11d4103ae0 0x7f11d419a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:26.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.730+0000 7f11d9cf9640 1 -- 172.21.15.69:0/326333146 >> 172.21.15.69:0/326333146 conn(0x7f11d40fecd0 msgr2=0x7f11d4105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:26.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.730+0000 7f11d9cf9640 1 -- 172.21.15.69:0/326333146 shutdown_connections 2024-04-25T11:38:26.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.730+0000 7f11d9cf9640 1 -- 172.21.15.69:0/326333146 wait complete. 2024-04-25T11:38:26.729 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:38:26.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.850+0000 7f5102a42640 1 -- 172.21.15.69:0/1196302403 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50fc1045d0 msgr2=0x7f50fc1069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:26.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.850+0000 7f5102a42640 1 --2- 172.21.15.69:0/1196302403 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50fc1045d0 0x7f50fc1069f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f50ec009900 tx=0x7f50ec02a410 comp rx=0 tx=0).stop 2024-04-25T11:38:26.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.850+0000 7f5102a42640 1 -- 172.21.15.69:0/1196302403 shutdown_connections 2024-04-25T11:38:26.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.850+0000 7f5102a42640 1 --2- 172.21.15.69:0/1196302403 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f50fc109890 0x7f50fc10bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:26.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.850+0000 7f5102a42640 1 --2- 172.21.15.69:0/1196302403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f50fc106f30 0x7f50fc109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:26.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.850+0000 7f5102a42640 1 --2- 172.21.15.69:0/1196302403 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50fc1045d0 0x7f50fc1069f0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:26.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.850+0000 7f5102a42640 1 -- 172.21.15.69:0/1196302403 >> 172.21.15.69:0/1196302403 conn(0x7f50fc0fdbd0 msgr2=0x7f50fc100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:26.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.850+0000 7f5102a42640 1 -- 172.21.15.69:0/1196302403 shutdown_connections 2024-04-25T11:38:26.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.850+0000 7f5102a42640 1 -- 172.21.15.69:0/1196302403 wait complete. 2024-04-25T11:38:26.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.850+0000 7f5102a42640 1 Processor -- start 2024-04-25T11:38:26.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.850+0000 7f5102a42640 1 -- start start 2024-04-25T11:38:26.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.851+0000 7f5102a42640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50fc1045d0 0x7f50fc19a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:26.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.851+0000 7f5102a42640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f50fc106f30 0x7f50fc19a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:26.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.851+0000 7f5102a42640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f50fc109890 0x7f50fc1a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:26.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.851+0000 7f5102a42640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc10f490 con 0x7f50fc1045d0 2024-04-25T11:38:26.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.851+0000 7f5102a42640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc10f310 con 0x7f50fc109890 2024-04-25T11:38:26.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.851+0000 7f5102a42640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc10f610 con 0x7f50fc106f30 2024-04-25T11:38:26.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.851+0000 7f50fbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50fc1045d0 0x7f50fc19a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:26.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.851+0000 7f50fbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50fc1045d0 0x7f50fc19a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59462/0 (socket says 172.21.15.69:59462) 2024-04-25T11:38:26.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.851+0000 7f50fbfff640 1 -- 172.21.15.69:0/274610664 learned_addr learned my addr 172.21.15.69:0/274610664 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:38:26.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.851+0000 7f50fb7fe640 1 --2- 172.21.15.69:0/274610664 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f50fc106f30 0x7f50fc19a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:26.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.851+0000 7f5100fdf640 1 --2- 172.21.15.69:0/274610664 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f50fc109890 0x7f50fc1a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:26.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.851+0000 7f50fbfff640 1 -- 172.21.15.69:0/274610664 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f50fc106f30 msgr2=0x7f50fc19a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:26.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.851+0000 7f50fbfff640 1 --2- 172.21.15.69:0/274610664 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f50fc106f30 0x7f50fc19a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:26.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.851+0000 7f50fbfff640 1 -- 172.21.15.69:0/274610664 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f50fc109890 msgr2=0x7f50fc1a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:26.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.851+0000 7f50fbfff640 1 --2- 172.21.15.69:0/274610664 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f50fc109890 0x7f50fc1a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:26.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.851+0000 7f50fbfff640 1 -- 172.21.15.69:0/274610664 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50fc1a1c40 con 0x7f50fc1045d0 2024-04-25T11:38:26.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.851+0000 7f50fbfff640 1 --2- 172.21.15.69:0/274610664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50fc1045d0 0x7f50fc19a1b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f50ec005ab0 tx=0x7f50ec0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:26.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.851+0000 7f50f97fa640 1 -- 172.21.15.69:0/274610664 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec03f030 con 0x7f50fc1045d0 2024-04-25T11:38:26.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.851+0000 7f50f97fa640 1 -- 172.21.15.69:0/274610664 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f50ec003c90 con 0x7f50fc1045d0 2024-04-25T11:38:26.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.851+0000 7f5102a42640 1 -- 172.21.15.69:0/274610664 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50fc1a1ed0 con 0x7f50fc1045d0 2024-04-25T11:38:26.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.851+0000 7f50f97fa640 1 -- 172.21.15.69:0/274610664 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec035070 con 0x7f50fc1045d0 2024-04-25T11:38:26.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.851+0000 7f5102a42640 1 -- 172.21.15.69:0/274610664 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50fc1a22e0 con 0x7f50fc1045d0 2024-04-25T11:38:26.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.853+0000 7f5102a42640 1 -- 172.21.15.69:0/274610664 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50fc0ff6f0 con 0x7f50fc1045d0 2024-04-25T11:38:26.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.854+0000 7f50f97fa640 1 -- 172.21.15.69:0/274610664 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f50ec042030 con 0x7f50fc1045d0 2024-04-25T11:38:26.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.854+0000 7f50f97fa640 1 --2- 172.21.15.69:0/274610664 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f50e40afaf0 0x7f50e40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:26.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.855+0000 7f50fb7fe640 1 --2- 172.21.15.69:0/274610664 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f50e40afaf0 0x7f50e40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:26.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.855+0000 7f50f97fa640 1 -- 172.21.15.69:0/274610664 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f50ec0ff710 con 0x7f50fc1045d0 2024-04-25T11:38:26.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.855+0000 7f50fb7fe640 1 --2- 172.21.15.69:0/274610664 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f50e40afaf0 0x7f50e40b2000 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f50f0005e10 tx=0x7f50f0005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:26.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:26.859+0000 7f50f97fa640 1 -- 172.21.15.69:0/274610664 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f50ec0c7f20 con 0x7f50fc1045d0 2024-04-25T11:38:27.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.074+0000 7f5102a42640 1 -- 172.21.15.69:0/274610664 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f50fc02dd70 con 0x7f50fc1045d0 2024-04-25T11:38:27.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.075+0000 7f50f97fa640 1 -- 172.21.15.69:0/274610664 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11944 (secure 0 0 0) 0x7f50ec0c7820 con 0x7f50fc1045d0 2024-04-25T11:38:27.062 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:38:27.062 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/3399424094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":3399424094},{"type":"v1","addr":"172.21.15.69:6835","nonce":3399424094}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/1463217771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":1463217771},{"type":"v1","addr":"172.21.15.69:6837","nonce":1463217771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T11:23:40.984930+0000","modified":"2024-04-25T11:26:07.156294+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24515,"mds_1":24485,"mds_2":24475,"mds_3":24458,"mds_4":14625},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6839/966316001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":966316001},{"type":"v1","addr":"172.21.15.69:6839","nonce":966316001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14625":{"gid":14625,"name":"a","rank":4,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.69:6841/2060225181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":2060225181},{"type":"v1","addr":"172.21.15.69:6841","nonce":2060225181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6833/4197482473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":4197482473},{"type":"v1","addr":"172.21.15.152:6833","nonce":4197482473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/4267885323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":4267885323},{"type":"v1","addr":"172.21.15.146:6833","nonce":4267885323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6835/3188843362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":3188843362},{"type":"v1","addr":"172.21.15.152:6835","nonce":3188843362}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6837/563274982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":563274982},{"type":"v1","addr":"172.21.15.152:6837","nonce":563274982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.146:6835/1876533699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1876533699},{"type":"v1","addr":"172.21.15.146:6835","nonce":1876533699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.152:6839/3008615657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":3008615657},{"type":"v1","addr":"172.21.15.152:6839","nonce":3008615657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.146:6837/2730317134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":2730317134},{"type":"v1","addr":"172.21.15.146:6837","nonce":2730317134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.146:6839/2013283448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":2013283448},{"type":"v1","addr":"172.21.15.146:6839","nonce":2013283448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24515,"qdb_cluster":[14625,24458,24475,24485,24515]},"id":1}]} 2024-04-25T11:38:27.063 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 29 2024-04-25T11:38:27.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.077+0000 7f5102a42640 1 -- 172.21.15.69:0/274610664 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f50e40afaf0 msgr2=0x7f50e40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:27.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.077+0000 7f5102a42640 1 --2- 172.21.15.69:0/274610664 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f50e40afaf0 0x7f50e40b2000 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f50f0005e10 tx=0x7f50f0005da0 comp rx=0 tx=0).stop 2024-04-25T11:38:27.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.077+0000 7f5102a42640 1 -- 172.21.15.69:0/274610664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50fc1045d0 msgr2=0x7f50fc19a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:27.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.077+0000 7f5102a42640 1 --2- 172.21.15.69:0/274610664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50fc1045d0 0x7f50fc19a1b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f50ec005ab0 tx=0x7f50ec0042f0 comp rx=0 tx=0).stop 2024-04-25T11:38:27.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.077+0000 7f5102a42640 1 -- 172.21.15.69:0/274610664 shutdown_connections 2024-04-25T11:38:27.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.077+0000 7f5102a42640 1 --2- 172.21.15.69:0/274610664 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f50e40afaf0 0x7f50e40b2000 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:27.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.077+0000 7f5102a42640 1 --2- 172.21.15.69:0/274610664 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f50fc109890 0x7f50fc1a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:27.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.077+0000 7f5102a42640 1 --2- 172.21.15.69:0/274610664 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f50fc106f30 0x7f50fc19a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:27.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.077+0000 7f5102a42640 1 --2- 172.21.15.69:0/274610664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50fc1045d0 0x7f50fc19a1b0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:27.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.077+0000 7f5102a42640 1 -- 172.21.15.69:0/274610664 >> 172.21.15.69:0/274610664 conn(0x7f50fc0fdbd0 msgr2=0x7f50fc106690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:27.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.077+0000 7f5102a42640 1 -- 172.21.15.69:0/274610664 shutdown_connections 2024-04-25T11:38:27.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.077+0000 7f5102a42640 1 -- 172.21.15.69:0/274610664 wait complete. 2024-04-25T11:38:27.076 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:38:27.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.197+0000 7fce795f3640 1 -- 172.21.15.69:0/2112259988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce74108940 msgr2=0x7fce7410ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:27.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.197+0000 7fce795f3640 1 --2- 172.21.15.69:0/2112259988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce74108940 0x7fce7410ad60 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fce6c009900 tx=0x7fce6c02a410 comp rx=0 tx=0).stop 2024-04-25T11:38:27.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.197+0000 7fce795f3640 1 -- 172.21.15.69:0/2112259988 shutdown_connections 2024-04-25T11:38:27.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.197+0000 7fce795f3640 1 --2- 172.21.15.69:0/2112259988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce74108940 0x7fce7410ad60 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:27.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.197+0000 7fce795f3640 1 --2- 172.21.15.69:0/2112259988 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce74103c40 0x7fce74104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:27.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.197+0000 7fce795f3640 1 --2- 172.21.15.69:0/2112259988 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce74102a00 0x7fce74103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:27.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.197+0000 7fce795f3640 1 -- 172.21.15.69:0/2112259988 >> 172.21.15.69:0/2112259988 conn(0x7fce740fdbf0 msgr2=0x7fce741000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:27.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.197+0000 7fce795f3640 1 -- 172.21.15.69:0/2112259988 shutdown_connections 2024-04-25T11:38:27.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.197+0000 7fce795f3640 1 -- 172.21.15.69:0/2112259988 wait complete. 2024-04-25T11:38:27.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.197+0000 7fce795f3640 1 Processor -- start 2024-04-25T11:38:27.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.197+0000 7fce795f3640 1 -- start start 2024-04-25T11:38:27.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.198+0000 7fce795f3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce74102a00 0x7fce74075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:27.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.198+0000 7fce795f3640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce74103c40 0x7fce7419ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:27.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.198+0000 7fce795f3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce74108940 0x7fce741a1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:27.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.198+0000 7fce795f3640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fce7410e3c0 con 0x7fce74108940 2024-04-25T11:38:27.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.198+0000 7fce795f3640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fce7410e240 con 0x7fce74102a00 2024-04-25T11:38:27.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.198+0000 7fce795f3640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fce7410e540 con 0x7fce74103c40 2024-04-25T11:38:27.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.198+0000 7fce737fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce74108940 0x7fce741a1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:27.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.198+0000 7fce737fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce74108940 0x7fce741a1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59482/0 (socket says 172.21.15.69:59482) 2024-04-25T11:38:27.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.198+0000 7fce737fe640 1 -- 172.21.15.69:0/603886094 learned_addr learned my addr 172.21.15.69:0/603886094 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:38:27.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.198+0000 7fce72ffd640 1 --2- 172.21.15.69:0/603886094 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce74102a00 0x7fce74075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:27.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.198+0000 7fce727fc640 1 --2- 172.21.15.69:0/603886094 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce74103c40 0x7fce7419ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:27.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.198+0000 7fce737fe640 1 -- 172.21.15.69:0/603886094 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce74103c40 msgr2=0x7fce7419ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:27.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.198+0000 7fce737fe640 1 --2- 172.21.15.69:0/603886094 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce74103c40 0x7fce7419ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:27.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.198+0000 7fce737fe640 1 -- 172.21.15.69:0/603886094 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce74102a00 msgr2=0x7fce74075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:27.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.198+0000 7fce737fe640 1 --2- 172.21.15.69:0/603886094 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce74102a00 0x7fce74075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:27.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.198+0000 7fce737fe640 1 -- 172.21.15.69:0/603886094 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce741a1ba0 con 0x7fce74108940 2024-04-25T11:38:27.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.198+0000 7fce727fc640 1 --2- 172.21.15.69:0/603886094 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce74103c40 0x7fce7419ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:27.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.198+0000 7fce737fe640 1 --2- 172.21.15.69:0/603886094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce74108940 0x7fce741a1380 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fce6c005ab0 tx=0x7fce6c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:27.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.198+0000 7fce5bfff640 1 -- 172.21.15.69:0/603886094 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6c03f030 con 0x7fce74108940 2024-04-25T11:38:27.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.198+0000 7fce5bfff640 1 -- 172.21.15.69:0/603886094 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fce6c003c90 con 0x7fce74108940 2024-04-25T11:38:27.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.198+0000 7fce5bfff640 1 -- 172.21.15.69:0/603886094 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6c035070 con 0x7fce74108940 2024-04-25T11:38:27.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.199+0000 7fce795f3640 1 -- 172.21.15.69:0/603886094 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce741a1e30 con 0x7fce74108940 2024-04-25T11:38:27.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.199+0000 7fce795f3640 1 -- 172.21.15.69:0/603886094 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce741a2290 con 0x7fce74108940 2024-04-25T11:38:27.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.200+0000 7fce795f3640 1 -- 172.21.15.69:0/603886094 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce74107750 con 0x7fce74108940 2024-04-25T11:38:27.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.201+0000 7fce5bfff640 1 -- 172.21.15.69:0/603886094 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7fce6c042030 con 0x7fce74108940 2024-04-25T11:38:27.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.202+0000 7fce5bfff640 1 --2- 172.21.15.69:0/603886094 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fce5c0afb70 0x7fce5c0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:27.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.202+0000 7fce72ffd640 1 --2- 172.21.15.69:0/603886094 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fce5c0afb70 0x7fce5c0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:27.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.202+0000 7fce5bfff640 1 -- 172.21.15.69:0/603886094 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7fce6c0ffb60 con 0x7fce74108940 2024-04-25T11:38:27.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.202+0000 7fce72ffd640 1 --2- 172.21.15.69:0/603886094 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fce5c0afb70 0x7fce5c0b2080 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fce74103600 tx=0x7fce6400a590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:27.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.205+0000 7fce5bfff640 1 -- 172.21.15.69:0/603886094 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fce6c0c8370 con 0x7fce74108940 2024-04-25T11:38:27.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.383+0000 7fce795f3640 1 -- 172.21.15.69:0/603886094 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fce7402dde0 con 0x7fce74108940 2024-04-25T11:38:27.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.385+0000 7fce5bfff640 1 -- 172.21.15.69:0/603886094 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+24862 (secure 0 0 0) 0x7fce6c0c7c70 con 0x7fce74108940 2024-04-25T11:38:27.373 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T11:38:27.373 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":84,"fsid":"f7ed73dc-02f4-11ef-bc93-c7b262605968","created":"2024-04-25T11:15:37.965281+0000","modified":"2024-04-25T11:25:33.888184+0000","last_up_change":"2024-04-25T11:22:50.691869+0000","last_in_change":"2024-04-25T11:22:30.271959+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:18:57.781263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:23:38.685110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:23:39.943803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T11:23:43.650521+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a6a0d179-4998-4754-bf82-2ea3dac54e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6803","nonce":3756025235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6805","nonce":3756025235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6809","nonce":3756025235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3756025235},{"type":"v1","addr":"172.21.15.69:6807","nonce":3756025235}]},"public_addr":"172.21.15.69:6803/3756025235","cluster_addr":"172.21.15.69:6805/3756025235","heartbeat_back_addr":"172.21.15.69:6809/3756025235","heartbeat_front_addr":"172.21.15.69:6807/3756025235","state":["exists","up"]},{"osd":1,"uuid":"c0c66930-4233-493f-b2bb-99c473994b59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6801","nonce":3166305213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6803","nonce":3166305213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6807","nonce":3166305213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":3166305213},{"type":"v1","addr":"172.21.15.146:6805","nonce":3166305213}]},"public_addr":"172.21.15.146:6801/3166305213","cluster_addr":"172.21.15.146:6803/3166305213","heartbeat_back_addr":"172.21.15.146:6807/3166305213","heartbeat_front_addr":"172.21.15.146:6805/3166305213","state":["exists","up"]},{"osd":2,"uuid":"d4ef735b-9786-4cb9-83ef-7c3f0691a729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6801","nonce":1474319175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6803","nonce":1474319175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6807","nonce":1474319175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1474319175},{"type":"v1","addr":"172.21.15.152:6805","nonce":1474319175}]},"public_addr":"172.21.15.152:6801/1474319175","cluster_addr":"172.21.15.152:6803/1474319175","heartbeat_back_addr":"172.21.15.152:6807/1474319175","heartbeat_front_addr":"172.21.15.152:6805/1474319175","state":["exists","up"]},{"osd":3,"uuid":"0ab85fb8-5bbb-4ffc-bddf-cae7644a64ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6811","nonce":1809071296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6813","nonce":1809071296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6817","nonce":1809071296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":1809071296},{"type":"v1","addr":"172.21.15.69:6815","nonce":1809071296}]},"public_addr":"172.21.15.69:6811/1809071296","cluster_addr":"172.21.15.69:6813/1809071296","heartbeat_back_addr":"172.21.15.69:6817/1809071296","heartbeat_front_addr":"172.21.15.69:6815/1809071296","state":["exists","up"]},{"osd":4,"uuid":"273e0592-bbbc-4188-8eeb-46a3879cf632","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6809","nonce":1898520972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6811","nonce":1898520972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6815","nonce":1898520972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":1898520972},{"type":"v1","addr":"172.21.15.146:6813","nonce":1898520972}]},"public_addr":"172.21.15.146:6809/1898520972","cluster_addr":"172.21.15.146:6811/1898520972","heartbeat_back_addr":"172.21.15.146:6815/1898520972","heartbeat_front_addr":"172.21.15.146:6813/1898520972","state":["exists","up"]},{"osd":5,"uuid":"bbeae945-9fa2-48f9-a1d6-8bf18dcaef19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6809","nonce":1996555108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6811","nonce":1996555108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6815","nonce":1996555108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":1996555108},{"type":"v1","addr":"172.21.15.152:6813","nonce":1996555108}]},"public_addr":"172.21.15.152:6809/1996555108","cluster_addr":"172.21.15.152:6811/1996555108","heartbeat_back_addr":"172.21.15.152:6815/1996555108","heartbeat_front_addr":"172.21.15.152:6813/1996555108","state":["exists","up"]},{"osd":6,"uuid":"dbb61253-30bc-418e-bd75-118c80bbf931","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6819","nonce":491215762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6821","nonce":491215762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6825","nonce":491215762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":491215762},{"type":"v1","addr":"172.21.15.69:6823","nonce":491215762}]},"public_addr":"172.21.15.69:6819/491215762","cluster_addr":"172.21.15.69:6821/491215762","heartbeat_back_addr":"172.21.15.69:6825/491215762","heartbeat_front_addr":"172.21.15.69:6823/491215762","state":["exists","up"]},{"osd":7,"uuid":"75f23854-e283-4e3e-94cf-7b4079d83876","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6817","nonce":2330294690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6819","nonce":2330294690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6823","nonce":2330294690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2330294690},{"type":"v1","addr":"172.21.15.146:6821","nonce":2330294690}]},"public_addr":"172.21.15.146:6817/2330294690","cluster_addr":"172.21.15.146:6819/2330294690","heartbeat_back_addr":"172.21.15.146:6823/2330294690","heartbeat_front_addr":"172.21.15.146:6821/2330294690","state":["exists","up"]},{"osd":8,"uuid":"1aa1060c-0c14-4abd-be56-5ebfeb78759c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6817","nonce":950669381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6819","nonce":950669381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6823","nonce":950669381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":950669381},{"type":"v1","addr":"172.21.15.152:6821","nonce":950669381}]},"public_addr":"172.21.15.152:6817/950669381","cluster_addr":"172.21.15.152:6819/950669381","heartbeat_back_addr":"172.21.15.152:6823/950669381","heartbeat_front_addr":"172.21.15.152:6821/950669381","state":["exists","up"]},{"osd":9,"uuid":"3f4bf800-4090-4138-9bb3-d001f13f450c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6827","nonce":2414186758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6829","nonce":2414186758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6833","nonce":2414186758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2414186758},{"type":"v1","addr":"172.21.15.69:6831","nonce":2414186758}]},"public_addr":"172.21.15.69:6827/2414186758","cluster_addr":"172.21.15.69:6829/2414186758","heartbeat_back_addr":"172.21.15.69:6833/2414186758","heartbeat_front_addr":"172.21.15.69:6831/2414186758","state":["exists","up"]},{"osd":10,"uuid":"19e967e5-51ce-4506-80c9-0a95441baf99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6825","nonce":707256400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6827","nonce":707256400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6831","nonce":707256400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":707256400},{"type":"v1","addr":"172.21.15.146:6829","nonce":707256400}]},"public_addr":"172.21.15.146:6825/707256400","cluster_addr":"172.21.15.146:6827/707256400","heartbeat_back_addr":"172.21.15.146:6831/707256400","heartbeat_front_addr":"172.21.15.146:6829/707256400","state":["exists","up"]},{"osd":11,"uuid":"7788c2de-6930-4808-be0f-f1fd1ae0cb4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6825","nonce":2934336833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6827","nonce":2934336833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6831","nonce":2934336833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":2934336833},{"type":"v1","addr":"172.21.15.152:6829","nonce":2934336833}]},"public_addr":"172.21.15.152:6825/2934336833","cluster_addr":"172.21.15.152:6827/2934336833","heartbeat_back_addr":"172.21.15.152:6831/2934336833","heartbeat_front_addr":"172.21.15.152:6829/2934336833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:07.497448+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:31.227494+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:18:55.027978+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:20.713437+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:19:46.214704+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:20:11.199983+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:20:37.177070+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:02.951240+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:28.963680+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:21:56.095807+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:22:22.221744+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:22:48.929334+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1b","mappings":[{"from":5,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/1372727546":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6801/1097124842":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6800/1508198215":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3103465990":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3540132960":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/3899912190":"2024-04-26T11:16:31.688159+0000","172.21.15.69:6800/1097124842":"2024-04-26T11:16:31.688159+0000","172.21.15.69:0/1199907202":"2024-04-26T11:16:04.860792+0000","172.21.15.69:6801/1508198215":"2024-04-26T11:16:04.860792+0000","172.21.15.69:0/2879102529":"2024-04-26T11:16:31.688159+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:38:27.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.387+0000 7fce795f3640 1 -- 172.21.15.69:0/603886094 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fce5c0afb70 msgr2=0x7fce5c0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:27.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.387+0000 7fce795f3640 1 --2- 172.21.15.69:0/603886094 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fce5c0afb70 0x7fce5c0b2080 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fce74103600 tx=0x7fce6400a590 comp rx=0 tx=0).stop 2024-04-25T11:38:27.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.387+0000 7fce795f3640 1 -- 172.21.15.69:0/603886094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce74108940 msgr2=0x7fce741a1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:27.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.387+0000 7fce795f3640 1 --2- 172.21.15.69:0/603886094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce74108940 0x7fce741a1380 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fce6c005ab0 tx=0x7fce6c0042f0 comp rx=0 tx=0).stop 2024-04-25T11:38:27.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.387+0000 7fce795f3640 1 -- 172.21.15.69:0/603886094 shutdown_connections 2024-04-25T11:38:27.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.387+0000 7fce795f3640 1 --2- 172.21.15.69:0/603886094 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7fce5c0afb70 0x7fce5c0b2080 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:27.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.387+0000 7fce795f3640 1 --2- 172.21.15.69:0/603886094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce74108940 0x7fce741a1380 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:27.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.387+0000 7fce795f3640 1 --2- 172.21.15.69:0/603886094 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce74103c40 0x7fce7419ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:27.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.387+0000 7fce795f3640 1 --2- 172.21.15.69:0/603886094 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fce74102a00 0x7fce74075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:27.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.387+0000 7fce795f3640 1 -- 172.21.15.69:0/603886094 >> 172.21.15.69:0/603886094 conn(0x7fce740fdbf0 msgr2=0x7fce74104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:27.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.387+0000 7fce795f3640 1 -- 172.21.15.69:0/603886094 shutdown_connections 2024-04-25T11:38:27.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.387+0000 7fce795f3640 1 -- 172.21.15.69:0/603886094 wait complete. 2024-04-25T11:38:27.385 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:27 smithi069 ceph-mon[29188]: pgmap v731: 129 pgs: 129 active+clean; 7.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-04-25T11:38:27.386 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:27 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/326333146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:38:27.386 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:27 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/274610664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:38:27.388 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:38:27.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.510+0000 7f1259c25640 1 -- 172.21.15.69:0/720026958 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1254104d40 msgr2=0x7f1254105150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:27.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.510+0000 7f1259c25640 1 --2- 172.21.15.69:0/720026958 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1254104d40 0x7f1254105150 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1248009900 tx=0x7f124802a410 comp rx=0 tx=0).stop 2024-04-25T11:38:27.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.510+0000 7f1259c25640 1 -- 172.21.15.69:0/720026958 shutdown_connections 2024-04-25T11:38:27.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.510+0000 7f1259c25640 1 --2- 172.21.15.69:0/720026958 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1254109a40 0x7f125410be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:27.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.510+0000 7f1259c25640 1 --2- 172.21.15.69:0/720026958 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1254104d40 0x7f1254105150 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:27.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.510+0000 7f1259c25640 1 --2- 172.21.15.69:0/720026958 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1254103b00 0x7f1254104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:27.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.510+0000 7f1259c25640 1 -- 172.21.15.69:0/720026958 >> 172.21.15.69:0/720026958 conn(0x7f12540fecf0 msgr2=0x7f12541011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:27.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.510+0000 7f1259c25640 1 -- 172.21.15.69:0/720026958 shutdown_connections 2024-04-25T11:38:27.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.510+0000 7f1259c25640 1 -- 172.21.15.69:0/720026958 wait complete. 2024-04-25T11:38:27.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.510+0000 7f1259c25640 1 Processor -- start 2024-04-25T11:38:27.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.510+0000 7f1259c25640 1 -- start start 2024-04-25T11:38:27.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.511+0000 7f1259c25640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1254103b00 0x7f125419a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:27.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.511+0000 7f1259c25640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1254104d40 0x7f125419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:27.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.511+0000 7f1259c25640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1254109a40 0x7f12541a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:27.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.511+0000 7f1259c25640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f125410f4e0 con 0x7f1254103b00 2024-04-25T11:38:27.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.511+0000 7f1259c25640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f125410f360 con 0x7f1254109a40 2024-04-25T11:38:27.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.511+0000 7f1259c25640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f125410f660 con 0x7f1254104d40 2024-04-25T11:38:27.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.511+0000 7f12537fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1254103b00 0x7f125419a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:27.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.511+0000 7f12537fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1254103b00 0x7f125419a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59504/0 (socket says 172.21.15.69:59504) 2024-04-25T11:38:27.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.511+0000 7f12537fe640 1 -- 172.21.15.69:0/2599343258 learned_addr learned my addr 172.21.15.69:0/2599343258 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:38:27.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.511+0000 7f1253fff640 1 --2- 172.21.15.69:0/2599343258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1254109a40 0x7f12541a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:27.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.511+0000 7f1252ffd640 1 --2- 172.21.15.69:0/2599343258 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1254104d40 0x7f125419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:27.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.511+0000 7f12537fe640 1 -- 172.21.15.69:0/2599343258 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1254104d40 msgr2=0x7f125419a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:27.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.511+0000 7f12537fe640 1 --2- 172.21.15.69:0/2599343258 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1254104d40 0x7f125419a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:27.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.511+0000 7f12537fe640 1 -- 172.21.15.69:0/2599343258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1254109a40 msgr2=0x7f12541a1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:27.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.511+0000 7f12537fe640 1 --2- 172.21.15.69:0/2599343258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1254109a40 0x7f12541a1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:27.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.511+0000 7f12537fe640 1 -- 172.21.15.69:0/2599343258 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12541a1bd0 con 0x7f1254103b00 2024-04-25T11:38:27.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.511+0000 7f1252ffd640 1 --2- 172.21.15.69:0/2599343258 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1254104d40 0x7f125419a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:27.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.511+0000 7f12537fe640 1 --2- 172.21.15.69:0/2599343258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1254103b00 0x7f125419a140 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f124400cce0 tx=0x7f124400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:27.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.511+0000 7f1250ff9640 1 -- 172.21.15.69:0/2599343258 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124400f450 con 0x7f1254103b00 2024-04-25T11:38:27.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.511+0000 7f1250ff9640 1 -- 172.21.15.69:0/2599343258 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f124400fa80 con 0x7f1254103b00 2024-04-25T11:38:27.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.511+0000 7f1259c25640 1 -- 172.21.15.69:0/2599343258 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12541a1ec0 con 0x7f1254103b00 2024-04-25T11:38:27.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.511+0000 7f1250ff9640 1 -- 172.21.15.69:0/2599343258 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1244018ad0 con 0x7f1254103b00 2024-04-25T11:38:27.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.512+0000 7f1259c25640 1 -- 172.21.15.69:0/2599343258 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12541a2320 con 0x7f1254103b00 2024-04-25T11:38:27.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.512+0000 7f1259c25640 1 -- 172.21.15.69:0/2599343258 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f125406afa0 con 0x7f1254103b00 2024-04-25T11:38:27.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.514+0000 7f1250ff9640 1 -- 172.21.15.69:0/2599343258 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f1244018c70 con 0x7f1254103b00 2024-04-25T11:38:27.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.514+0000 7f1250ff9640 1 --2- 172.21.15.69:0/2599343258 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f123c0afaf0 0x7f123c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:27.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.514+0000 7f1252ffd640 1 --2- 172.21.15.69:0/2599343258 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f123c0afaf0 0x7f123c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:27.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.515+0000 7f1250ff9640 1 -- 172.21.15.69:0/2599343258 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9216+0+0 (secure 0 0 0) 0x7f12440df5d0 con 0x7f1254103b00 2024-04-25T11:38:27.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.515+0000 7f1252ffd640 1 --2- 172.21.15.69:0/2599343258 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f123c0afaf0 0x7f123c0b2000 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1248005f70 tx=0x7f1248005720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:27.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.518+0000 7f1250ff9640 1 -- 172.21.15.69:0/2599343258 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1244052ff0 con 0x7f1254103b00 2024-04-25T11:38:27.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:27 smithi146 ceph-mon[32821]: pgmap v731: 129 pgs: 129 active+clean; 7.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-04-25T11:38:27.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:27 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/326333146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:38:27.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:27 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/274610664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:38:27.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.733+0000 7f1259c25640 1 -- 172.21.15.69:0/2599343258 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f12541088a0 con 0x7f1254103b00 2024-04-25T11:38:27.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.736+0000 7f1250ff9640 1 -- 172.21.15.69:0/2599343258 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v30) ==== 161+0+0 (secure 0 0 0) 0x7f12541088a0 con 0x7f1254103b00 2024-04-25T11:38:27.723 INFO:teuthology.orchestra.run.smithi069.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:38:27.724 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:27 smithi152 ceph-mon[32701]: pgmap v731: 129 pgs: 129 active+clean; 7.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-04-25T11:38:27.724 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:27 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/326333146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:38:27.724 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:27 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/274610664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:38:27.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.738+0000 7f1259c25640 1 -- 172.21.15.69:0/2599343258 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f123c0afaf0 msgr2=0x7f123c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:27.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.738+0000 7f1259c25640 1 --2- 172.21.15.69:0/2599343258 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f123c0afaf0 0x7f123c0b2000 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1248005f70 tx=0x7f1248005720 comp rx=0 tx=0).stop 2024-04-25T11:38:27.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.738+0000 7f1259c25640 1 -- 172.21.15.69:0/2599343258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1254103b00 msgr2=0x7f125419a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:27.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.738+0000 7f1259c25640 1 --2- 172.21.15.69:0/2599343258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1254103b00 0x7f125419a140 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f124400cce0 tx=0x7f124400a620 comp rx=0 tx=0).stop 2024-04-25T11:38:27.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.739+0000 7f1259c25640 1 -- 172.21.15.69:0/2599343258 shutdown_connections 2024-04-25T11:38:27.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.739+0000 7f1259c25640 1 --2- 172.21.15.69:0/2599343258 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f123c0afaf0 0x7f123c0b2000 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:27.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.739+0000 7f1259c25640 1 --2- 172.21.15.69:0/2599343258 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1254109a40 0x7f12541a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:27.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.739+0000 7f1259c25640 1 --2- 172.21.15.69:0/2599343258 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1254104d40 0x7f125419a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:27.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.739+0000 7f1259c25640 1 --2- 172.21.15.69:0/2599343258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1254103b00 0x7f125419a140 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:27.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.739+0000 7f1259c25640 1 -- 172.21.15.69:0/2599343258 >> 172.21.15.69:0/2599343258 conn(0x7f12540fecf0 msgr2=0x7f1254105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:27.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.739+0000 7f1259c25640 1 -- 172.21.15.69:0/2599343258 shutdown_connections 2024-04-25T11:38:27.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.739+0000 7f1259c25640 1 -- 172.21.15.69:0/2599343258 wait complete. 2024-04-25T11:38:27.739 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T11:38:27.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.870+0000 7ff151d84640 1 -- 172.21.15.69:0/722778544 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff14c104460 msgr2=0x7ff14c10abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:27.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.870+0000 7ff151d84640 1 --2- 172.21.15.69:0/722778544 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff14c104460 0x7ff14c10abb0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff144009ca0 tx=0x7ff14402a8c0 comp rx=0 tx=0).stop 2024-04-25T11:38:27.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.870+0000 7ff151d84640 1 -- 172.21.15.69:0/722778544 shutdown_connections 2024-04-25T11:38:27.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.870+0000 7ff151d84640 1 --2- 172.21.15.69:0/722778544 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff14c104460 0x7ff14c10abb0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:27.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.870+0000 7ff151d84640 1 --2- 172.21.15.69:0/722778544 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff14c103940 0x7ff14c103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:27.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.870+0000 7ff151d84640 1 --2- 172.21.15.69:0/722778544 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff14c102f60 0x7ff14c103370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:27.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.870+0000 7ff151d84640 1 -- 172.21.15.69:0/722778544 >> 172.21.15.69:0/722778544 conn(0x7ff14c0fd340 msgr2=0x7ff14c0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:27.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.870+0000 7ff151d84640 1 -- 172.21.15.69:0/722778544 shutdown_connections 2024-04-25T11:38:27.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.870+0000 7ff151d84640 1 -- 172.21.15.69:0/722778544 wait complete. 2024-04-25T11:38:27.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.870+0000 7ff151d84640 1 Processor -- start 2024-04-25T11:38:27.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.871+0000 7ff151d84640 1 -- start start 2024-04-25T11:38:27.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.871+0000 7ff151d84640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff14c102f60 0x7ff14c102220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:27.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.871+0000 7ff151d84640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff14c103940 0x7ff14c19ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:27.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.871+0000 7ff151d84640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff14c104460 0x7ff14c1a13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:27.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.871+0000 7ff151d84640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff14c10e430 con 0x7ff14c103940 2024-04-25T11:38:27.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.871+0000 7ff151d84640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff14c10e2b0 con 0x7ff14c102f60 2024-04-25T11:38:27.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.871+0000 7ff151d84640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff14c10e5b0 con 0x7ff14c104460 2024-04-25T11:38:27.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.871+0000 7ff14affd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff14c103940 0x7ff14c19ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:27.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.871+0000 7ff14affd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff14c103940 0x7ff14c19ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59560/0 (socket says 172.21.15.69:59560) 2024-04-25T11:38:27.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.871+0000 7ff14affd640 1 -- 172.21.15.69:0/2225442652 learned_addr learned my addr 172.21.15.69:0/2225442652 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:38:27.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.871+0000 7ff14b7fe640 1 --2- 172.21.15.69:0/2225442652 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff14c102f60 0x7ff14c102220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:27.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.871+0000 7ff14bfff640 1 --2- 172.21.15.69:0/2225442652 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff14c104460 0x7ff14c1a13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:27.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.871+0000 7ff14affd640 1 -- 172.21.15.69:0/2225442652 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff14c104460 msgr2=0x7ff14c1a13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:27.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.871+0000 7ff14affd640 1 --2- 172.21.15.69:0/2225442652 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff14c104460 0x7ff14c1a13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:27.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.871+0000 7ff14affd640 1 -- 172.21.15.69:0/2225442652 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff14c102f60 msgr2=0x7ff14c102220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:27.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.871+0000 7ff14affd640 1 --2- 172.21.15.69:0/2225442652 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff14c102f60 0x7ff14c102220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:27.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.871+0000 7ff14affd640 1 -- 172.21.15.69:0/2225442652 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff14c1a1bc0 con 0x7ff14c103940 2024-04-25T11:38:27.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.871+0000 7ff14affd640 1 --2- 172.21.15.69:0/2225442652 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff14c103940 0x7ff14c19ee30 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff13c00e9f0 tx=0x7ff13c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:27.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.872+0000 7ff148ff9640 1 -- 172.21.15.69:0/2225442652 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13c00cd80 con 0x7ff14c103940 2024-04-25T11:38:27.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.872+0000 7ff148ff9640 1 -- 172.21.15.69:0/2225442652 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff13c0054b0 con 0x7ff14c103940 2024-04-25T11:38:27.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.872+0000 7ff151d84640 1 -- 172.21.15.69:0/2225442652 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff14c1a1eb0 con 0x7ff14c103940 2024-04-25T11:38:27.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.872+0000 7ff148ff9640 1 -- 172.21.15.69:0/2225442652 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13c010430 con 0x7ff14c103940 2024-04-25T11:38:27.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.872+0000 7ff151d84640 1 -- 172.21.15.69:0/2225442652 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff14c1a2310 con 0x7ff14c103940 2024-04-25T11:38:27.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.873+0000 7ff151d84640 1 -- 172.21.15.69:0/2225442652 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff14c109910 con 0x7ff14c103940 2024-04-25T11:38:27.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.874+0000 7ff148ff9640 1 -- 172.21.15.69:0/2225442652 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7ff13c0046c0 con 0x7ff14c103940 2024-04-25T11:38:27.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.875+0000 7ff148ff9640 1 --2- 172.21.15.69:0/2225442652 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ff1340afaf0 0x7ff1340b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:27.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.875+0000 7ff14b7fe640 1 --2- 172.21.15.69:0/2225442652 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ff1340afaf0 0x7ff1340b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:27.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.875+0000 7ff148ff9640 1 -- 172.21.15.69:0/2225442652 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9646+0+0 (secure 0 0 0) 0x7ff13c0df7e0 con 0x7ff14c103940 2024-04-25T11:38:27.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.875+0000 7ff14b7fe640 1 --2- 172.21.15.69:0/2225442652 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ff1340afaf0 0x7ff1340b2000 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff140005d00 tx=0x7ff140005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:27.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:27.878+0000 7ff148ff9640 1 -- 172.21.15.69:0/2225442652 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff13c053ff0 con 0x7ff14c103940 2024-04-25T11:38:28.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.095+0000 7ff151d84640 1 -- 172.21.15.69:0/2225442652 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ff14c0637a0 con 0x7ff14c103940 2024-04-25T11:38:28.093 INFO:journalctl@ceph.mds.e.smithi146.stdout:Apr 25 11:38:27 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-e[62273]: ignoring --setuser ceph since I am not root 2024-04-25T11:38:28.093 INFO:journalctl@ceph.mds.e.smithi146.stdout:Apr 25 11:38:27 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-e[62273]: ignoring --setgroup ceph since I am not root 2024-04-25T11:38:28.093 INFO:journalctl@ceph.mds.e.smithi146.stdout:Apr 25 11:38:27 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-e[62273]: starting mds.e at 2024-04-25T11:38:28.093 INFO:journalctl@ceph.mds.h.smithi146.stdout:Apr 25 11:38:27 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-h[62580]: ignoring --setuser ceph since I am not root 2024-04-25T11:38:28.093 INFO:journalctl@ceph.mds.h.smithi146.stdout:Apr 25 11:38:27 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-h[62580]: ignoring --setgroup ceph since I am not root 2024-04-25T11:38:28.093 INFO:journalctl@ceph.mds.h.smithi146.stdout:Apr 25 11:38:27 smithi146 ceph-mds[62584]: -- 172.21.15.146:0/3357573990 <== mon.1 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56104b3ffa40 con 0x56104b447600 2024-04-25T11:38:28.093 INFO:journalctl@ceph.mds.h.smithi146.stdout:Apr 25 11:38:27 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-h[62580]: starting mds.h at 2024-04-25T11:38:28.094 INFO:journalctl@ceph.mds.k.smithi146.stdout:Apr 25 11:38:27 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-k[61952]: ignoring --setuser ceph since I am not root 2024-04-25T11:38:28.094 INFO:journalctl@ceph.mds.k.smithi146.stdout:Apr 25 11:38:27 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-k[61952]: ignoring --setgroup ceph since I am not root 2024-04-25T11:38:28.094 INFO:journalctl@ceph.mds.k.smithi146.stdout:Apr 25 11:38:27 smithi146 ceph-mds[61956]: -- 172.21.15.146:0/565936090 <== mon.1 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555d371c5a40 con 0x555d3720da80 2024-04-25T11:38:28.094 INFO:journalctl@ceph.mds.k.smithi146.stdout:Apr 25 11:38:27 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-k[61952]: starting mds.k at 2024-04-25T11:38:28.094 INFO:journalctl@ceph.mds.b.smithi146.stdout:Apr 25 11:38:27 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-b[62892]: ignoring --setuser ceph since I am not root 2024-04-25T11:38:28.094 INFO:journalctl@ceph.mds.b.smithi146.stdout:Apr 25 11:38:27 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-b[62892]: ignoring --setgroup ceph since I am not root 2024-04-25T11:38:28.095 INFO:journalctl@ceph.mds.b.smithi146.stdout:Apr 25 11:38:27 smithi146 ceph-mds[62896]: -- 172.21.15.146:0/1145974215 <== mon.0 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b4bef89a40 con 0x55b4befd1180 2024-04-25T11:38:28.095 INFO:journalctl@ceph.mds.b.smithi146.stdout:Apr 25 11:38:27 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-b[62892]: starting mds.b at 2024-04-25T11:38:28.186 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:27 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mon-a[29184]: 2024-04-25T11:38:27.734+0000 7f6d76e93700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:38:28.187 INFO:journalctl@ceph.mds.g.smithi069.stdout:Apr 25 11:38:27 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-g[70517]: ignoring --setuser ceph since I am not root 2024-04-25T11:38:28.187 INFO:journalctl@ceph.mds.g.smithi069.stdout:Apr 25 11:38:27 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-g[70517]: ignoring --setgroup ceph since I am not root 2024-04-25T11:38:28.187 INFO:journalctl@ceph.mds.g.smithi069.stdout:Apr 25 11:38:27 smithi069 ceph-mds[70521]: -- 172.21.15.69:0/3763978654 <== mon.2 v2:172.21.15.146:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c20c9cba40 con 0x55c20ca13a80 2024-04-25T11:38:28.187 INFO:journalctl@ceph.mds.g.smithi069.stdout:Apr 25 11:38:27 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-g[70517]: starting mds.g at 2024-04-25T11:38:28.187 INFO:journalctl@ceph.mds.a.smithi069.stdout:Apr 25 11:38:27 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-a[70877]: ignoring --setuser ceph since I am not root 2024-04-25T11:38:28.188 INFO:journalctl@ceph.mds.a.smithi069.stdout:Apr 25 11:38:27 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-a[70877]: ignoring --setgroup ceph since I am not root 2024-04-25T11:38:28.188 INFO:journalctl@ceph.mds.a.smithi069.stdout:Apr 25 11:38:27 smithi069 ceph-mds[70881]: -- 172.21.15.69:0/2583406484 <== mon.0 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a2c98c5a40 con 0x55a2c990d180 2024-04-25T11:38:28.188 INFO:journalctl@ceph.mds.a.smithi069.stdout:Apr 25 11:38:27 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-a[70877]: starting mds.a at 2024-04-25T11:38:28.224 INFO:journalctl@ceph.mds.l.smithi152.stdout:Apr 25 11:38:27 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-l[62974]: ignoring --setuser ceph since I am not root 2024-04-25T11:38:28.224 INFO:journalctl@ceph.mds.l.smithi152.stdout:Apr 25 11:38:27 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-l[62974]: ignoring --setgroup ceph since I am not root 2024-04-25T11:38:28.225 INFO:journalctl@ceph.mds.l.smithi152.stdout:Apr 25 11:38:27 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-l[62974]: starting mds.l at 2024-04-25T11:38:28.225 INFO:journalctl@ceph.mds.c.smithi152.stdout:Apr 25 11:38:27 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-c[63595]: ignoring --setuser ceph since I am not root 2024-04-25T11:38:28.225 INFO:journalctl@ceph.mds.c.smithi152.stdout:Apr 25 11:38:27 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-c[63595]: ignoring --setgroup ceph since I am not root 2024-04-25T11:38:28.225 INFO:journalctl@ceph.mds.c.smithi152.stdout:Apr 25 11:38:27 smithi152 ceph-mds[63599]: -- 172.21.15.152:0/3633144515 <== mon.1 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e765105a40 con 0x55e76514d600 2024-04-25T11:38:28.225 INFO:journalctl@ceph.mds.c.smithi152.stdout:Apr 25 11:38:27 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-c[63595]: starting mds.c at 2024-04-25T11:38:28.225 INFO:journalctl@ceph.mds.f.smithi152.stdout:Apr 25 11:38:27 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-f[63285]: ignoring --setuser ceph since I am not root 2024-04-25T11:38:28.225 INFO:journalctl@ceph.mds.f.smithi152.stdout:Apr 25 11:38:27 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-f[63285]: ignoring --setgroup ceph since I am not root 2024-04-25T11:38:28.226 INFO:journalctl@ceph.mds.f.smithi152.stdout:Apr 25 11:38:27 smithi152 ceph-mds[63289]: -- 172.21.15.152:0/3561887454 <== mon.1 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5624b77eda40 con 0x5624b7835a80 2024-04-25T11:38:28.226 INFO:journalctl@ceph.mds.f.smithi152.stdout:Apr 25 11:38:27 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-f[63285]: starting mds.f at 2024-04-25T11:38:28.226 INFO:journalctl@ceph.mds.i.smithi152.stdout:Apr 25 11:38:27 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-i[63906]: ignoring --setuser ceph since I am not root 2024-04-25T11:38:28.226 INFO:journalctl@ceph.mds.i.smithi152.stdout:Apr 25 11:38:27 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-i[63906]: ignoring --setgroup ceph since I am not root 2024-04-25T11:38:28.226 INFO:journalctl@ceph.mds.i.smithi152.stdout:Apr 25 11:38:27 smithi152 ceph-mds[63910]: -- 172.21.15.152:0/3357820772 <== mon.2 v2:172.21.15.146:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555a0c8eba40 con 0x555a0c933600 2024-04-25T11:38:28.226 INFO:journalctl@ceph.mds.i.smithi152.stdout:Apr 25 11:38:27 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-i[63906]: starting mds.i at 2024-04-25T11:38:28.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:28 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/603886094' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:38:28.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:28 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/2599343258' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T11:38:28.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:28 smithi146 ceph-mon[32821]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T11:38:28.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:28 smithi146 ceph-mon[32821]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:38:28.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:28 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/2599343258' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T11:38:28.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:28 smithi146 ceph-mon[32821]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T11:38:28.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:28 smithi146 ceph-mon[32821]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-25T11:38:28.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:28 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/2225442652' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T11:38:28.686 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:28 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/603886094' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:38:28.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:28 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2599343258' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T11:38:28.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:28 smithi069 ceph-mon[29188]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T11:38:28.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:28 smithi069 ceph-mon[29188]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:38:28.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:28 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2599343258' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T11:38:28.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:28 smithi069 ceph-mon[29188]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T11:38:28.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:28 smithi069 ceph-mon[29188]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-25T11:38:28.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:28 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2225442652' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T11:38:28.724 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:28 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/603886094' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:38:28.724 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:28 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/2599343258' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T11:38:28.724 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:28 smithi152 ceph-mon[32701]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T11:38:28.724 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:28 smithi152 ceph-mon[32701]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:38:28.724 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:28 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/2599343258' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T11:38:28.724 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:28 smithi152 ceph-mon[32701]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T11:38:28.725 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:28 smithi152 ceph-mon[32701]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-25T11:38:28.725 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:28 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/2225442652' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T11:38:28.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.743+0000 7ff148ff9640 1 -- 172.21.15.69:0/2225442652 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v31) ==== 104+0+0 (secure 0 0 0) 0x7ff13c0a7e40 con 0x7ff14c103940 2024-04-25T11:38:28.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.745+0000 7ff151d84640 1 -- 172.21.15.69:0/2225442652 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ff1340afaf0 msgr2=0x7ff1340b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:28.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.745+0000 7ff151d84640 1 --2- 172.21.15.69:0/2225442652 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ff1340afaf0 0x7ff1340b2000 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff140005d00 tx=0x7ff140005c50 comp rx=0 tx=0).stop 2024-04-25T11:38:28.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.745+0000 7ff151d84640 1 -- 172.21.15.69:0/2225442652 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff14c103940 msgr2=0x7ff14c19ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:28.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.745+0000 7ff151d84640 1 --2- 172.21.15.69:0/2225442652 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff14c103940 0x7ff14c19ee30 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff13c00e9f0 tx=0x7ff13c00edf0 comp rx=0 tx=0).stop 2024-04-25T11:38:28.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.745+0000 7ff151d84640 1 -- 172.21.15.69:0/2225442652 shutdown_connections 2024-04-25T11:38:28.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.746+0000 7ff151d84640 1 --2- 172.21.15.69:0/2225442652 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7ff1340afaf0 0x7ff1340b2000 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:28.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.746+0000 7ff151d84640 1 --2- 172.21.15.69:0/2225442652 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff14c104460 0x7ff14c1a13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:28.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.746+0000 7ff151d84640 1 --2- 172.21.15.69:0/2225442652 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff14c103940 0x7ff14c19ee30 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:28.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.746+0000 7ff151d84640 1 --2- 172.21.15.69:0/2225442652 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff14c102f60 0x7ff14c102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:28.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.746+0000 7ff151d84640 1 -- 172.21.15.69:0/2225442652 >> 172.21.15.69:0/2225442652 conn(0x7ff14c0fd340 msgr2=0x7ff14c106ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:28.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.746+0000 7ff151d84640 1 -- 172.21.15.69:0/2225442652 shutdown_connections 2024-04-25T11:38:28.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.746+0000 7ff151d84640 1 -- 172.21.15.69:0/2225442652 wait complete. 2024-04-25T11:38:28.749 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T11:38:28.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.877+0000 7efe22fee640 1 -- 172.21.15.69:0/2591659744 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe1c105090 msgr2=0x7efe1c1054a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:28.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.877+0000 7efe22fee640 1 --2- 172.21.15.69:0/2591659744 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe1c105090 0x7efe1c1054a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7efe10009900 tx=0x7efe1002a410 comp rx=0 tx=0).stop 2024-04-25T11:38:28.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.877+0000 7efe22fee640 1 -- 172.21.15.69:0/2591659744 shutdown_connections 2024-04-25T11:38:28.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.877+0000 7efe22fee640 1 --2- 172.21.15.69:0/2591659744 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe1c102240 0x7efe1c10bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:28.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.877+0000 7efe22fee640 1 --2- 172.21.15.69:0/2591659744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe1c101740 0x7efe1c101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:28.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.877+0000 7efe22fee640 1 --2- 172.21.15.69:0/2591659744 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe1c105090 0x7efe1c1054a0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:28.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.877+0000 7efe22fee640 1 -- 172.21.15.69:0/2591659744 >> 172.21.15.69:0/2591659744 conn(0x7efe1c0fd340 msgr2=0x7efe1c0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:28.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.877+0000 7efe22fee640 1 -- 172.21.15.69:0/2591659744 shutdown_connections 2024-04-25T11:38:28.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.877+0000 7efe22fee640 1 -- 172.21.15.69:0/2591659744 wait complete. 2024-04-25T11:38:28.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.877+0000 7efe22fee640 1 Processor -- start 2024-04-25T11:38:28.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.877+0000 7efe22fee640 1 -- start start 2024-04-25T11:38:28.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.878+0000 7efe22fee640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe1c101740 0x7efe1c104240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:28.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.878+0000 7efe22fee640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe1c102240 0x7efe1c1028e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:28.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.878+0000 7efe22fee640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe1c105090 0x7efe1c102e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:28.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.878+0000 7efe22fee640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efe1c10f490 con 0x7efe1c102240 2024-04-25T11:38:28.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.878+0000 7efe22fee640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7efe1c10f310 con 0x7efe1c101740 2024-04-25T11:38:28.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.878+0000 7efe22fee640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7efe1c10f610 con 0x7efe1c105090 2024-04-25T11:38:28.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.878+0000 7efe1bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe1c102240 0x7efe1c1028e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:28.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.878+0000 7efe1bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe1c102240 0x7efe1c1028e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59566/0 (socket says 172.21.15.69:59566) 2024-04-25T11:38:28.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.878+0000 7efe1bfff640 1 -- 172.21.15.69:0/3475060972 learned_addr learned my addr 172.21.15.69:0/3475060972 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:38:28.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.878+0000 7efe20d8a640 1 --2- 172.21.15.69:0/3475060972 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe1c101740 0x7efe1c104240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:28.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.878+0000 7efe2158b640 1 --2- 172.21.15.69:0/3475060972 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe1c105090 0x7efe1c102e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:28.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.878+0000 7efe1bfff640 1 -- 172.21.15.69:0/3475060972 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe1c105090 msgr2=0x7efe1c102e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:28.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.878+0000 7efe1bfff640 1 --2- 172.21.15.69:0/3475060972 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe1c105090 0x7efe1c102e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:28.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.878+0000 7efe1bfff640 1 -- 172.21.15.69:0/3475060972 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe1c101740 msgr2=0x7efe1c104240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:28.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.878+0000 7efe1bfff640 1 --2- 172.21.15.69:0/3475060972 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe1c101740 0x7efe1c104240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:28.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.878+0000 7efe1bfff640 1 -- 172.21.15.69:0/3475060972 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe1c1034c0 con 0x7efe1c102240 2024-04-25T11:38:28.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.878+0000 7efe1bfff640 1 --2- 172.21.15.69:0/3475060972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe1c102240 0x7efe1c1028e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7efe0800d950 tx=0x7efe0800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:28.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.878+0000 7efe19ffb640 1 -- 172.21.15.69:0/3475060972 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe08017070 con 0x7efe1c102240 2024-04-25T11:38:28.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.878+0000 7efe22fee640 1 -- 172.21.15.69:0/3475060972 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe1c1a4250 con 0x7efe1c102240 2024-04-25T11:38:28.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.878+0000 7efe19ffb640 1 -- 172.21.15.69:0/3475060972 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe08009c30 con 0x7efe1c102240 2024-04-25T11:38:28.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.878+0000 7efe19ffb640 1 -- 172.21.15.69:0/3475060972 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0800f620 con 0x7efe1c102240 2024-04-25T11:38:28.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.879+0000 7efe22fee640 1 -- 172.21.15.69:0/3475060972 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe1c1a46b0 con 0x7efe1c102240 2024-04-25T11:38:28.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.880+0000 7efe22fee640 1 -- 172.21.15.69:0/3475060972 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe1c108a20 con 0x7efe1c102240 2024-04-25T11:38:28.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.882+0000 7efe19ffb640 1 -- 172.21.15.69:0/3475060972 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7efe08010460 con 0x7efe1c102240 2024-04-25T11:38:28.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.882+0000 7efe19ffb640 1 --2- 172.21.15.69:0/3475060972 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7efe0c0afaf0 0x7efe0c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:28.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.883+0000 7efe20d8a640 1 --2- 172.21.15.69:0/3475060972 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7efe0c0afaf0 0x7efe0c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:28.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.883+0000 7efe19ffb640 1 -- 172.21.15.69:0/3475060972 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9646+0+0 (secure 0 0 0) 0x7efe080ded00 con 0x7efe1c102240 2024-04-25T11:38:28.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.883+0000 7efe20d8a640 1 --2- 172.21.15.69:0/3475060972 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7efe0c0afaf0 0x7efe0c0b2000 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7efe10005760 tx=0x7efe10000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:28.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:28.886+0000 7efe19ffb640 1 -- 172.21.15.69:0/3475060972 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efe08009de0 con 0x7efe1c102240 2024-04-25T11:38:29.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.071+0000 7efe22fee640 1 -- 172.21.15.69:0/3475060972 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7efe1c02db50 con 0x7efe1c102240 2024-04-25T11:38:29.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:29 smithi146 ceph-mon[32821]: pgmap v733: 129 pgs: 129 active+clean; 7.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-04-25T11:38:29.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:29 smithi146 ceph-mon[32821]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T11:38:29.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:29 smithi146 ceph-mon[32821]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T11:38:29.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:29 smithi146 ceph-mon[32821]: Cluster is now healthy 2024-04-25T11:38:29.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:29 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/2225442652' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T11:38:29.593 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:29 smithi146 ceph-mon[32821]: mds.? [v2:172.21.15.152:6832/945520752,v1:172.21.15.152:6834/945520752] up:boot 2024-04-25T11:38:29.594 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:29 smithi146 ceph-mon[32821]: mds.? [v2:172.21.15.152:6833/3850116389,v1:172.21.15.152:6835/3850116389] up:boot 2024-04-25T11:38:29.594 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:29 smithi146 ceph-mon[32821]: mds.? [v2:172.21.15.69:6838/3093631629,v1:172.21.15.69:6840/3093631629] up:boot 2024-04-25T11:38:29.594 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:29 smithi146 ceph-mon[32821]: mds.? [v2:172.21.15.69:6839/2718054972,v1:172.21.15.69:6841/2718054972] up:boot 2024-04-25T11:38:29.594 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:29 smithi146 ceph-mon[32821]: mds.? [v2:172.21.15.146:6833/2298224344,v1:172.21.15.146:6835/2298224344] up:boot 2024-04-25T11:38:29.594 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:29 smithi146 ceph-mon[32821]: mds.? [v2:172.21.15.146:6832/3606957983,v1:172.21.15.146:6834/3606957983] up:boot 2024-04-25T11:38:29.594 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:29 smithi146 ceph-mon[32821]: mds.? [v2:172.21.15.152:6836/3793751649,v1:172.21.15.152:6839/3793751649] up:boot 2024-04-25T11:38:29.594 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:29 smithi146 ceph-mon[32821]: mds.? [v2:172.21.15.152:6837/3797791898,v1:172.21.15.152:6838/3797791898] up:boot 2024-04-25T11:38:29.594 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:29 smithi146 ceph-mon[32821]: mds.? [v2:172.21.15.146:6836/2574434917,v1:172.21.15.146:6838/2574434917] up:boot 2024-04-25T11:38:29.594 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:29 smithi146 ceph-mon[32821]: mds.? [v2:172.21.15.146:6837/2152729845,v1:172.21.15.146:6839/2152729845] up:boot 2024-04-25T11:38:29.594 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:29 smithi146 ceph-mon[32821]: fsmap 12 up:standby 2024-04-25T11:38:29.594 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:29 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T11:38:29.594 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:29 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T11:38:29.595 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:29 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T11:38:29.595 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:29 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T11:38:29.595 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:29 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T11:38:29.595 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:29 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T11:38:29.595 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:29 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T11:38:29.595 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:29 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T11:38:29.595 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:29 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T11:38:29.595 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:29 smithi146 ceph-mon[32821]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T11:38:29.595 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:29 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/3475060972' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T11:38:29.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:29 smithi069 ceph-mon[29188]: pgmap v733: 129 pgs: 129 active+clean; 7.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-04-25T11:38:29.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:29 smithi069 ceph-mon[29188]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T11:38:29.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:29 smithi069 ceph-mon[29188]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T11:38:29.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:29 smithi069 ceph-mon[29188]: Cluster is now healthy 2024-04-25T11:38:29.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:29 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/2225442652' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T11:38:29.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:29 smithi069 ceph-mon[29188]: mds.? [v2:172.21.15.152:6832/945520752,v1:172.21.15.152:6834/945520752] up:boot 2024-04-25T11:38:29.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:29 smithi069 ceph-mon[29188]: mds.? [v2:172.21.15.152:6833/3850116389,v1:172.21.15.152:6835/3850116389] up:boot 2024-04-25T11:38:29.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:29 smithi069 ceph-mon[29188]: mds.? [v2:172.21.15.69:6838/3093631629,v1:172.21.15.69:6840/3093631629] up:boot 2024-04-25T11:38:29.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:29 smithi069 ceph-mon[29188]: mds.? [v2:172.21.15.69:6839/2718054972,v1:172.21.15.69:6841/2718054972] up:boot 2024-04-25T11:38:29.687 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:29 smithi069 ceph-mon[29188]: mds.? [v2:172.21.15.146:6833/2298224344,v1:172.21.15.146:6835/2298224344] up:boot 2024-04-25T11:38:29.688 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:29 smithi069 ceph-mon[29188]: mds.? [v2:172.21.15.146:6832/3606957983,v1:172.21.15.146:6834/3606957983] up:boot 2024-04-25T11:38:29.688 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:29 smithi069 ceph-mon[29188]: mds.? [v2:172.21.15.152:6836/3793751649,v1:172.21.15.152:6839/3793751649] up:boot 2024-04-25T11:38:29.688 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:29 smithi069 ceph-mon[29188]: mds.? [v2:172.21.15.152:6837/3797791898,v1:172.21.15.152:6838/3797791898] up:boot 2024-04-25T11:38:29.688 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:29 smithi069 ceph-mon[29188]: mds.? [v2:172.21.15.146:6836/2574434917,v1:172.21.15.146:6838/2574434917] up:boot 2024-04-25T11:38:29.688 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:29 smithi069 ceph-mon[29188]: mds.? [v2:172.21.15.146:6837/2152729845,v1:172.21.15.146:6839/2152729845] up:boot 2024-04-25T11:38:29.688 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:29 smithi069 ceph-mon[29188]: fsmap 12 up:standby 2024-04-25T11:38:29.688 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:29 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T11:38:29.688 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:29 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T11:38:29.688 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:29 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T11:38:29.688 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:29 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T11:38:29.688 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:29 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T11:38:29.688 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:29 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T11:38:29.688 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:29 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T11:38:29.688 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:29 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T11:38:29.688 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:29 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T11:38:29.689 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:29 smithi069 ceph-mon[29188]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T11:38:29.689 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:29 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/3475060972' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T11:38:29.724 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:29 smithi152 ceph-mon[32701]: pgmap v733: 129 pgs: 129 active+clean; 7.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-04-25T11:38:29.724 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:29 smithi152 ceph-mon[32701]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T11:38:29.724 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:29 smithi152 ceph-mon[32701]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T11:38:29.725 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:29 smithi152 ceph-mon[32701]: Cluster is now healthy 2024-04-25T11:38:29.725 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:29 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/2225442652' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T11:38:29.725 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:29 smithi152 ceph-mon[32701]: mds.? [v2:172.21.15.152:6832/945520752,v1:172.21.15.152:6834/945520752] up:boot 2024-04-25T11:38:29.725 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:29 smithi152 ceph-mon[32701]: mds.? [v2:172.21.15.152:6833/3850116389,v1:172.21.15.152:6835/3850116389] up:boot 2024-04-25T11:38:29.725 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:29 smithi152 ceph-mon[32701]: mds.? [v2:172.21.15.69:6838/3093631629,v1:172.21.15.69:6840/3093631629] up:boot 2024-04-25T11:38:29.725 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:29 smithi152 ceph-mon[32701]: mds.? [v2:172.21.15.69:6839/2718054972,v1:172.21.15.69:6841/2718054972] up:boot 2024-04-25T11:38:29.725 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:29 smithi152 ceph-mon[32701]: mds.? [v2:172.21.15.146:6833/2298224344,v1:172.21.15.146:6835/2298224344] up:boot 2024-04-25T11:38:29.725 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:29 smithi152 ceph-mon[32701]: mds.? [v2:172.21.15.146:6832/3606957983,v1:172.21.15.146:6834/3606957983] up:boot 2024-04-25T11:38:29.725 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:29 smithi152 ceph-mon[32701]: mds.? [v2:172.21.15.152:6836/3793751649,v1:172.21.15.152:6839/3793751649] up:boot 2024-04-25T11:38:29.725 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:29 smithi152 ceph-mon[32701]: mds.? [v2:172.21.15.152:6837/3797791898,v1:172.21.15.152:6838/3797791898] up:boot 2024-04-25T11:38:29.725 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:29 smithi152 ceph-mon[32701]: mds.? [v2:172.21.15.146:6836/2574434917,v1:172.21.15.146:6838/2574434917] up:boot 2024-04-25T11:38:29.725 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:29 smithi152 ceph-mon[32701]: mds.? [v2:172.21.15.146:6837/2152729845,v1:172.21.15.146:6839/2152729845] up:boot 2024-04-25T11:38:29.725 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:29 smithi152 ceph-mon[32701]: fsmap 12 up:standby 2024-04-25T11:38:29.725 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:29 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T11:38:29.726 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:29 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T11:38:29.726 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:29 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T11:38:29.726 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:29 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T11:38:29.726 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:29 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T11:38:29.726 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:29 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T11:38:29.726 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:29 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T11:38:29.726 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:29 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T11:38:29.726 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:29 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T11:38:29.726 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:29 smithi152 ceph-mon[32701]: from='mgr.14152 172.21.15.69:0/3548157125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T11:38:29.726 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:29 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/3475060972' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T11:38:29.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.744+0000 7efe19ffb640 1 -- 172.21.15.69:0/3475060972 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v86) ==== 181+0+0 (secure 0 0 0) 0x7efe080a6d10 con 0x7efe1c102240 2024-04-25T11:38:29.732 INFO:teuthology.orchestra.run.smithi069.stderr:pool 'cephfs_metadata' removed 2024-04-25T11:38:29.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.747+0000 7efe22fee640 1 -- 172.21.15.69:0/3475060972 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7efe0c0afaf0 msgr2=0x7efe0c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:29.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.747+0000 7efe22fee640 1 --2- 172.21.15.69:0/3475060972 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7efe0c0afaf0 0x7efe0c0b2000 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7efe10005760 tx=0x7efe10000bc0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.747+0000 7efe22fee640 1 -- 172.21.15.69:0/3475060972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe1c102240 msgr2=0x7efe1c1028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:29.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.747+0000 7efe22fee640 1 --2- 172.21.15.69:0/3475060972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe1c102240 0x7efe1c1028e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7efe0800d950 tx=0x7efe0800de10 comp rx=0 tx=0).stop 2024-04-25T11:38:29.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.747+0000 7efe22fee640 1 -- 172.21.15.69:0/3475060972 shutdown_connections 2024-04-25T11:38:29.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.747+0000 7efe22fee640 1 --2- 172.21.15.69:0/3475060972 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7efe0c0afaf0 0x7efe0c0b2000 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.747+0000 7efe22fee640 1 --2- 172.21.15.69:0/3475060972 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe1c105090 0x7efe1c102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.747+0000 7efe22fee640 1 --2- 172.21.15.69:0/3475060972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe1c102240 0x7efe1c1028e0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.747+0000 7efe22fee640 1 --2- 172.21.15.69:0/3475060972 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efe1c101740 0x7efe1c104240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.747+0000 7efe22fee640 1 -- 172.21.15.69:0/3475060972 >> 172.21.15.69:0/3475060972 conn(0x7efe1c0fd340 msgr2=0x7efe1c105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:29.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.747+0000 7efe22fee640 1 -- 172.21.15.69:0/3475060972 shutdown_connections 2024-04-25T11:38:29.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.747+0000 7efe22fee640 1 -- 172.21.15.69:0/3475060972 wait complete. 2024-04-25T11:38:29.748 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T11:38:29.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.874+0000 7f4b1dbce640 1 -- 172.21.15.69:0/751024809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b18103c10 msgr2=0x7f4b18104020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:29.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.874+0000 7f4b1dbce640 1 --2- 172.21.15.69:0/751024809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b18103c10 0x7f4b18104020 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c009900 tx=0x7f4b0c02a410 comp rx=0 tx=0).stop 2024-04-25T11:38:29.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.874+0000 7f4b1dbce640 1 -- 172.21.15.69:0/751024809 shutdown_connections 2024-04-25T11:38:29.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.874+0000 7f4b1dbce640 1 --2- 172.21.15.69:0/751024809 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b18108910 0x7f4b1810ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.874+0000 7f4b1dbce640 1 --2- 172.21.15.69:0/751024809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b18103c10 0x7f4b18104020 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.874+0000 7f4b1dbce640 1 --2- 172.21.15.69:0/751024809 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b181029d0 0x7f4b181036d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.874+0000 7f4b1dbce640 1 -- 172.21.15.69:0/751024809 >> 172.21.15.69:0/751024809 conn(0x7f4b180fdba0 msgr2=0x7f4b18100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:29.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.874+0000 7f4b1dbce640 1 -- 172.21.15.69:0/751024809 shutdown_connections 2024-04-25T11:38:29.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.874+0000 7f4b1dbce640 1 -- 172.21.15.69:0/751024809 wait complete. 2024-04-25T11:38:29.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.875+0000 7f4b1dbce640 1 Processor -- start 2024-04-25T11:38:29.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.875+0000 7f4b1dbce640 1 -- start start 2024-04-25T11:38:29.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.875+0000 7f4b1dbce640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b181029d0 0x7f4b1806bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:29.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.875+0000 7f4b1dbce640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b18103c10 0x7f4b181a12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:29.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.875+0000 7f4b1dbce640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b18108910 0x7f4b181a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:29.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.875+0000 7f4b1dbce640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1810e380 con 0x7f4b181029d0 2024-04-25T11:38:29.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.875+0000 7f4b1dbce640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1810e200 con 0x7f4b18108910 2024-04-25T11:38:29.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.875+0000 7f4b1dbce640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1810e500 con 0x7f4b18103c10 2024-04-25T11:38:29.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.875+0000 7f4b177fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b181029d0 0x7f4b1806bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:29.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.875+0000 7f4b177fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b181029d0 0x7f4b1806bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59580/0 (socket says 172.21.15.69:59580) 2024-04-25T11:38:29.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.875+0000 7f4b177fe640 1 -- 172.21.15.69:0/3155189807 learned_addr learned my addr 172.21.15.69:0/3155189807 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:38:29.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.875+0000 7f4b16ffd640 1 --2- 172.21.15.69:0/3155189807 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b18103c10 0x7f4b181a12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:29.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.875+0000 7f4b17fff640 1 --2- 172.21.15.69:0/3155189807 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b18108910 0x7f4b181a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:29.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.875+0000 7f4b177fe640 1 -- 172.21.15.69:0/3155189807 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b18103c10 msgr2=0x7f4b181a12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:29.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.875+0000 7f4b177fe640 1 --2- 172.21.15.69:0/3155189807 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b18103c10 0x7f4b181a12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.875+0000 7f4b177fe640 1 -- 172.21.15.69:0/3155189807 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b18108910 msgr2=0x7f4b181a3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:29.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.875+0000 7f4b177fe640 1 --2- 172.21.15.69:0/3155189807 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b18108910 0x7f4b181a3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:29.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.875+0000 7f4b177fe640 1 -- 172.21.15.69:0/3155189807 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b181a3e50 con 0x7f4b181029d0 2024-04-25T11:38:29.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.875+0000 7f4b17fff640 1 --2- 172.21.15.69:0/3155189807 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b18108910 0x7f4b181a3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:38:29.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.876+0000 7f4b177fe640 1 --2- 172.21.15.69:0/3155189807 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b181029d0 0x7f4b1806bd40 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4b0800c970 tx=0x7f4b0800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:29.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.876+0000 7f4b14ff9640 1 -- 172.21.15.69:0/3155189807 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b08004220 con 0x7f4b181029d0 2024-04-25T11:38:29.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.876+0000 7f4b14ff9640 1 -- 172.21.15.69:0/3155189807 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4b08007e40 con 0x7f4b181029d0 2024-04-25T11:38:29.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.876+0000 7f4b1dbce640 1 -- 172.21.15.69:0/3155189807 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b181a4140 con 0x7f4b181029d0 2024-04-25T11:38:29.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.876+0000 7f4b14ff9640 1 -- 172.21.15.69:0/3155189807 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0800f450 con 0x7f4b181029d0 2024-04-25T11:38:29.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.876+0000 7f4b1dbce640 1 -- 172.21.15.69:0/3155189807 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b181a4690 con 0x7f4b181029d0 2024-04-25T11:38:29.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.877+0000 7f4b1dbce640 1 -- 172.21.15.69:0/3155189807 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b18107720 con 0x7f4b181029d0 2024-04-25T11:38:29.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.878+0000 7f4b14ff9640 1 -- 172.21.15.69:0/3155189807 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f4b080079b0 con 0x7f4b181029d0 2024-04-25T11:38:29.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.879+0000 7f4b14ff9640 1 --2- 172.21.15.69:0/3155189807 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f4b000afaf0 0x7f4b000b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:29.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.879+0000 7f4b16ffd640 1 --2- 172.21.15.69:0/3155189807 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f4b000afaf0 0x7f4b000b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:29.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.879+0000 7f4b14ff9640 1 -- 172.21.15.69:0/3155189807 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9063+0+0 (secure 0 0 0) 0x7f4b080df0b0 con 0x7f4b181029d0 2024-04-25T11:38:29.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.880+0000 7f4b16ffd640 1 --2- 172.21.15.69:0/3155189807 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f4b000afaf0 0x7f4b000b2000 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c006010 tx=0x7f4b0c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:29.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:29.887+0000 7f4b14ff9640 1 -- 172.21.15.69:0/3155189807 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4b08014030 con 0x7f4b181029d0 2024-04-25T11:38:30.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.077+0000 7f4b1dbce640 1 -- 172.21.15.69:0/3155189807 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4b1802db50 con 0x7f4b181029d0 2024-04-25T11:38:30.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.750+0000 7f4b14ff9640 1 -- 172.21.15.69:0/3155189807 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v87) ==== 169+0+0 (secure 0 0 0) 0x7f4b080a7950 con 0x7f4b181029d0 2024-04-25T11:38:30.739 INFO:teuthology.orchestra.run.smithi069.stderr:pool 'cephfs_data' removed 2024-04-25T11:38:30.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.754+0000 7f4b1dbce640 1 -- 172.21.15.69:0/3155189807 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f4b000afaf0 msgr2=0x7f4b000b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:30.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.754+0000 7f4b1dbce640 1 --2- 172.21.15.69:0/3155189807 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f4b000afaf0 0x7f4b000b2000 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c006010 tx=0x7f4b0c000bc0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.754+0000 7f4b1dbce640 1 -- 172.21.15.69:0/3155189807 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b181029d0 msgr2=0x7f4b1806bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:30.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.754+0000 7f4b1dbce640 1 --2- 172.21.15.69:0/3155189807 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b181029d0 0x7f4b1806bd40 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4b0800c970 tx=0x7f4b0800ce30 comp rx=0 tx=0).stop 2024-04-25T11:38:30.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.754+0000 7f4b1dbce640 1 -- 172.21.15.69:0/3155189807 shutdown_connections 2024-04-25T11:38:30.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.754+0000 7f4b1dbce640 1 --2- 172.21.15.69:0/3155189807 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f4b000afaf0 0x7f4b000b2000 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.754+0000 7f4b1dbce640 1 --2- 172.21.15.69:0/3155189807 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4b18108910 0x7f4b181a3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.754+0000 7f4b1dbce640 1 --2- 172.21.15.69:0/3155189807 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b18103c10 0x7f4b181a12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.754+0000 7f4b1dbce640 1 --2- 172.21.15.69:0/3155189807 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b181029d0 0x7f4b1806bd40 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.754+0000 7f4b1dbce640 1 -- 172.21.15.69:0/3155189807 >> 172.21.15.69:0/3155189807 conn(0x7f4b180fdba0 msgr2=0x7f4b181048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:30.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.754+0000 7f4b1dbce640 1 -- 172.21.15.69:0/3155189807 shutdown_connections 2024-04-25T11:38:30.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.754+0000 7f4b1dbce640 1 -- 172.21.15.69:0/3155189807 wait complete. 2024-04-25T11:38:30.758 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-25T11:38:30.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.886+0000 7f1795738640 1 -- 172.21.15.69:0/2830714052 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17901029e0 msgr2=0x7f17901036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:30.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.886+0000 7f1795738640 1 --2- 172.21.15.69:0/2830714052 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17901029e0 0x7f17901036e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f1784009900 tx=0x7f178402a410 comp rx=0 tx=0).stop 2024-04-25T11:38:30.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.886+0000 7f1795738640 1 -- 172.21.15.69:0/2830714052 shutdown_connections 2024-04-25T11:38:30.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.886+0000 7f1795738640 1 --2- 172.21.15.69:0/2830714052 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1790108920 0x7f179010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.886+0000 7f1795738640 1 --2- 172.21.15.69:0/2830714052 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1790103c20 0x7f1790104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.886+0000 7f1795738640 1 --2- 172.21.15.69:0/2830714052 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17901029e0 0x7f17901036e0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.886+0000 7f1795738640 1 -- 172.21.15.69:0/2830714052 >> 172.21.15.69:0/2830714052 conn(0x7f17900fdbd0 msgr2=0x7f1790100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:30.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.886+0000 7f1795738640 1 -- 172.21.15.69:0/2830714052 shutdown_connections 2024-04-25T11:38:30.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.886+0000 7f1795738640 1 -- 172.21.15.69:0/2830714052 wait complete. 2024-04-25T11:38:30.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.887+0000 7f1795738640 1 Processor -- start 2024-04-25T11:38:30.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.887+0000 7f1795738640 1 -- start start 2024-04-25T11:38:30.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.887+0000 7f1795738640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17901029e0 0x7f179019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:30.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.887+0000 7f1795738640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1790103c20 0x7f179019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:30.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.887+0000 7f1795738640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1790108920 0x7f17901a1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:30.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.887+0000 7f1795738640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f179010f4c0 con 0x7f1790108920 2024-04-25T11:38:30.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.887+0000 7f1795738640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f179010f340 con 0x7f17901029e0 2024-04-25T11:38:30.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.887+0000 7f1795738640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f179010f640 con 0x7f1790103c20 2024-04-25T11:38:30.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.887+0000 7f178f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1790108920 0x7f17901a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:30.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.887+0000 7f178f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1790108920 0x7f17901a1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59590/0 (socket says 172.21.15.69:59590) 2024-04-25T11:38:30.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.887+0000 7f178f7fe640 1 -- 172.21.15.69:0/4229228982 learned_addr learned my addr 172.21.15.69:0/4229228982 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T11:38:30.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.887+0000 7f178effd640 1 --2- 172.21.15.69:0/4229228982 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17901029e0 0x7f179019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:30.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.887+0000 7f178e7fc640 1 --2- 172.21.15.69:0/4229228982 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1790103c20 0x7f179019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:30.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.887+0000 7f178f7fe640 1 -- 172.21.15.69:0/4229228982 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1790103c20 msgr2=0x7f179019a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:30.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.887+0000 7f178f7fe640 1 --2- 172.21.15.69:0/4229228982 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1790103c20 0x7f179019a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.887+0000 7f178f7fe640 1 -- 172.21.15.69:0/4229228982 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17901029e0 msgr2=0x7f179019a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:30.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.887+0000 7f178f7fe640 1 --2- 172.21.15.69:0/4229228982 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17901029e0 0x7f179019a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:30.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.887+0000 7f178f7fe640 1 -- 172.21.15.69:0/4229228982 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17901a1ff0 con 0x7f1790108920 2024-04-25T11:38:30.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.888+0000 7f178f7fe640 1 --2- 172.21.15.69:0/4229228982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1790108920 0x7f17901a1980 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1788009fd0 tx=0x7f178800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:30.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.888+0000 7f1777fff640 1 -- 172.21.15.69:0/4229228982 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1788017070 con 0x7f1790108920 2024-04-25T11:38:30.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.888+0000 7f1777fff640 1 -- 172.21.15.69:0/4229228982 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1788003e00 con 0x7f1790108920 2024-04-25T11:38:30.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.888+0000 7f1795738640 1 -- 172.21.15.69:0/4229228982 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17901a22e0 con 0x7f1790108920 2024-04-25T11:38:30.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.888+0000 7f1777fff640 1 -- 172.21.15.69:0/4229228982 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1788010460 con 0x7f1790108920 2024-04-25T11:38:30.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.888+0000 7f1795738640 1 -- 172.21.15.69:0/4229228982 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17901a2830 con 0x7f1790108920 2024-04-25T11:38:30.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.889+0000 7f1795738640 1 -- 172.21.15.69:0/4229228982 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1790107730 con 0x7f1790108920 2024-04-25T11:38:30.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.890+0000 7f1777fff640 1 -- 172.21.15.69:0/4229228982 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 20) ==== 147520+0+0 (secure 0 0 0) 0x7f1788003970 con 0x7f1790108920 2024-04-25T11:38:30.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.891+0000 7f1777fff640 1 --2- 172.21.15.69:0/4229228982 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f17780afb70 0x7f17780b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:30.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.891+0000 7f178effd640 1 --2- 172.21.15.69:0/4229228982 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f17780afb70 0x7f17780b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:30.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.891+0000 7f1777fff640 1 -- 172.21.15.69:0/4229228982 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8496+0+0 (secure 0 0 0) 0x7f17880dee70 con 0x7f1790108920 2024-04-25T11:38:30.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.891+0000 7f178effd640 1 --2- 172.21.15.69:0/4229228982 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f17780afb70 0x7f17780b2080 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1784005760 tx=0x7f1784000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:30.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:30.894+0000 7f1777fff640 1 -- 172.21.15.69:0/4229228982 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f17880a79d0 con 0x7f1790108920 2024-04-25T11:38:31.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:31.073+0000 7f1795738640 1 -- 172.21.15.69:0/4229228982 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f179006c2a0 con 0x7f1790108920 2024-04-25T11:38:31.092 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:30 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/3475060972' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T11:38:31.093 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:30 smithi146 ceph-mon[32821]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T11:38:31.093 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:30 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/3155189807' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T11:38:31.186 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:30 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/3475060972' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T11:38:31.186 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:30 smithi069 ceph-mon[29188]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T11:38:31.187 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:30 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/3155189807' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T11:38:31.224 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:30 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/3475060972' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T11:38:31.224 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:30 smithi152 ceph-mon[32701]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T11:38:31.224 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:30 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/3155189807' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T11:38:31.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:31.757+0000 7f1777fff640 1 -- 172.21.15.69:0/4229228982 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v88) ==== 178+0+0 (secure 0 0 0) 0x7f17880a72d0 con 0x7f1790108920 2024-04-25T11:38:31.746 INFO:teuthology.orchestra.run.smithi069.stderr:pool 'cephfs_data_ec' removed 2024-04-25T11:38:31.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:31.760+0000 7f1795738640 1 -- 172.21.15.69:0/4229228982 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f17780afb70 msgr2=0x7f17780b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:31.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:31.760+0000 7f1795738640 1 --2- 172.21.15.69:0/4229228982 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f17780afb70 0x7f17780b2080 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1784005760 tx=0x7f1784000bc0 comp rx=0 tx=0).stop 2024-04-25T11:38:31.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:31.760+0000 7f1795738640 1 -- 172.21.15.69:0/4229228982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1790108920 msgr2=0x7f17901a1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:31.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:31.760+0000 7f1795738640 1 --2- 172.21.15.69:0/4229228982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1790108920 0x7f17901a1980 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1788009fd0 tx=0x7f178800b260 comp rx=0 tx=0).stop 2024-04-25T11:38:31.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:31.760+0000 7f1795738640 1 -- 172.21.15.69:0/4229228982 shutdown_connections 2024-04-25T11:38:31.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:31.760+0000 7f1795738640 1 --2- 172.21.15.69:0/4229228982 >> [v2:172.21.15.69:6800/4220065409,v1:172.21.15.69:6801/4220065409] conn(0x7f17780afb70 0x7f17780b2080 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:31.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:31.760+0000 7f1795738640 1 --2- 172.21.15.69:0/4229228982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1790108920 0x7f17901a1980 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:31.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:31.760+0000 7f1795738640 1 --2- 172.21.15.69:0/4229228982 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1790103c20 0x7f179019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:31.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:31.760+0000 7f1795738640 1 --2- 172.21.15.69:0/4229228982 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f17901029e0 0x7f179019a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:31.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:31.760+0000 7f1795738640 1 -- 172.21.15.69:0/4229228982 >> 172.21.15.69:0/4229228982 conn(0x7f17900fdbd0 msgr2=0x7f1790104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:31.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:31.761+0000 7f1795738640 1 -- 172.21.15.69:0/4229228982 shutdown_connections 2024-04-25T11:38:31.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T11:38:31.761+0000 7f1795738640 1 -- 172.21.15.69:0/4229228982 wait complete. 2024-04-25T11:38:31.763 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T11:38:31.788 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T11:38:31.821 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T11:38:31.853 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-25T11:38:31.854 DEBUG:teuthology.orchestra.run.smithi069:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T11:38:31.869 DEBUG:teuthology.orchestra.run.smithi146:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T11:38:31.885 DEBUG:teuthology.orchestra.run.smithi152:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T11:38:31.911 INFO:tasks.cephadm:Stopping all daemons... 2024-04-25T11:38:31.912 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-25T11:38:31.912 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mon.a 2024-04-25T11:38:32.092 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:31 smithi146 ceph-mon[32821]: pgmap v735: 97 pgs: 97 active+clean; 577 KiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:38:32.093 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:31 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/3155189807' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T11:38:32.093 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:31 smithi146 ceph-mon[32821]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T11:38:32.093 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:31 smithi146 ceph-mon[32821]: from='client.? 172.21.15.69:0/4229228982' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T11:38:32.186 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:31 smithi069 ceph-mon[29188]: pgmap v735: 97 pgs: 97 active+clean; 577 KiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:38:32.187 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:31 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/3155189807' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T11:38:32.187 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:31 smithi069 ceph-mon[29188]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T11:38:32.187 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:31 smithi069 ceph-mon[29188]: from='client.? 172.21.15.69:0/4229228982' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T11:38:32.187 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:31 smithi069 systemd[1]: Stopping Ceph mon.a for f7ed73dc-02f4-11ef-bc93-c7b262605968... 2024-04-25T11:38:32.224 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:31 smithi152 ceph-mon[32701]: pgmap v735: 97 pgs: 97 active+clean; 577 KiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:38:32.224 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:31 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/3155189807' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T11:38:32.224 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:31 smithi152 ceph-mon[32701]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T11:38:32.224 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:31 smithi152 ceph-mon[32701]: from='client.? 172.21.15.69:0/4229228982' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T11:38:32.840 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:32 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mon-a[29184]: 2024-04-25T11:38:32.492+0000 7f6d7c69e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:38:32.840 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:32 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mon-a[29184]: 2024-04-25T11:38:32.492+0000 7f6d7c69e700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-25T11:38:32.840 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:32 smithi069 podman[88127]: 2024-04-25 11:38:32.545089983 +0000 UTC m=+0.432391622 container died a2cd3fa6825851085799332cf869756d214713181871e6aa820ed8572268610b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-25T11:38:33.187 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:32 smithi069 podman[88127]: 2024-04-25 11:38:32.952463788 +0000 UTC m=+0.839765429 container cleanup a2cd3fa6825851085799332cf869756d214713181871e6aa820ed8572268610b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, ceph=True) 2024-04-25T11:38:33.187 INFO:journalctl@ceph.mon.a.smithi069.stdout:Apr 25 11:38:32 smithi069 bash[88127]: ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mon-a 2024-04-25T11:38:33.355 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mon.a.service' 2024-04-25T11:38:33.399 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:38:33.399 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-25T11:38:33.399 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-25T11:38:33.399 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mon.b 2024-04-25T11:38:33.709 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:33 smithi146 systemd[1]: Stopping Ceph mon.b for f7ed73dc-02f4-11ef-bc93-c7b262605968... 2024-04-25T11:38:34.093 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:33 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mon-b[32817]: 2024-04-25T11:38:33.721+0000 7f7276e54700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:38:34.093 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:33 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mon-b[32817]: 2024-04-25T11:38:33.721+0000 7f7276e54700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-25T11:38:34.094 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:33 smithi146 podman[67412]: 2024-04-25 11:38:33.885484975 +0000 UTC m=+0.276558595 container died 9c36beef881f1863ceb50ecd9019e73af0fe79e2cf43963db81e97882b343cdb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mon-b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T11:38:34.527 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:34 smithi146 podman[67412]: 2024-04-25 11:38:34.264496568 +0000 UTC m=+0.655570188 container cleanup 9c36beef881f1863ceb50ecd9019e73af0fe79e2cf43963db81e97882b343cdb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mon-b, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-25T11:38:34.527 INFO:journalctl@ceph.mon.b.smithi146.stdout:Apr 25 11:38:34 smithi146 bash[67412]: ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mon-b 2024-04-25T11:38:34.669 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mon.b.service' 2024-04-25T11:38:34.720 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:38:34.720 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-25T11:38:34.720 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-25T11:38:34.720 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mon.c 2024-04-25T11:38:35.046 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:34 smithi152 systemd[1]: Stopping Ceph mon.c for f7ed73dc-02f4-11ef-bc93-c7b262605968... 2024-04-25T11:38:35.474 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:35 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mon-c[32684]: 2024-04-25T11:38:35.058+0000 7fc787442700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:38:35.474 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:35 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mon-c[32684]: 2024-04-25T11:38:35.058+0000 7fc787442700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-25T11:38:35.474 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:35 smithi152 podman[69019]: 2024-04-25 11:38:35.283484649 +0000 UTC m=+0.349810727 container died 98da677f5104826629a2a889644244f965324decf7602bc20998963815ff7fc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mon-c, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-25T11:38:35.908 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:35 smithi152 podman[69019]: 2024-04-25 11:38:35.692784928 +0000 UTC m=+0.759111015 container cleanup 98da677f5104826629a2a889644244f965324decf7602bc20998963815ff7fc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mon-c, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-25T11:38:35.908 INFO:journalctl@ceph.mon.c.smithi152.stdout:Apr 25 11:38:35 smithi152 bash[69019]: ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mon-c 2024-04-25T11:38:36.062 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mon.c.service' 2024-04-25T11:38:36.103 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:38:36.103 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-25T11:38:36.104 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-25T11:38:36.104 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mgr.x 2024-04-25T11:38:36.395 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:38:36 smithi069 systemd[1]: Stopping Ceph mgr.x for f7ed73dc-02f4-11ef-bc93-c7b262605968... 2024-04-25T11:38:36.687 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:38:36 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-25T11:38:36.431+0000 7f270528d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:38:36.687 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:38:36 smithi069 podman[88251]: 2024-04-25 11:38:36.462697111 +0000 UTC m=+0.144783140 container died 447f3c9a8cd80fcf7f87c60baededff0c15a2ac69f348c2b27a39408a6957124 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-25T11:38:37.124 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:38:36 smithi069 podman[88251]: 2024-04-25 11:38:36.858118942 +0000 UTC m=+0.540204998 container cleanup 447f3c9a8cd80fcf7f87c60baededff0c15a2ac69f348c2b27a39408a6957124 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-25T11:38:37.124 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Apr 25 11:38:36 smithi069 bash[88251]: ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-x 2024-04-25T11:38:37.279 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-25T11:38:37.313 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:38:37.313 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-25T11:38:37.314 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-25T11:38:37.314 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mgr.y 2024-04-25T11:38:37.593 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Apr 25 11:38:37 smithi146 systemd[1]: Stopping Ceph mgr.y for f7ed73dc-02f4-11ef-bc93-c7b262605968... 2024-04-25T11:38:37.915 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Apr 25 11:38:37 smithi146 podman[67534]: 2024-04-25 11:38:37.664665283 +0000 UTC m=+0.143067206 container died 91e9d14228743e7ef9d70c163a4beb80f5e7240e8ed76761a1d4e0c5e879881e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-y, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-25T11:38:38.174 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Apr 25 11:38:38 smithi146 podman[67534]: 2024-04-25 11:38:38.003344121 +0000 UTC m=+0.481746036 container cleanup 91e9d14228743e7ef9d70c163a4beb80f5e7240e8ed76761a1d4e0c5e879881e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-25T11:38:38.174 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Apr 25 11:38:38 smithi146 bash[67534]: ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-y 2024-04-25T11:38:38.408 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-25T11:38:38.429 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Apr 25 11:38:38 smithi146 podman[67548]: 2024-04-25 11:38:38.18787369 +0000 UTC m=+0.518869853 container remove 91e9d14228743e7ef9d70c163a4beb80f5e7240e8ed76761a1d4e0c5e879881e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-y, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True) 2024-04-25T11:38:38.429 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Apr 25 11:38:38 smithi146 systemd[1]: ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-04-25T11:38:38.430 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Apr 25 11:38:38 smithi146 systemd[1]: ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mgr.y.service: Failed with result 'exit-code'. 2024-04-25T11:38:38.430 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Apr 25 11:38:38 smithi146 systemd[1]: Stopped Ceph mgr.y for f7ed73dc-02f4-11ef-bc93-c7b262605968. 2024-04-25T11:38:38.430 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Apr 25 11:38:38 smithi146 systemd[1]: ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mgr.y.service: Consumed 13.569s CPU time. 2024-04-25T11:38:38.442 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:38:38.442 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-25T11:38:38.442 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-25T11:38:38.442 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mgr.z 2024-04-25T11:38:38.724 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Apr 25 11:38:38 smithi152 systemd[1]: Stopping Ceph mgr.z for f7ed73dc-02f4-11ef-bc93-c7b262605968... 2024-04-25T11:38:39.062 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Apr 25 11:38:38 smithi152 podman[69141]: 2024-04-25 11:38:38.795977315 +0000 UTC m=+0.145747266 container died 36ffa8d398c7360af331d9793c19b9653e3b24df639ff6fe0ce740174f1209cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-z, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T11:38:39.439 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Apr 25 11:38:39 smithi152 podman[69141]: 2024-04-25 11:38:39.184439424 +0000 UTC m=+0.534209380 container cleanup 36ffa8d398c7360af331d9793c19b9653e3b24df639ff6fe0ce740174f1209cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240424) 2024-04-25T11:38:39.439 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Apr 25 11:38:39 smithi152 bash[69141]: ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mgr-z 2024-04-25T11:38:39.589 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-25T11:38:39.623 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:38:39.623 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-25T11:38:39.623 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-25T11:38:39.623 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@osd.0 2024-04-25T11:38:39.933 INFO:journalctl@ceph.osd.0.smithi069.stdout:Apr 25 11:38:39 smithi069 systemd[1]: Stopping Ceph osd.0 for f7ed73dc-02f4-11ef-bc93-c7b262605968... 2024-04-25T11:38:40.186 INFO:journalctl@ceph.osd.0.smithi069.stdout:Apr 25 11:38:39 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-0[44381]: 2024-04-25T11:38:39.945+0000 7f4349057700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:38:40.187 INFO:journalctl@ceph.osd.0.smithi069.stdout:Apr 25 11:38:39 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-0[44381]: 2024-04-25T11:38:39.945+0000 7f4349057700 -1 osd.0 88 *** Got signal Terminated *** 2024-04-25T11:38:40.187 INFO:journalctl@ceph.osd.0.smithi069.stdout:Apr 25 11:38:39 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-0[44381]: 2024-04-25T11:38:39.945+0000 7f4349057700 -1 osd.0 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T11:38:45.687 INFO:journalctl@ceph.osd.0.smithi069.stdout:Apr 25 11:38:45 smithi069 podman[88372]: 2024-04-25 11:38:45.432651069 +0000 UTC m=+5.603979527 container died 232293110a413c770b5d05e6b2c09db9381025b70c53554875585ccf6622302d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T11:38:46.093 INFO:journalctl@ceph.osd.0.smithi069.stdout:Apr 25 11:38:45 smithi069 podman[88372]: 2024-04-25 11:38:45.82151537 +0000 UTC m=+5.992843872 container cleanup 232293110a413c770b5d05e6b2c09db9381025b70c53554875585ccf6622302d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-0, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-25T11:38:46.093 INFO:journalctl@ceph.osd.0.smithi069.stdout:Apr 25 11:38:45 smithi069 bash[88372]: ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-0 2024-04-25T11:38:46.432 INFO:journalctl@ceph.osd.0.smithi069.stdout:Apr 25 11:38:46 smithi069 podman[88401]: 2024-04-25 11:38:46.106168217 +0000 UTC m=+0.673112388 container remove 232293110a413c770b5d05e6b2c09db9381025b70c53554875585ccf6622302d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True) 2024-04-25T11:38:46.686 INFO:journalctl@ceph.osd.0.smithi069.stdout:Apr 25 11:38:46 smithi069 podman[88496]: 2024-04-25 11:38:46.348712848 +0000 UTC m=+0.018015727 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T11:38:46.687 INFO:journalctl@ceph.osd.0.smithi069.stdout:Apr 25 11:38:46 smithi069 podman[88496]: 2024-04-25 11:38:46.465794853 +0000 UTC m=+0.135097731 container create ac82742d58f70c889e9310fa5d7bdaecee18967d52d79242db78b191d6d9cac2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-0-deactivate, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-25T11:38:46.988 INFO:journalctl@ceph.osd.0.smithi069.stdout:Apr 25 11:38:46 smithi069 podman[88496]: 2024-04-25 11:38:46.710250788 +0000 UTC m=+0.379553668 container init ac82742d58f70c889e9310fa5d7bdaecee18967d52d79242db78b191d6d9cac2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True) 2024-04-25T11:38:46.989 INFO:journalctl@ceph.osd.0.smithi069.stdout:Apr 25 11:38:46 smithi069 podman[88496]: 2024-04-25 11:38:46.71325218 +0000 UTC m=+0.382555062 container start ac82742d58f70c889e9310fa5d7bdaecee18967d52d79242db78b191d6d9cac2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T11:38:46.989 INFO:journalctl@ceph.osd.0.smithi069.stdout:Apr 25 11:38:46 smithi069 podman[88496]: 2024-04-25 11:38:46.818617987 +0000 UTC m=+0.487920866 container attach ac82742d58f70c889e9310fa5d7bdaecee18967d52d79242db78b191d6d9cac2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T11:38:47.350 INFO:journalctl@ceph.osd.0.smithi069.stdout:Apr 25 11:38:47 smithi069 podman[88496]: 2024-04-25 11:38:47.002197114 +0000 UTC m=+0.671499985 container died ac82742d58f70c889e9310fa5d7bdaecee18967d52d79242db78b191d6d9cac2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T11:38:47.692 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@osd.0.service' 2024-04-25T11:38:47.731 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:38:47.731 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-25T11:38:47.731 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-25T11:38:47.732 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@osd.1 2024-04-25T11:38:48.052 INFO:journalctl@ceph.osd.1.smithi146.stdout:Apr 25 11:38:47 smithi146 systemd[1]: Stopping Ceph osd.1 for f7ed73dc-02f4-11ef-bc93-c7b262605968... 2024-04-25T11:38:48.342 INFO:journalctl@ceph.osd.1.smithi146.stdout:Apr 25 11:38:48 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-1[37409]: 2024-04-25T11:38:48.064+0000 7f268e925700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:38:48.343 INFO:journalctl@ceph.osd.1.smithi146.stdout:Apr 25 11:38:48 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-1[37409]: 2024-04-25T11:38:48.064+0000 7f268e925700 -1 osd.1 88 *** Got signal Terminated *** 2024-04-25T11:38:48.343 INFO:journalctl@ceph.osd.1.smithi146.stdout:Apr 25 11:38:48 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-1[37409]: 2024-04-25T11:38:48.064+0000 7f268e925700 -1 osd.1 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T11:38:53.843 INFO:journalctl@ceph.osd.1.smithi146.stdout:Apr 25 11:38:53 smithi146 podman[67657]: 2024-04-25 11:38:53.580105524 +0000 UTC m=+5.639083873 container died 4a8ee29f3ed7a88d5496c1f6938a51f2a08bcdeff5187e445dffdbf8fc18fbce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-1, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-25T11:38:54.195 INFO:journalctl@ceph.osd.1.smithi146.stdout:Apr 25 11:38:53 smithi146 podman[67657]: 2024-04-25 11:38:53.990673014 +0000 UTC m=+6.049651371 container cleanup 4a8ee29f3ed7a88d5496c1f6938a51f2a08bcdeff5187e445dffdbf8fc18fbce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-1, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-25T11:38:54.195 INFO:journalctl@ceph.osd.1.smithi146.stdout:Apr 25 11:38:53 smithi146 bash[67657]: ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-1 2024-04-25T11:38:54.593 INFO:journalctl@ceph.osd.1.smithi146.stdout:Apr 25 11:38:54 smithi146 podman[67674]: 2024-04-25 11:38:54.208873978 +0000 UTC m=+0.631106125 container remove 4a8ee29f3ed7a88d5496c1f6938a51f2a08bcdeff5187e445dffdbf8fc18fbce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-1, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T11:38:54.883 INFO:journalctl@ceph.osd.1.smithi146.stdout:Apr 25 11:38:54 smithi146 podman[67775]: 2024-04-25 11:38:54.524381154 +0000 UTC m=+0.018092336 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T11:38:54.883 INFO:journalctl@ceph.osd.1.smithi146.stdout:Apr 25 11:38:54 smithi146 podman[67775]: 2024-04-25 11:38:54.651397762 +0000 UTC m=+0.145108928 container create f0dff336e5c9faa0164c26565452598cf5dccee3c9d0009881f19a3484572a7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-25T11:38:55.197 INFO:journalctl@ceph.osd.1.smithi146.stdout:Apr 25 11:38:54 smithi146 podman[67775]: 2024-04-25 11:38:54.895583664 +0000 UTC m=+0.389294831 container init f0dff336e5c9faa0164c26565452598cf5dccee3c9d0009881f19a3484572a7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-25T11:38:55.197 INFO:journalctl@ceph.osd.1.smithi146.stdout:Apr 25 11:38:54 smithi146 podman[67775]: 2024-04-25 11:38:54.899814066 +0000 UTC m=+0.393525235 container start f0dff336e5c9faa0164c26565452598cf5dccee3c9d0009881f19a3484572a7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-25T11:38:55.197 INFO:journalctl@ceph.osd.1.smithi146.stdout:Apr 25 11:38:54 smithi146 podman[67775]: 2024-04-25 11:38:54.987476075 +0000 UTC m=+0.481187275 container attach f0dff336e5c9faa0164c26565452598cf5dccee3c9d0009881f19a3484572a7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-1-deactivate, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T11:38:55.535 INFO:journalctl@ceph.osd.1.smithi146.stdout:Apr 25 11:38:55 smithi146 podman[67775]: 2024-04-25 11:38:55.210882411 +0000 UTC m=+0.704593578 container died f0dff336e5c9faa0164c26565452598cf5dccee3c9d0009881f19a3484572a7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-25T11:38:55.882 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@osd.1.service' 2024-04-25T11:38:55.922 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:38:55.923 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-25T11:38:55.923 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-25T11:38:55.923 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@osd.2 2024-04-25T11:38:56.224 INFO:journalctl@ceph.osd.2.smithi152.stdout:Apr 25 11:38:55 smithi152 systemd[1]: Stopping Ceph osd.2 for f7ed73dc-02f4-11ef-bc93-c7b262605968... 2024-04-25T11:38:56.724 INFO:journalctl@ceph.osd.2.smithi152.stdout:Apr 25 11:38:56 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-2[37847]: 2024-04-25T11:38:56.304+0000 7f7950b03700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:38:56.724 INFO:journalctl@ceph.osd.2.smithi152.stdout:Apr 25 11:38:56 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-2[37847]: 2024-04-25T11:38:56.304+0000 7f7950b03700 -1 osd.2 88 *** Got signal Terminated *** 2024-04-25T11:38:56.724 INFO:journalctl@ceph.osd.2.smithi152.stdout:Apr 25 11:38:56 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-2[37847]: 2024-04-25T11:38:56.304+0000 7f7950b03700 -1 osd.2 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T11:39:01.974 INFO:journalctl@ceph.osd.2.smithi152.stdout:Apr 25 11:39:01 smithi152 podman[69276]: 2024-04-25 11:39:01.72946045 +0000 UTC m=+5.600175930 container died 679e9636a1e39275bd14023eaf6bc4e925459959d65c060e23ae374890fb44e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424) 2024-04-25T11:39:02.385 INFO:journalctl@ceph.osd.2.smithi152.stdout:Apr 25 11:39:02 smithi152 podman[69276]: 2024-04-25 11:39:02.138811796 +0000 UTC m=+6.009527281 container cleanup 679e9636a1e39275bd14023eaf6bc4e925459959d65c060e23ae374890fb44e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-25T11:39:02.385 INFO:journalctl@ceph.osd.2.smithi152.stdout:Apr 25 11:39:02 smithi152 bash[69276]: ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-2 2024-04-25T11:39:02.724 INFO:journalctl@ceph.osd.2.smithi152.stdout:Apr 25 11:39:02 smithi152 podman[69295]: 2024-04-25 11:39:02.398547192 +0000 UTC m=+0.672210250 container remove 679e9636a1e39275bd14023eaf6bc4e925459959d65c060e23ae374890fb44e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T11:39:03.022 INFO:journalctl@ceph.osd.2.smithi152.stdout:Apr 25 11:39:02 smithi152 podman[69387]: 2024-04-25 11:39:02.655829148 +0000 UTC m=+0.017665293 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T11:39:03.022 INFO:journalctl@ceph.osd.2.smithi152.stdout:Apr 25 11:39:02 smithi152 podman[69387]: 2024-04-25 11:39:02.757842634 +0000 UTC m=+0.119678770 container create e088bae08c2d373f6b5a959ac2e994267a3d83f6c74c97bb3b6aed1c7e84a10c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-25T11:39:03.022 INFO:journalctl@ceph.osd.2.smithi152.stdout:Apr 25 11:39:02 smithi152 podman[69387]: 2024-04-25 11:39:02.952327055 +0000 UTC m=+0.314163186 container init e088bae08c2d373f6b5a959ac2e994267a3d83f6c74c97bb3b6aed1c7e84a10c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-25T11:39:03.022 INFO:journalctl@ceph.osd.2.smithi152.stdout:Apr 25 11:39:02 smithi152 podman[69387]: 2024-04-25 11:39:02.955354637 +0000 UTC m=+0.317190776 container start e088bae08c2d373f6b5a959ac2e994267a3d83f6c74c97bb3b6aed1c7e84a10c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T11:39:03.474 INFO:journalctl@ceph.osd.2.smithi152.stdout:Apr 25 11:39:03 smithi152 podman[69387]: 2024-04-25 11:39:03.03553582 +0000 UTC m=+0.397371967 container attach e088bae08c2d373f6b5a959ac2e994267a3d83f6c74c97bb3b6aed1c7e84a10c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-25T11:39:03.474 INFO:journalctl@ceph.osd.2.smithi152.stdout:Apr 25 11:39:03 smithi152 podman[69387]: 2024-04-25 11:39:03.252111972 +0000 UTC m=+0.613948098 container died e088bae08c2d373f6b5a959ac2e994267a3d83f6c74c97bb3b6aed1c7e84a10c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5) 2024-04-25T11:39:03.935 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@osd.2.service' 2024-04-25T11:39:03.973 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:39:03.973 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-25T11:39:03.973 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-25T11:39:03.973 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@osd.3 2024-04-25T11:39:04.436 INFO:journalctl@ceph.osd.3.smithi069.stdout:Apr 25 11:39:04 smithi069 systemd[1]: Stopping Ceph osd.3 for f7ed73dc-02f4-11ef-bc93-c7b262605968... 2024-04-25T11:39:04.937 INFO:journalctl@ceph.osd.3.smithi069.stdout:Apr 25 11:39:04 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-3[50905]: 2024-04-25T11:39:04.507+0000 7fe666b79700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:39:04.937 INFO:journalctl@ceph.osd.3.smithi069.stdout:Apr 25 11:39:04 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-3[50905]: 2024-04-25T11:39:04.507+0000 7fe666b79700 -1 osd.3 88 *** Got signal Terminated *** 2024-04-25T11:39:04.937 INFO:journalctl@ceph.osd.3.smithi069.stdout:Apr 25 11:39:04 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-3[50905]: 2024-04-25T11:39:04.507+0000 7fe666b79700 -1 osd.3 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T11:39:06.186 INFO:journalctl@ceph.osd.3.smithi069.stdout:Apr 25 11:39:05 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-3[50905]: 2024-04-25T11:39:05.773+0000 7fe66312f700 -1 osd.3 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:43.886885+0000 front 2024-04-25T11:38:43.886942+0000 (oldest deadline 2024-04-25T11:39:05.587155+0000) 2024-04-25T11:39:07.186 INFO:journalctl@ceph.osd.3.smithi069.stdout:Apr 25 11:39:06 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-3[50905]: 2024-04-25T11:39:06.774+0000 7fe66312f700 -1 osd.3 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:43.886885+0000 front 2024-04-25T11:38:43.886942+0000 (oldest deadline 2024-04-25T11:39:05.587155+0000) 2024-04-25T11:39:07.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:06 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:06.921+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:08.186 INFO:journalctl@ceph.osd.3.smithi069.stdout:Apr 25 11:39:07 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-3[50905]: 2024-04-25T11:39:07.780+0000 7fe66312f700 -1 osd.3 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:43.886885+0000 front 2024-04-25T11:38:43.886942+0000 (oldest deadline 2024-04-25T11:39:05.587155+0000) 2024-04-25T11:39:08.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:07 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:07.873+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:09.088 INFO:journalctl@ceph.osd.3.smithi069.stdout:Apr 25 11:39:08 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-3[50905]: 2024-04-25T11:39:08.730+0000 7fe66312f700 -1 osd.3 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:43.886885+0000 front 2024-04-25T11:38:43.886942+0000 (oldest deadline 2024-04-25T11:39:05.587155+0000) 2024-04-25T11:39:09.342 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:08 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:08.886+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:09.724 INFO:journalctl@ceph.osd.5.smithi152.stdout:Apr 25 11:39:09 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-5[44355]: 2024-04-25T11:39:09.518+0000 7f26f0e5f700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.146790+0000 front 2024-04-25T11:38:44.146932+0000 (oldest deadline 2024-04-25T11:39:08.846614+0000) 2024-04-25T11:39:09.725 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:09 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:09.419+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:10.278 INFO:journalctl@ceph.osd.3.smithi069.stdout:Apr 25 11:39:09 smithi069 podman[88607]: 2024-04-25 11:39:09.957086474 +0000 UTC m=+5.779478324 container died 01056ca5a80b8d0c8667250f6d765733a2fd67a45c397a2ce4eed55b26c19f60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-3, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-25T11:39:10.278 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:10 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:10.070+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.662848+0000 front 2024-04-25T11:38:44.663184+0000 (oldest deadline 2024-04-25T11:39:09.363059+0000) 2024-04-25T11:39:10.342 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:09 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:09.883+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:10.687 INFO:journalctl@ceph.osd.3.smithi069.stdout:Apr 25 11:39:10 smithi069 podman[88607]: 2024-04-25 11:39:10.291257293 +0000 UTC m=+6.113649152 container cleanup 01056ca5a80b8d0c8667250f6d765733a2fd67a45c397a2ce4eed55b26c19f60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-3, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux ) 2024-04-25T11:39:10.687 INFO:journalctl@ceph.osd.3.smithi069.stdout:Apr 25 11:39:10 smithi069 bash[88607]: ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-3 2024-04-25T11:39:10.687 INFO:journalctl@ceph.osd.3.smithi069.stdout:Apr 25 11:39:10 smithi069 podman[88624]: 2024-04-25 11:39:10.534431621 +0000 UTC m=+0.572999935 container remove 01056ca5a80b8d0c8667250f6d765733a2fd67a45c397a2ce4eed55b26c19f60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-3, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-25T11:39:10.724 INFO:journalctl@ceph.osd.5.smithi152.stdout:Apr 25 11:39:10 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-5[44355]: 2024-04-25T11:39:10.525+0000 7f26f0e5f700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.146790+0000 front 2024-04-25T11:38:44.146932+0000 (oldest deadline 2024-04-25T11:39:08.846614+0000) 2024-04-25T11:39:10.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:10 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:10.456+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:11.092 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:10 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:10.852+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:11.216 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:11 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:11.115+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.662848+0000 front 2024-04-25T11:38:44.663184+0000 (oldest deadline 2024-04-25T11:39:09.363059+0000) 2024-04-25T11:39:11.216 INFO:journalctl@ceph.osd.3.smithi069.stdout:Apr 25 11:39:10 smithi069 podman[88724]: 2024-04-25 11:39:10.862950508 +0000 UTC m=+0.018316422 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T11:39:11.216 INFO:journalctl@ceph.osd.3.smithi069.stdout:Apr 25 11:39:10 smithi069 podman[88724]: 2024-04-25 11:39:10.99398311 +0000 UTC m=+0.149349001 container create 8c83702d747fd31bd02565d38bbbdd0b7e7f0a10f1dc3e6d3440ef71da1ab1b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-25T11:39:11.514 INFO:journalctl@ceph.osd.3.smithi069.stdout:Apr 25 11:39:11 smithi069 podman[88724]: 2024-04-25 11:39:11.229976665 +0000 UTC m=+0.385342545 container init 8c83702d747fd31bd02565d38bbbdd0b7e7f0a10f1dc3e6d3440ef71da1ab1b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-25T11:39:11.514 INFO:journalctl@ceph.osd.3.smithi069.stdout:Apr 25 11:39:11 smithi069 podman[88724]: 2024-04-25 11:39:11.233166594 +0000 UTC m=+0.388532486 container start 8c83702d747fd31bd02565d38bbbdd0b7e7f0a10f1dc3e6d3440ef71da1ab1b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-25T11:39:11.515 INFO:journalctl@ceph.osd.3.smithi069.stdout:Apr 25 11:39:11 smithi069 podman[88724]: 2024-04-25 11:39:11.305072776 +0000 UTC m=+0.460438664 container attach 8c83702d747fd31bd02565d38bbbdd0b7e7f0a10f1dc3e6d3440ef71da1ab1b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-25T11:39:11.515 INFO:journalctl@ceph.osd.3.smithi069.stdout:Apr 25 11:39:11 smithi069 podman[88724]: 2024-04-25 11:39:11.528035059 +0000 UTC m=+0.683400952 container died 8c83702d747fd31bd02565d38bbbdd0b7e7f0a10f1dc3e6d3440ef71da1ab1b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-25T11:39:11.974 INFO:journalctl@ceph.osd.5.smithi152.stdout:Apr 25 11:39:11 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-5[44355]: 2024-04-25T11:39:11.569+0000 7f26f0e5f700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.146790+0000 front 2024-04-25T11:38:44.146932+0000 (oldest deadline 2024-04-25T11:39:08.846614+0000) 2024-04-25T11:39:11.974 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:11 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:11.500+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:12.092 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:11 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:11.824+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:12.120 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@osd.3.service' 2024-04-25T11:39:12.158 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:39:12.158 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-25T11:39:12.158 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-25T11:39:12.158 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@osd.4 2024-04-25T11:39:12.436 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:12 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:12.137+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.662848+0000 front 2024-04-25T11:38:44.663184+0000 (oldest deadline 2024-04-25T11:39:09.363059+0000) 2024-04-25T11:39:12.530 INFO:journalctl@ceph.osd.4.smithi146.stdout:Apr 25 11:39:12 smithi146 systemd[1]: Stopping Ceph osd.4 for f7ed73dc-02f4-11ef-bc93-c7b262605968... 2024-04-25T11:39:12.820 INFO:journalctl@ceph.osd.4.smithi146.stdout:Apr 25 11:39:12 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-4[43915]: 2024-04-25T11:39:12.542+0000 7f2eac9c2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:39:12.820 INFO:journalctl@ceph.osd.4.smithi146.stdout:Apr 25 11:39:12 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-4[43915]: 2024-04-25T11:39:12.542+0000 7f2eac9c2700 -1 osd.4 88 *** Got signal Terminated *** 2024-04-25T11:39:12.820 INFO:journalctl@ceph.osd.4.smithi146.stdout:Apr 25 11:39:12 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-4[43915]: 2024-04-25T11:39:12.542+0000 7f2eac9c2700 -1 osd.4 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T11:39:12.974 INFO:journalctl@ceph.osd.5.smithi152.stdout:Apr 25 11:39:12 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-5[44355]: 2024-04-25T11:39:12.551+0000 7f26f0e5f700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.146790+0000 front 2024-04-25T11:38:44.146932+0000 (oldest deadline 2024-04-25T11:39:08.846614+0000) 2024-04-25T11:39:12.974 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:12 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:12.488+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:13.092 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:12 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:12.833+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:13.436 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:13 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:13.102+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.662848+0000 front 2024-04-25T11:38:44.663184+0000 (oldest deadline 2024-04-25T11:39:09.363059+0000) 2024-04-25T11:39:13.724 INFO:journalctl@ceph.osd.5.smithi152.stdout:Apr 25 11:39:13 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-5[44355]: 2024-04-25T11:39:13.590+0000 7f26f0e5f700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.146790+0000 front 2024-04-25T11:38:44.146932+0000 (oldest deadline 2024-04-25T11:39:08.846614+0000) 2024-04-25T11:39:13.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:13 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:13.441+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:14.092 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:13 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:13.830+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:14.110 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 25 11:39:13 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-6[57622]: 2024-04-25T11:39:13.854+0000 7fda72835700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:48.953521+0000 front 2024-04-25T11:38:48.953667+0000 (oldest deadline 2024-04-25T11:39:13.653542+0000) 2024-04-25T11:39:14.436 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:14 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:14.122+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.662848+0000 front 2024-04-25T11:38:44.663184+0000 (oldest deadline 2024-04-25T11:39:09.363059+0000) 2024-04-25T11:39:14.724 INFO:journalctl@ceph.osd.5.smithi152.stdout:Apr 25 11:39:14 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-5[44355]: 2024-04-25T11:39:14.639+0000 7f26f0e5f700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.146790+0000 front 2024-04-25T11:38:44.146932+0000 (oldest deadline 2024-04-25T11:39:08.846614+0000) 2024-04-25T11:39:14.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:14 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:14.459+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:15.092 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:14 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:14.808+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:15.186 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 25 11:39:14 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-6[57622]: 2024-04-25T11:39:14.879+0000 7fda72835700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:48.953521+0000 front 2024-04-25T11:38:48.953667+0000 (oldest deadline 2024-04-25T11:39:13.653542+0000) 2024-04-25T11:39:15.187 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:15 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:15.122+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.662848+0000 front 2024-04-25T11:38:44.663184+0000 (oldest deadline 2024-04-25T11:39:09.363059+0000) 2024-04-25T11:39:15.974 INFO:journalctl@ceph.osd.5.smithi152.stdout:Apr 25 11:39:15 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-5[44355]: 2024-04-25T11:39:15.627+0000 7f26f0e5f700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.146790+0000 front 2024-04-25T11:38:44.146932+0000 (oldest deadline 2024-04-25T11:39:08.846614+0000) 2024-04-25T11:39:15.974 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:15 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:15.506+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:16.092 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:15 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:15.834+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:16.186 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 25 11:39:15 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-6[57622]: 2024-04-25T11:39:15.904+0000 7fda72835700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:48.953521+0000 front 2024-04-25T11:38:48.953667+0000 (oldest deadline 2024-04-25T11:39:13.653542+0000) 2024-04-25T11:39:16.187 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:16 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:16.137+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.662848+0000 front 2024-04-25T11:38:44.663184+0000 (oldest deadline 2024-04-25T11:39:09.363059+0000) 2024-04-25T11:39:16.724 INFO:journalctl@ceph.osd.5.smithi152.stdout:Apr 25 11:39:16 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-5[44355]: 2024-04-25T11:39:16.658+0000 7f26f0e5f700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.146790+0000 front 2024-04-25T11:38:44.146932+0000 (oldest deadline 2024-04-25T11:39:08.846614+0000) 2024-04-25T11:39:16.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:16 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:16.465+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:16.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:16 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:16.465+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:39:16.725 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:16 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:16.482+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:52.418405+0000 front 2024-04-25T11:38:52.418385+0000 (oldest deadline 2024-04-25T11:39:15.918284+0000) 2024-04-25T11:39:17.092 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:16 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:16.851+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:17.093 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:17 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:17.017+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.703177+0000 front 2024-04-25T11:38:50.703148+0000 (oldest deadline 2024-04-25T11:39:16.603339+0000) 2024-04-25T11:39:17.186 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 25 11:39:16 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-6[57622]: 2024-04-25T11:39:16.935+0000 7fda72835700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:48.953521+0000 front 2024-04-25T11:38:48.953667+0000 (oldest deadline 2024-04-25T11:39:13.653542+0000) 2024-04-25T11:39:17.187 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:17 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:17.101+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.662848+0000 front 2024-04-25T11:38:44.663184+0000 (oldest deadline 2024-04-25T11:39:09.363059+0000) 2024-04-25T11:39:17.724 INFO:journalctl@ceph.osd.5.smithi152.stdout:Apr 25 11:39:17 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-5[44355]: 2024-04-25T11:39:17.614+0000 7f26f0e5f700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.146790+0000 front 2024-04-25T11:38:44.146932+0000 (oldest deadline 2024-04-25T11:39:08.846614+0000) 2024-04-25T11:39:17.724 INFO:journalctl@ceph.osd.5.smithi152.stdout:Apr 25 11:39:17 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-5[44355]: 2024-04-25T11:39:17.614+0000 7f26f0e5f700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:52.848698+0000 front 2024-04-25T11:38:52.848557+0000 (oldest deadline 2024-04-25T11:39:17.548551+0000) 2024-04-25T11:39:17.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:17 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:17.465+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:52.418405+0000 front 2024-04-25T11:38:52.418385+0000 (oldest deadline 2024-04-25T11:39:15.918284+0000) 2024-04-25T11:39:17.725 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:17 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:17.488+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:17.725 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:17 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:17.488+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:39:17.831 INFO:journalctl@ceph.osd.4.smithi146.stdout:Apr 25 11:39:17 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-4[43915]: 2024-04-25T11:39:17.513+0000 7f2ea8f78700 -1 osd.4 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:52.133178+0000 front 2024-04-25T11:38:52.133214+0000 (oldest deadline 2024-04-25T11:39:17.433383+0000) 2024-04-25T11:39:18.092 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:18 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:18.042+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.703177+0000 front 2024-04-25T11:38:50.703148+0000 (oldest deadline 2024-04-25T11:39:16.603339+0000) 2024-04-25T11:39:18.093 INFO:journalctl@ceph.osd.4.smithi146.stdout:Apr 25 11:39:17 smithi146 podman[67880]: 2024-04-25 11:39:17.84414822 +0000 UTC m=+5.472470888 container died b2786c557fd49dd4758eed2eecac5b47bc8cdec6b8736f7dc31a602f41f08b08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-4, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-25T11:39:18.093 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:17 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:17.894+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:18.093 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:17 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:17.894+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:51.169170+0000 front 2024-04-25T11:38:51.169194+0000 (oldest deadline 2024-04-25T11:39:17.069398+0000) 2024-04-25T11:39:18.436 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 25 11:39:17 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-6[57622]: 2024-04-25T11:39:17.969+0000 7fda72835700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:48.953521+0000 front 2024-04-25T11:38:48.953667+0000 (oldest deadline 2024-04-25T11:39:13.653542+0000) 2024-04-25T11:39:18.437 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:18 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:18.105+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.662848+0000 front 2024-04-25T11:38:44.663184+0000 (oldest deadline 2024-04-25T11:39:09.363059+0000) 2024-04-25T11:39:18.455 INFO:journalctl@ceph.osd.4.smithi146.stdout:Apr 25 11:39:18 smithi146 podman[67880]: 2024-04-25 11:39:18.201615465 +0000 UTC m=+5.829938148 container cleanup b2786c557fd49dd4758eed2eecac5b47bc8cdec6b8736f7dc31a602f41f08b08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-4, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.build-date=20240424) 2024-04-25T11:39:18.455 INFO:journalctl@ceph.osd.4.smithi146.stdout:Apr 25 11:39:18 smithi146 bash[67880]: ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-4 2024-04-25T11:39:18.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:18 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:18.466+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:18.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:18 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:18.466+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:39:18.724 INFO:journalctl@ceph.osd.5.smithi152.stdout:Apr 25 11:39:18 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-5[44355]: 2024-04-25T11:39:18.629+0000 7f26f0e5f700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.146790+0000 front 2024-04-25T11:38:44.146932+0000 (oldest deadline 2024-04-25T11:39:08.846614+0000) 2024-04-25T11:39:18.724 INFO:journalctl@ceph.osd.5.smithi152.stdout:Apr 25 11:39:18 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-5[44355]: 2024-04-25T11:39:18.629+0000 7f26f0e5f700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:52.848698+0000 front 2024-04-25T11:38:52.848557+0000 (oldest deadline 2024-04-25T11:39:17.548551+0000) 2024-04-25T11:39:18.725 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:18 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:18.488+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:52.418405+0000 front 2024-04-25T11:38:52.418385+0000 (oldest deadline 2024-04-25T11:39:15.918284+0000) 2024-04-25T11:39:18.790 INFO:journalctl@ceph.osd.4.smithi146.stdout:Apr 25 11:39:18 smithi146 podman[67899]: 2024-04-25 11:39:18.469250705 +0000 UTC m=+0.618733489 container remove b2786c557fd49dd4758eed2eecac5b47bc8cdec6b8736f7dc31a602f41f08b08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-4, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-25T11:39:19.067 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:19 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:19.029+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.703177+0000 front 2024-04-25T11:38:50.703148+0000 (oldest deadline 2024-04-25T11:39:16.603339+0000) 2024-04-25T11:39:19.068 INFO:journalctl@ceph.osd.4.smithi146.stdout:Apr 25 11:39:18 smithi146 podman[67989]: 2024-04-25 11:39:18.803762642 +0000 UTC m=+0.110675416 container create 5289707eac4517fef70d5bca43e4b1494814b548609fc41e5bbcea5104b81183 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T11:39:19.068 INFO:journalctl@ceph.osd.4.smithi146.stdout:Apr 25 11:39:18 smithi146 podman[67989]: 2024-04-25 11:39:18.711457533 +0000 UTC m=+0.018370325 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T11:39:19.068 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:18 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:18.899+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:19.068 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:18 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:18.899+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:51.169170+0000 front 2024-04-25T11:38:51.169194+0000 (oldest deadline 2024-04-25T11:39:17.069398+0000) 2024-04-25T11:39:19.343 INFO:journalctl@ceph.osd.4.smithi146.stdout:Apr 25 11:39:19 smithi146 podman[67989]: 2024-04-25 11:39:19.081320388 +0000 UTC m=+0.388233162 container init 5289707eac4517fef70d5bca43e4b1494814b548609fc41e5bbcea5104b81183 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-4-deactivate, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-25T11:39:19.343 INFO:journalctl@ceph.osd.4.smithi146.stdout:Apr 25 11:39:19 smithi146 podman[67989]: 2024-04-25 11:39:19.08455361 +0000 UTC m=+0.391466388 container start 5289707eac4517fef70d5bca43e4b1494814b548609fc41e5bbcea5104b81183 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD) 2024-04-25T11:39:19.343 INFO:journalctl@ceph.osd.4.smithi146.stdout:Apr 25 11:39:19 smithi146 podman[67989]: 2024-04-25 11:39:19.156151634 +0000 UTC m=+0.463064410 container attach 5289707eac4517fef70d5bca43e4b1494814b548609fc41e5bbcea5104b81183 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T11:39:19.436 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 25 11:39:18 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-6[57622]: 2024-04-25T11:39:18.987+0000 7fda72835700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:48.953521+0000 front 2024-04-25T11:38:48.953667+0000 (oldest deadline 2024-04-25T11:39:13.653542+0000) 2024-04-25T11:39:19.437 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:19 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:19.123+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.662848+0000 front 2024-04-25T11:38:44.663184+0000 (oldest deadline 2024-04-25T11:39:09.363059+0000) 2024-04-25T11:39:19.645 INFO:journalctl@ceph.osd.4.smithi146.stdout:Apr 25 11:39:19 smithi146 podman[67989]: 2024-04-25 11:39:19.378247655 +0000 UTC m=+0.685160424 container died 5289707eac4517fef70d5bca43e4b1494814b548609fc41e5bbcea5104b81183 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T11:39:19.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:19 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:19.424+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:19.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:19 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:19.424+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:39:19.724 INFO:journalctl@ceph.osd.5.smithi152.stdout:Apr 25 11:39:19 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-5[44355]: 2024-04-25T11:39:19.641+0000 7f26f0e5f700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.146790+0000 front 2024-04-25T11:38:44.146932+0000 (oldest deadline 2024-04-25T11:39:08.846614+0000) 2024-04-25T11:39:19.724 INFO:journalctl@ceph.osd.5.smithi152.stdout:Apr 25 11:39:19 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-5[44355]: 2024-04-25T11:39:19.641+0000 7f26f0e5f700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:52.848698+0000 front 2024-04-25T11:38:52.848557+0000 (oldest deadline 2024-04-25T11:39:17.548551+0000) 2024-04-25T11:39:19.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:19 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:19.496+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:52.418405+0000 front 2024-04-25T11:38:52.418385+0000 (oldest deadline 2024-04-25T11:39:15.918284+0000) 2024-04-25T11:39:19.909 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:19 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:19.906+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:19.909 INFO:journalctl@ceph.osd.4.smithi146.stdout:Apr 25 11:39:19 smithi146 podman[68014]: 2024-04-25 11:39:19.901490251 +0000 UTC m=+0.517756402 container remove 5289707eac4517fef70d5bca43e4b1494814b548609fc41e5bbcea5104b81183 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux ) 2024-04-25T11:39:19.914 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@osd.4.service' 2024-04-25T11:39:19.951 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:39:19.952 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-25T11:39:19.952 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-25T11:39:19.952 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@osd.5 2024-04-25T11:39:20.186 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 25 11:39:19 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-6[57622]: 2024-04-25T11:39:19.947+0000 7fda72835700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:48.953521+0000 front 2024-04-25T11:38:48.953667+0000 (oldest deadline 2024-04-25T11:39:13.653542+0000) 2024-04-25T11:39:20.187 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:20 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:20.172+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.662848+0000 front 2024-04-25T11:38:44.663184+0000 (oldest deadline 2024-04-25T11:39:09.363059+0000) 2024-04-25T11:39:20.187 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:20 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:20.172+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:49.864482+0000 front 2024-04-25T11:38:49.864465+0000 (oldest deadline 2024-04-25T11:39:19.264749+0000) 2024-04-25T11:39:20.311 INFO:journalctl@ceph.osd.5.smithi152.stdout:Apr 25 11:39:20 smithi152 systemd[1]: Stopping Ceph osd.5 for f7ed73dc-02f4-11ef-bc93-c7b262605968... 2024-04-25T11:39:20.343 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:20 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:20.035+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.703177+0000 front 2024-04-25T11:38:50.703148+0000 (oldest deadline 2024-04-25T11:39:16.603339+0000) 2024-04-25T11:39:20.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:19 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:19.906+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:51.169170+0000 front 2024-04-25T11:38:51.169194+0000 (oldest deadline 2024-04-25T11:39:17.069398+0000) 2024-04-25T11:39:20.641 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:20 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:20.463+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:52.418405+0000 front 2024-04-25T11:38:52.418385+0000 (oldest deadline 2024-04-25T11:39:15.918284+0000) 2024-04-25T11:39:20.641 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:20 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:20.463+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:20.642 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:20 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:20.463+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:39:20.642 INFO:journalctl@ceph.osd.5.smithi152.stdout:Apr 25 11:39:20 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-5[44355]: 2024-04-25T11:39:20.323+0000 7f26f88b1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:39:20.642 INFO:journalctl@ceph.osd.5.smithi152.stdout:Apr 25 11:39:20 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-5[44355]: 2024-04-25T11:39:20.323+0000 7f26f88b1700 -1 osd.5 88 *** Got signal Terminated *** 2024-04-25T11:39:20.642 INFO:journalctl@ceph.osd.5.smithi152.stdout:Apr 25 11:39:20 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-5[44355]: 2024-04-25T11:39:20.323+0000 7f26f88b1700 -1 osd.5 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T11:39:20.974 INFO:journalctl@ceph.osd.5.smithi152.stdout:Apr 25 11:39:20 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-5[44355]: 2024-04-25T11:39:20.654+0000 7f26f0e5f700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.146790+0000 front 2024-04-25T11:38:44.146932+0000 (oldest deadline 2024-04-25T11:39:08.846614+0000) 2024-04-25T11:39:20.974 INFO:journalctl@ceph.osd.5.smithi152.stdout:Apr 25 11:39:20 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-5[44355]: 2024-04-25T11:39:20.654+0000 7f26f0e5f700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:52.848698+0000 front 2024-04-25T11:38:52.848557+0000 (oldest deadline 2024-04-25T11:39:17.548551+0000) 2024-04-25T11:39:21.137 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 25 11:39:20 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-6[57622]: 2024-04-25T11:39:20.954+0000 7fda72835700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:48.953521+0000 front 2024-04-25T11:38:48.953667+0000 (oldest deadline 2024-04-25T11:39:13.653542+0000) 2024-04-25T11:39:21.342 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:21 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:21.030+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.703177+0000 front 2024-04-25T11:38:50.703148+0000 (oldest deadline 2024-04-25T11:39:16.603339+0000) 2024-04-25T11:39:21.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:20 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:20.913+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:21.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:20 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:20.913+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:51.169170+0000 front 2024-04-25T11:38:51.169194+0000 (oldest deadline 2024-04-25T11:39:17.069398+0000) 2024-04-25T11:39:21.410 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:21 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:21.150+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.662848+0000 front 2024-04-25T11:38:44.663184+0000 (oldest deadline 2024-04-25T11:39:09.363059+0000) 2024-04-25T11:39:21.410 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:21 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:21.150+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:49.864482+0000 front 2024-04-25T11:38:49.864465+0000 (oldest deadline 2024-04-25T11:39:19.264749+0000) 2024-04-25T11:39:21.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:21 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:21.502+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:52.418405+0000 front 2024-04-25T11:38:52.418385+0000 (oldest deadline 2024-04-25T11:39:15.918284+0000) 2024-04-25T11:39:21.724 INFO:journalctl@ceph.osd.5.smithi152.stdout:Apr 25 11:39:21 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-5[44355]: 2024-04-25T11:39:21.647+0000 7f26f0e5f700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.146790+0000 front 2024-04-25T11:38:44.146932+0000 (oldest deadline 2024-04-25T11:39:08.846614+0000) 2024-04-25T11:39:21.724 INFO:journalctl@ceph.osd.5.smithi152.stdout:Apr 25 11:39:21 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-5[44355]: 2024-04-25T11:39:21.647+0000 7f26f0e5f700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:52.848698+0000 front 2024-04-25T11:38:52.848557+0000 (oldest deadline 2024-04-25T11:39:17.548551+0000) 2024-04-25T11:39:21.725 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:21 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:21.434+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:21.725 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:21 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:21.434+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:39:22.343 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:21 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:21.989+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.703177+0000 front 2024-04-25T11:38:50.703148+0000 (oldest deadline 2024-04-25T11:39:16.603339+0000) 2024-04-25T11:39:22.343 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:21 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:21.989+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:56.603886+0000 front 2024-04-25T11:38:56.603908+0000 (oldest deadline 2024-04-25T11:39:21.903841+0000) 2024-04-25T11:39:22.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:21 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:21.943+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:22.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:21 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:21.943+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:51.169170+0000 front 2024-04-25T11:38:51.169194+0000 (oldest deadline 2024-04-25T11:39:17.069398+0000) 2024-04-25T11:39:22.436 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 25 11:39:21 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-6[57622]: 2024-04-25T11:39:21.976+0000 7fda72835700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:48.953521+0000 front 2024-04-25T11:38:48.953667+0000 (oldest deadline 2024-04-25T11:39:13.653542+0000) 2024-04-25T11:39:22.436 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:22 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:22.121+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.662848+0000 front 2024-04-25T11:38:44.663184+0000 (oldest deadline 2024-04-25T11:39:09.363059+0000) 2024-04-25T11:39:22.436 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:22 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:22.121+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:49.864482+0000 front 2024-04-25T11:38:49.864465+0000 (oldest deadline 2024-04-25T11:39:19.264749+0000) 2024-04-25T11:39:22.437 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:22 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:22.121+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.765972+0000 front 2024-04-25T11:38:59.765844+0000 (oldest deadline 2024-04-25T11:39:22.065909+0000) 2024-04-25T11:39:22.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:22 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:22.505+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:52.418405+0000 front 2024-04-25T11:38:52.418385+0000 (oldest deadline 2024-04-25T11:39:15.918284+0000) 2024-04-25T11:39:22.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:22 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:22.505+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:55.918916+0000 front 2024-04-25T11:38:55.919016+0000 (oldest deadline 2024-04-25T11:39:21.819129+0000) 2024-04-25T11:39:22.724 INFO:journalctl@ceph.osd.5.smithi152.stdout:Apr 25 11:39:22 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-5[44355]: 2024-04-25T11:39:22.660+0000 7f26f0e5f700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.146790+0000 front 2024-04-25T11:38:44.146932+0000 (oldest deadline 2024-04-25T11:39:08.846614+0000) 2024-04-25T11:39:22.724 INFO:journalctl@ceph.osd.5.smithi152.stdout:Apr 25 11:39:22 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-5[44355]: 2024-04-25T11:39:22.660+0000 7f26f0e5f700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:52.848698+0000 front 2024-04-25T11:38:52.848557+0000 (oldest deadline 2024-04-25T11:39:17.548551+0000) 2024-04-25T11:39:22.725 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:22 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:22.417+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:22.725 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:22 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:22.417+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:39:23.186 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 25 11:39:22 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-6[57622]: 2024-04-25T11:39:22.935+0000 7fda72835700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:48.953521+0000 front 2024-04-25T11:38:48.953667+0000 (oldest deadline 2024-04-25T11:39:13.653542+0000) 2024-04-25T11:39:23.187 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:23 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:23.084+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.662848+0000 front 2024-04-25T11:38:44.663184+0000 (oldest deadline 2024-04-25T11:39:09.363059+0000) 2024-04-25T11:39:23.187 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:23 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:23.084+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:49.864482+0000 front 2024-04-25T11:38:49.864465+0000 (oldest deadline 2024-04-25T11:39:19.264749+0000) 2024-04-25T11:39:23.187 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:23 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:23.084+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.765972+0000 front 2024-04-25T11:38:59.765844+0000 (oldest deadline 2024-04-25T11:39:22.065909+0000) 2024-04-25T11:39:23.244 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:22 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:22.943+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.703177+0000 front 2024-04-25T11:38:50.703148+0000 (oldest deadline 2024-04-25T11:39:16.603339+0000) 2024-04-25T11:39:23.244 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:22 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:22.943+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:56.603886+0000 front 2024-04-25T11:38:56.603908+0000 (oldest deadline 2024-04-25T11:39:21.903841+0000) 2024-04-25T11:39:23.244 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:22 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:22.903+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:23.244 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:22 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:22.903+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:51.169170+0000 front 2024-04-25T11:38:51.169194+0000 (oldest deadline 2024-04-25T11:39:17.069398+0000) 2024-04-25T11:39:23.244 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:22 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:22.903+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.170746+0000 front 2024-04-25T11:39:01.170670+0000 (oldest deadline 2024-04-25T11:39:22.870611+0000) 2024-04-25T11:39:23.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:23 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:23.418+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:23.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:23 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:23.418+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:39:23.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:23 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:23.494+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:52.418405+0000 front 2024-04-25T11:38:52.418385+0000 (oldest deadline 2024-04-25T11:39:15.918284+0000) 2024-04-25T11:39:23.725 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:23 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:23.494+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:55.918916+0000 front 2024-04-25T11:38:55.919016+0000 (oldest deadline 2024-04-25T11:39:21.819129+0000) 2024-04-25T11:39:23.725 INFO:journalctl@ceph.osd.5.smithi152.stdout:Apr 25 11:39:23 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-5[44355]: 2024-04-25T11:39:23.665+0000 7f26f0e5f700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.146790+0000 front 2024-04-25T11:38:44.146932+0000 (oldest deadline 2024-04-25T11:39:08.846614+0000) 2024-04-25T11:39:23.725 INFO:journalctl@ceph.osd.5.smithi152.stdout:Apr 25 11:39:23 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-5[44355]: 2024-04-25T11:39:23.665+0000 7f26f0e5f700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:52.848698+0000 front 2024-04-25T11:38:52.848557+0000 (oldest deadline 2024-04-25T11:39:17.548551+0000) 2024-04-25T11:39:23.725 INFO:journalctl@ceph.osd.5.smithi152.stdout:Apr 25 11:39:23 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-5[44355]: 2024-04-25T11:39:23.665+0000 7f26f0e5f700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:57.549012+0000 front 2024-04-25T11:38:57.549161+0000 (oldest deadline 2024-04-25T11:39:23.349919+0000) 2024-04-25T11:39:24.187 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 25 11:39:23 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-6[57622]: 2024-04-25T11:39:23.926+0000 7fda72835700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:48.953521+0000 front 2024-04-25T11:38:48.953667+0000 (oldest deadline 2024-04-25T11:39:13.653542+0000) 2024-04-25T11:39:24.187 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:24 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:24.130+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.662848+0000 front 2024-04-25T11:38:44.663184+0000 (oldest deadline 2024-04-25T11:39:09.363059+0000) 2024-04-25T11:39:24.187 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:24 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:24.130+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:49.864482+0000 front 2024-04-25T11:38:49.864465+0000 (oldest deadline 2024-04-25T11:39:19.264749+0000) 2024-04-25T11:39:24.187 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:24 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:24.130+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.765972+0000 front 2024-04-25T11:38:59.765844+0000 (oldest deadline 2024-04-25T11:39:22.065909+0000) 2024-04-25T11:39:24.263 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:23 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:23.914+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.703177+0000 front 2024-04-25T11:38:50.703148+0000 (oldest deadline 2024-04-25T11:39:16.603339+0000) 2024-04-25T11:39:24.263 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:23 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:23.914+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:56.603886+0000 front 2024-04-25T11:38:56.603908+0000 (oldest deadline 2024-04-25T11:39:21.903841+0000) 2024-04-25T11:39:24.264 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:23 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:23.897+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:24.264 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:23 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:23.897+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:51.169170+0000 front 2024-04-25T11:38:51.169194+0000 (oldest deadline 2024-04-25T11:39:17.069398+0000) 2024-04-25T11:39:24.264 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:23 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:23.897+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.170746+0000 front 2024-04-25T11:39:01.170670+0000 (oldest deadline 2024-04-25T11:39:22.870611+0000) 2024-04-25T11:39:24.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:24 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:24.455+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:52.418405+0000 front 2024-04-25T11:38:52.418385+0000 (oldest deadline 2024-04-25T11:39:15.918284+0000) 2024-04-25T11:39:24.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:24 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:24.455+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:55.918916+0000 front 2024-04-25T11:38:55.919016+0000 (oldest deadline 2024-04-25T11:39:21.819129+0000) 2024-04-25T11:39:24.724 INFO:journalctl@ceph.osd.5.smithi152.stdout:Apr 25 11:39:24 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-5[44355]: 2024-04-25T11:39:24.637+0000 7f26f0e5f700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.146790+0000 front 2024-04-25T11:38:44.146932+0000 (oldest deadline 2024-04-25T11:39:08.846614+0000) 2024-04-25T11:39:24.724 INFO:journalctl@ceph.osd.5.smithi152.stdout:Apr 25 11:39:24 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-5[44355]: 2024-04-25T11:39:24.637+0000 7f26f0e5f700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:52.848698+0000 front 2024-04-25T11:38:52.848557+0000 (oldest deadline 2024-04-25T11:39:17.548551+0000) 2024-04-25T11:39:24.725 INFO:journalctl@ceph.osd.5.smithi152.stdout:Apr 25 11:39:24 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-5[44355]: 2024-04-25T11:39:24.637+0000 7f26f0e5f700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:57.549012+0000 front 2024-04-25T11:38:57.549161+0000 (oldest deadline 2024-04-25T11:39:23.349919+0000) 2024-04-25T11:39:24.725 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:24 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:24.404+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:24.725 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:24 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:24.404+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:39:24.725 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:24 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:24.404+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:39:25.186 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 25 11:39:24 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-6[57622]: 2024-04-25T11:39:24.929+0000 7fda72835700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:48.953521+0000 front 2024-04-25T11:38:48.953667+0000 (oldest deadline 2024-04-25T11:39:13.653542+0000) 2024-04-25T11:39:25.187 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:25 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:25.163+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.662848+0000 front 2024-04-25T11:38:44.663184+0000 (oldest deadline 2024-04-25T11:39:09.363059+0000) 2024-04-25T11:39:25.187 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:25 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:25.163+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:49.864482+0000 front 2024-04-25T11:38:49.864465+0000 (oldest deadline 2024-04-25T11:39:19.264749+0000) 2024-04-25T11:39:25.187 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:25 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:25.163+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.765972+0000 front 2024-04-25T11:38:59.765844+0000 (oldest deadline 2024-04-25T11:39:22.065909+0000) 2024-04-25T11:39:25.342 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:24 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:24.929+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.703177+0000 front 2024-04-25T11:38:50.703148+0000 (oldest deadline 2024-04-25T11:39:16.603339+0000) 2024-04-25T11:39:25.343 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:24 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:24.929+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:56.603886+0000 front 2024-04-25T11:38:56.603908+0000 (oldest deadline 2024-04-25T11:39:21.903841+0000) 2024-04-25T11:39:25.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:24 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:24.941+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:25.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:24 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:24.941+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:51.169170+0000 front 2024-04-25T11:38:51.169194+0000 (oldest deadline 2024-04-25T11:39:17.069398+0000) 2024-04-25T11:39:25.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:24 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:24.941+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.170746+0000 front 2024-04-25T11:39:01.170670+0000 (oldest deadline 2024-04-25T11:39:22.870611+0000) 2024-04-25T11:39:25.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:25 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:25.436+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:52.418405+0000 front 2024-04-25T11:38:52.418385+0000 (oldest deadline 2024-04-25T11:39:15.918284+0000) 2024-04-25T11:39:25.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:25 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:25.436+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:55.918916+0000 front 2024-04-25T11:38:55.919016+0000 (oldest deadline 2024-04-25T11:39:21.819129+0000) 2024-04-25T11:39:25.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:25 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:25.413+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:25.725 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:25 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:25.413+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:39:25.725 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:25 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:25.413+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:39:26.104 INFO:journalctl@ceph.osd.5.smithi152.stdout:Apr 25 11:39:25 smithi152 podman[69519]: 2024-04-25 11:39:25.813517022 +0000 UTC m=+5.622794742 container died 5dabd07b1f3605a81286e782ce1fce52a30d963c2326c018cfab81ec48499bd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-5, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-25T11:39:26.182 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 25 11:39:25 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-6[57622]: 2024-04-25T11:39:25.907+0000 7fda72835700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:48.953521+0000 front 2024-04-25T11:38:48.953667+0000 (oldest deadline 2024-04-25T11:39:13.653542+0000) 2024-04-25T11:39:26.182 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 25 11:39:25 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-6[57622]: 2024-04-25T11:39:25.907+0000 7fda72835700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.255033+0000 front 2024-04-25T11:39:01.254837+0000 (oldest deadline 2024-04-25T11:39:25.354816+0000) 2024-04-25T11:39:26.342 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:25 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:25.976+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.703177+0000 front 2024-04-25T11:38:50.703148+0000 (oldest deadline 2024-04-25T11:39:16.603339+0000) 2024-04-25T11:39:26.343 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:25 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:25.976+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:56.603886+0000 front 2024-04-25T11:38:56.603908+0000 (oldest deadline 2024-04-25T11:39:21.903841+0000) 2024-04-25T11:39:26.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:25 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:25.950+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:26.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:25 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:25.950+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:51.169170+0000 front 2024-04-25T11:38:51.169194+0000 (oldest deadline 2024-04-25T11:39:17.069398+0000) 2024-04-25T11:39:26.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:25 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:25.950+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.170746+0000 front 2024-04-25T11:39:01.170670+0000 (oldest deadline 2024-04-25T11:39:22.870611+0000) 2024-04-25T11:39:26.379 INFO:journalctl@ceph.osd.5.smithi152.stdout:Apr 25 11:39:26 smithi152 podman[69519]: 2024-04-25 11:39:26.216830055 +0000 UTC m=+6.026107777 container cleanup 5dabd07b1f3605a81286e782ce1fce52a30d963c2326c018cfab81ec48499bd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T11:39:26.379 INFO:journalctl@ceph.osd.5.smithi152.stdout:Apr 25 11:39:26 smithi152 bash[69519]: ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-5 2024-04-25T11:39:26.436 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:26 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:26.194+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.662848+0000 front 2024-04-25T11:38:44.663184+0000 (oldest deadline 2024-04-25T11:39:09.363059+0000) 2024-04-25T11:39:26.436 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:26 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:26.194+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:49.864482+0000 front 2024-04-25T11:38:49.864465+0000 (oldest deadline 2024-04-25T11:39:19.264749+0000) 2024-04-25T11:39:26.436 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:26 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:26.194+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.765972+0000 front 2024-04-25T11:38:59.765844+0000 (oldest deadline 2024-04-25T11:39:22.065909+0000) 2024-04-25T11:39:26.724 INFO:journalctl@ceph.osd.5.smithi152.stdout:Apr 25 11:39:26 smithi152 podman[69536]: 2024-04-25 11:39:26.426521061 +0000 UTC m=+0.604312731 container remove 5dabd07b1f3605a81286e782ce1fce52a30d963c2326c018cfab81ec48499bd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T11:39:26.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:26 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:26.402+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:52.418405+0000 front 2024-04-25T11:38:52.418385+0000 (oldest deadline 2024-04-25T11:39:15.918284+0000) 2024-04-25T11:39:26.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:26 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:26.402+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:55.918916+0000 front 2024-04-25T11:38:55.919016+0000 (oldest deadline 2024-04-25T11:39:21.819129+0000) 2024-04-25T11:39:26.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:26 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:26.392+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:26.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:26 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:26.392+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:39:26.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:26 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:26.392+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:39:27.100 INFO:journalctl@ceph.osd.5.smithi152.stdout:Apr 25 11:39:26 smithi152 podman[69626]: 2024-04-25 11:39:26.72883097 +0000 UTC m=+0.017763975 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T11:39:27.100 INFO:journalctl@ceph.osd.5.smithi152.stdout:Apr 25 11:39:26 smithi152 podman[69626]: 2024-04-25 11:39:26.852455105 +0000 UTC m=+0.141388099 container create 8bf335af40b518ebe49311bfdc1da43c2ef8f024c4ca55452962b10202f80d81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-5-deactivate, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-25T11:39:27.231 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 25 11:39:26 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-6[57622]: 2024-04-25T11:39:26.955+0000 7fda72835700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:48.953521+0000 front 2024-04-25T11:38:48.953667+0000 (oldest deadline 2024-04-25T11:39:13.653542+0000) 2024-04-25T11:39:27.231 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 25 11:39:26 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-6[57622]: 2024-04-25T11:39:26.955+0000 7fda72835700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.255033+0000 front 2024-04-25T11:39:01.254837+0000 (oldest deadline 2024-04-25T11:39:25.354816+0000) 2024-04-25T11:39:27.342 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:27 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:27.011+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.703177+0000 front 2024-04-25T11:38:50.703148+0000 (oldest deadline 2024-04-25T11:39:16.603339+0000) 2024-04-25T11:39:27.343 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:27 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:27.011+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:56.603886+0000 front 2024-04-25T11:38:56.603908+0000 (oldest deadline 2024-04-25T11:39:21.903841+0000) 2024-04-25T11:39:27.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:26 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:26.967+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:27.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:26 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:26.967+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:51.169170+0000 front 2024-04-25T11:38:51.169194+0000 (oldest deadline 2024-04-25T11:39:17.069398+0000) 2024-04-25T11:39:27.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:26 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:26.967+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.170746+0000 front 2024-04-25T11:39:01.170670+0000 (oldest deadline 2024-04-25T11:39:22.870611+0000) 2024-04-25T11:39:27.354 INFO:journalctl@ceph.osd.5.smithi152.stdout:Apr 25 11:39:27 smithi152 podman[69626]: 2024-04-25 11:39:27.113578352 +0000 UTC m=+0.402511361 container init 8bf335af40b518ebe49311bfdc1da43c2ef8f024c4ca55452962b10202f80d81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-25T11:39:27.354 INFO:journalctl@ceph.osd.5.smithi152.stdout:Apr 25 11:39:27 smithi152 podman[69626]: 2024-04-25 11:39:27.11692103 +0000 UTC m=+0.405854034 container start 8bf335af40b518ebe49311bfdc1da43c2ef8f024c4ca55452962b10202f80d81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-5-deactivate, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T11:39:27.355 INFO:journalctl@ceph.osd.5.smithi152.stdout:Apr 25 11:39:27 smithi152 podman[69626]: 2024-04-25 11:39:27.213347544 +0000 UTC m=+0.502280560 container attach 8bf335af40b518ebe49311bfdc1da43c2ef8f024c4ca55452962b10202f80d81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-25T11:39:27.355 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:27 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:27.367+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:52.418405+0000 front 2024-04-25T11:38:52.418385+0000 (oldest deadline 2024-04-25T11:39:15.918284+0000) 2024-04-25T11:39:27.355 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:27 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:27.367+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:55.918916+0000 front 2024-04-25T11:38:55.919016+0000 (oldest deadline 2024-04-25T11:39:21.819129+0000) 2024-04-25T11:39:27.355 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:27 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:27.362+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:27.355 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:27 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:27.362+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:39:27.355 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:27 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:27.362+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:39:27.686 INFO:journalctl@ceph.osd.5.smithi152.stdout:Apr 25 11:39:27 smithi152 podman[69626]: 2024-04-25 11:39:27.41314937 +0000 UTC m=+0.702082355 container died 8bf335af40b518ebe49311bfdc1da43c2ef8f024c4ca55452962b10202f80d81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-25T11:39:27.686 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:27 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:27.244+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.662848+0000 front 2024-04-25T11:38:44.663184+0000 (oldest deadline 2024-04-25T11:39:09.363059+0000) 2024-04-25T11:39:27.686 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:27 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:27.244+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:49.864482+0000 front 2024-04-25T11:38:49.864465+0000 (oldest deadline 2024-04-25T11:39:19.264749+0000) 2024-04-25T11:39:27.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:27 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:27.244+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.765972+0000 front 2024-04-25T11:38:59.765844+0000 (oldest deadline 2024-04-25T11:39:22.065909+0000) 2024-04-25T11:39:28.011 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@osd.5.service' 2024-04-25T11:39:28.047 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:39:28.048 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-25T11:39:28.048 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-25T11:39:28.048 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@osd.6 2024-04-25T11:39:28.187 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 25 11:39:27 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-6[57622]: 2024-04-25T11:39:27.925+0000 7fda72835700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:48.953521+0000 front 2024-04-25T11:38:48.953667+0000 (oldest deadline 2024-04-25T11:39:13.653542+0000) 2024-04-25T11:39:28.187 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 25 11:39:27 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-6[57622]: 2024-04-25T11:39:27.925+0000 7fda72835700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.255033+0000 front 2024-04-25T11:39:01.254837+0000 (oldest deadline 2024-04-25T11:39:25.354816+0000) 2024-04-25T11:39:28.187 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 25 11:39:28 smithi069 systemd[1]: Stopping Ceph osd.6 for f7ed73dc-02f4-11ef-bc93-c7b262605968... 2024-04-25T11:39:28.343 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:28 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:27.999+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.703177+0000 front 2024-04-25T11:38:50.703148+0000 (oldest deadline 2024-04-25T11:39:16.603339+0000) 2024-04-25T11:39:28.343 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:28 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:27.999+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:56.603886+0000 front 2024-04-25T11:38:56.603908+0000 (oldest deadline 2024-04-25T11:39:21.903841+0000) 2024-04-25T11:39:28.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:27 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:27.970+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:28.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:27 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:27.970+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:51.169170+0000 front 2024-04-25T11:38:51.169194+0000 (oldest deadline 2024-04-25T11:39:17.069398+0000) 2024-04-25T11:39:28.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:27 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:27.970+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.170746+0000 front 2024-04-25T11:39:01.170670+0000 (oldest deadline 2024-04-25T11:39:22.870611+0000) 2024-04-25T11:39:28.687 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 25 11:39:28 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-6[57622]: 2024-04-25T11:39:28.360+0000 7fda7a287700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:39:28.687 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 25 11:39:28 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-6[57622]: 2024-04-25T11:39:28.360+0000 7fda7a287700 -1 osd.6 88 *** Got signal Terminated *** 2024-04-25T11:39:28.687 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 25 11:39:28 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-6[57622]: 2024-04-25T11:39:28.360+0000 7fda7a287700 -1 osd.6 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T11:39:28.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:28 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:28.267+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.662848+0000 front 2024-04-25T11:38:44.663184+0000 (oldest deadline 2024-04-25T11:39:09.363059+0000) 2024-04-25T11:39:28.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:28 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:28.267+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:49.864482+0000 front 2024-04-25T11:38:49.864465+0000 (oldest deadline 2024-04-25T11:39:19.264749+0000) 2024-04-25T11:39:28.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:28 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:28.267+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.765972+0000 front 2024-04-25T11:38:59.765844+0000 (oldest deadline 2024-04-25T11:39:22.065909+0000) 2024-04-25T11:39:28.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:28 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:28.328+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:52.418405+0000 front 2024-04-25T11:38:52.418385+0000 (oldest deadline 2024-04-25T11:39:15.918284+0000) 2024-04-25T11:39:28.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:28 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:28.328+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:55.918916+0000 front 2024-04-25T11:38:55.919016+0000 (oldest deadline 2024-04-25T11:39:21.819129+0000) 2024-04-25T11:39:28.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:28 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:28.328+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:28.725 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:28 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:28.328+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:39:28.725 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:28 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:28.328+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:39:29.186 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 25 11:39:28 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-6[57622]: 2024-04-25T11:39:28.883+0000 7fda72835700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:48.953521+0000 front 2024-04-25T11:38:48.953667+0000 (oldest deadline 2024-04-25T11:39:13.653542+0000) 2024-04-25T11:39:29.248 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 25 11:39:28 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-6[57622]: 2024-04-25T11:39:28.883+0000 7fda72835700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.255033+0000 front 2024-04-25T11:39:01.254837+0000 (oldest deadline 2024-04-25T11:39:25.354816+0000) 2024-04-25T11:39:29.343 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:29 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:29.006+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.703177+0000 front 2024-04-25T11:38:50.703148+0000 (oldest deadline 2024-04-25T11:39:16.603339+0000) 2024-04-25T11:39:29.343 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:29 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:29.006+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:56.603886+0000 front 2024-04-25T11:38:56.603908+0000 (oldest deadline 2024-04-25T11:39:21.903841+0000) 2024-04-25T11:39:29.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:28 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:28.985+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:29.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:28 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:28.985+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:51.169170+0000 front 2024-04-25T11:38:51.169194+0000 (oldest deadline 2024-04-25T11:39:17.069398+0000) 2024-04-25T11:39:29.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:28 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:28.985+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.170746+0000 front 2024-04-25T11:39:01.170670+0000 (oldest deadline 2024-04-25T11:39:22.870611+0000) 2024-04-25T11:39:29.686 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:29 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:29.302+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.662848+0000 front 2024-04-25T11:38:44.663184+0000 (oldest deadline 2024-04-25T11:39:09.363059+0000) 2024-04-25T11:39:29.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:29 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:29.302+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:49.864482+0000 front 2024-04-25T11:38:49.864465+0000 (oldest deadline 2024-04-25T11:39:19.264749+0000) 2024-04-25T11:39:29.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:29 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:29.302+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.765972+0000 front 2024-04-25T11:38:59.765844+0000 (oldest deadline 2024-04-25T11:39:22.065909+0000) 2024-04-25T11:39:29.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:29 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:29.366+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:52.418405+0000 front 2024-04-25T11:38:52.418385+0000 (oldest deadline 2024-04-25T11:39:15.918284+0000) 2024-04-25T11:39:29.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:29 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:29.366+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:55.918916+0000 front 2024-04-25T11:38:55.919016+0000 (oldest deadline 2024-04-25T11:39:21.819129+0000) 2024-04-25T11:39:29.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:29 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:29.375+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:29.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:29 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:29.375+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:39:29.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:29 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:29.375+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:39:30.186 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 25 11:39:29 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-6[57622]: 2024-04-25T11:39:29.848+0000 7fda72835700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:48.953521+0000 front 2024-04-25T11:38:48.953667+0000 (oldest deadline 2024-04-25T11:39:13.653542+0000) 2024-04-25T11:39:30.187 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 25 11:39:29 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-6[57622]: 2024-04-25T11:39:29.848+0000 7fda72835700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.255033+0000 front 2024-04-25T11:39:01.254837+0000 (oldest deadline 2024-04-25T11:39:25.354816+0000) 2024-04-25T11:39:30.342 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:29 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:29.972+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.703177+0000 front 2024-04-25T11:38:50.703148+0000 (oldest deadline 2024-04-25T11:39:16.603339+0000) 2024-04-25T11:39:30.343 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:29 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:29.972+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:56.603886+0000 front 2024-04-25T11:38:56.603908+0000 (oldest deadline 2024-04-25T11:39:21.903841+0000) 2024-04-25T11:39:30.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:30 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:30.015+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:30.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:30 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:30.015+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:51.169170+0000 front 2024-04-25T11:38:51.169194+0000 (oldest deadline 2024-04-25T11:39:17.069398+0000) 2024-04-25T11:39:30.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:30 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:30.015+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.170746+0000 front 2024-04-25T11:39:01.170670+0000 (oldest deadline 2024-04-25T11:39:22.870611+0000) 2024-04-25T11:39:30.686 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:30 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:30.262+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.662848+0000 front 2024-04-25T11:38:44.663184+0000 (oldest deadline 2024-04-25T11:39:09.363059+0000) 2024-04-25T11:39:30.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:30 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:30.262+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:49.864482+0000 front 2024-04-25T11:38:49.864465+0000 (oldest deadline 2024-04-25T11:39:19.264749+0000) 2024-04-25T11:39:30.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:30 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:30.262+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.765972+0000 front 2024-04-25T11:38:59.765844+0000 (oldest deadline 2024-04-25T11:39:22.065909+0000) 2024-04-25T11:39:30.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:30 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:30.354+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:52.418405+0000 front 2024-04-25T11:38:52.418385+0000 (oldest deadline 2024-04-25T11:39:15.918284+0000) 2024-04-25T11:39:30.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:30 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:30.354+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:55.918916+0000 front 2024-04-25T11:38:55.919016+0000 (oldest deadline 2024-04-25T11:39:21.819129+0000) 2024-04-25T11:39:30.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:30 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:30.345+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:30.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:30 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:30.345+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:39:30.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:30 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:30.345+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:39:31.186 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 25 11:39:30 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-6[57622]: 2024-04-25T11:39:30.864+0000 7fda72835700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:48.953521+0000 front 2024-04-25T11:38:48.953667+0000 (oldest deadline 2024-04-25T11:39:13.653542+0000) 2024-04-25T11:39:31.187 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 25 11:39:30 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-6[57622]: 2024-04-25T11:39:30.865+0000 7fda72835700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.255033+0000 front 2024-04-25T11:39:01.254837+0000 (oldest deadline 2024-04-25T11:39:25.354816+0000) 2024-04-25T11:39:31.187 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 25 11:39:30 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-6[57622]: 2024-04-25T11:39:30.865+0000 7fda72835700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.056002+0000 front 2024-04-25T11:39:07.055955+0000 (oldest deadline 2024-04-25T11:39:30.556008+0000) 2024-04-25T11:39:31.342 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:30 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:30.956+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.703177+0000 front 2024-04-25T11:38:50.703148+0000 (oldest deadline 2024-04-25T11:39:16.603339+0000) 2024-04-25T11:39:31.343 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:30 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:30.956+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:56.603886+0000 front 2024-04-25T11:38:56.603908+0000 (oldest deadline 2024-04-25T11:39:21.903841+0000) 2024-04-25T11:39:31.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:30 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:30.986+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:31.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:30 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:30.986+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:51.169170+0000 front 2024-04-25T11:38:51.169194+0000 (oldest deadline 2024-04-25T11:39:17.069398+0000) 2024-04-25T11:39:31.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:30 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:30.986+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.170746+0000 front 2024-04-25T11:39:01.170670+0000 (oldest deadline 2024-04-25T11:39:22.870611+0000) 2024-04-25T11:39:31.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:30 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:30.986+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.472156+0000 front 2024-04-25T11:39:07.472172+0000 (oldest deadline 2024-04-25T11:39:30.972244+0000) 2024-04-25T11:39:31.686 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:31 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:31.278+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.662848+0000 front 2024-04-25T11:38:44.663184+0000 (oldest deadline 2024-04-25T11:39:09.363059+0000) 2024-04-25T11:39:31.686 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:31 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:31.278+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:49.864482+0000 front 2024-04-25T11:38:49.864465+0000 (oldest deadline 2024-04-25T11:39:19.264749+0000) 2024-04-25T11:39:31.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:31 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:31.278+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.765972+0000 front 2024-04-25T11:38:59.765844+0000 (oldest deadline 2024-04-25T11:39:22.065909+0000) 2024-04-25T11:39:31.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:31 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:31.363+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:52.418405+0000 front 2024-04-25T11:38:52.418385+0000 (oldest deadline 2024-04-25T11:39:15.918284+0000) 2024-04-25T11:39:31.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:31 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:31.363+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:55.918916+0000 front 2024-04-25T11:38:55.919016+0000 (oldest deadline 2024-04-25T11:39:21.819129+0000) 2024-04-25T11:39:31.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:31 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:31.363+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:05.920608+0000 front 2024-04-25T11:39:05.920695+0000 (oldest deadline 2024-04-25T11:39:31.220387+0000) 2024-04-25T11:39:31.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:31 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:31.372+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:31.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:31 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:31.372+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:39:31.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:31 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:31.372+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:39:32.186 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 25 11:39:31 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-6[57622]: 2024-04-25T11:39:31.857+0000 7fda72835700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:48.953521+0000 front 2024-04-25T11:38:48.953667+0000 (oldest deadline 2024-04-25T11:39:13.653542+0000) 2024-04-25T11:39:32.187 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 25 11:39:31 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-6[57622]: 2024-04-25T11:39:31.857+0000 7fda72835700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.255033+0000 front 2024-04-25T11:39:01.254837+0000 (oldest deadline 2024-04-25T11:39:25.354816+0000) 2024-04-25T11:39:32.187 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 25 11:39:31 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-6[57622]: 2024-04-25T11:39:31.857+0000 7fda72835700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.056002+0000 front 2024-04-25T11:39:07.055955+0000 (oldest deadline 2024-04-25T11:39:30.556008+0000) 2024-04-25T11:39:32.342 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:31 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:31.957+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.703177+0000 front 2024-04-25T11:38:50.703148+0000 (oldest deadline 2024-04-25T11:39:16.603339+0000) 2024-04-25T11:39:32.343 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:31 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:31.957+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:56.603886+0000 front 2024-04-25T11:38:56.603908+0000 (oldest deadline 2024-04-25T11:39:21.903841+0000) 2024-04-25T11:39:32.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:31 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:31.977+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:32.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:31 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:31.977+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:51.169170+0000 front 2024-04-25T11:38:51.169194+0000 (oldest deadline 2024-04-25T11:39:17.069398+0000) 2024-04-25T11:39:32.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:31 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:31.977+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.170746+0000 front 2024-04-25T11:39:01.170670+0000 (oldest deadline 2024-04-25T11:39:22.870611+0000) 2024-04-25T11:39:32.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:31 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:31.977+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.472156+0000 front 2024-04-25T11:39:07.472172+0000 (oldest deadline 2024-04-25T11:39:30.972244+0000) 2024-04-25T11:39:32.686 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:32 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:32.271+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.662848+0000 front 2024-04-25T11:38:44.663184+0000 (oldest deadline 2024-04-25T11:39:09.363059+0000) 2024-04-25T11:39:32.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:32 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:32.271+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:49.864482+0000 front 2024-04-25T11:38:49.864465+0000 (oldest deadline 2024-04-25T11:39:19.264749+0000) 2024-04-25T11:39:32.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:32 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:32.271+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.765972+0000 front 2024-04-25T11:38:59.765844+0000 (oldest deadline 2024-04-25T11:39:22.065909+0000) 2024-04-25T11:39:32.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:32 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:32.365+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:52.418405+0000 front 2024-04-25T11:38:52.418385+0000 (oldest deadline 2024-04-25T11:39:15.918284+0000) 2024-04-25T11:39:32.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:32 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:32.365+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:55.918916+0000 front 2024-04-25T11:38:55.919016+0000 (oldest deadline 2024-04-25T11:39:21.819129+0000) 2024-04-25T11:39:32.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:32 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:32.365+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:05.920608+0000 front 2024-04-25T11:39:05.920695+0000 (oldest deadline 2024-04-25T11:39:31.220387+0000) 2024-04-25T11:39:32.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:32 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:32.368+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:32.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:32 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:32.368+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:39:32.725 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:32 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:32.368+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:39:33.186 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 25 11:39:32 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-6[57622]: 2024-04-25T11:39:32.841+0000 7fda72835700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:48.953521+0000 front 2024-04-25T11:38:48.953667+0000 (oldest deadline 2024-04-25T11:39:13.653542+0000) 2024-04-25T11:39:33.187 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 25 11:39:32 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-6[57622]: 2024-04-25T11:39:32.841+0000 7fda72835700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.255033+0000 front 2024-04-25T11:39:01.254837+0000 (oldest deadline 2024-04-25T11:39:25.354816+0000) 2024-04-25T11:39:33.187 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 25 11:39:32 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-6[57622]: 2024-04-25T11:39:32.841+0000 7fda72835700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.056002+0000 front 2024-04-25T11:39:07.055955+0000 (oldest deadline 2024-04-25T11:39:30.556008+0000) 2024-04-25T11:39:33.342 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:32 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:32.998+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.703177+0000 front 2024-04-25T11:38:50.703148+0000 (oldest deadline 2024-04-25T11:39:16.603339+0000) 2024-04-25T11:39:33.343 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:32 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:32.998+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:56.603886+0000 front 2024-04-25T11:38:56.603908+0000 (oldest deadline 2024-04-25T11:39:21.903841+0000) 2024-04-25T11:39:33.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:32 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:32.978+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:33.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:32 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:32.978+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:51.169170+0000 front 2024-04-25T11:38:51.169194+0000 (oldest deadline 2024-04-25T11:39:17.069398+0000) 2024-04-25T11:39:33.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:32 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:32.978+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.170746+0000 front 2024-04-25T11:39:01.170670+0000 (oldest deadline 2024-04-25T11:39:22.870611+0000) 2024-04-25T11:39:33.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:32 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:32.978+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.472156+0000 front 2024-04-25T11:39:07.472172+0000 (oldest deadline 2024-04-25T11:39:30.972244+0000) 2024-04-25T11:39:33.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:33 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:33.271+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.662848+0000 front 2024-04-25T11:38:44.663184+0000 (oldest deadline 2024-04-25T11:39:09.363059+0000) 2024-04-25T11:39:33.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:33 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:33.271+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:49.864482+0000 front 2024-04-25T11:38:49.864465+0000 (oldest deadline 2024-04-25T11:39:19.264749+0000) 2024-04-25T11:39:33.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:33 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:33.271+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.765972+0000 front 2024-04-25T11:38:59.765844+0000 (oldest deadline 2024-04-25T11:39:22.065909+0000) 2024-04-25T11:39:33.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:33 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:33.271+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.867091+0000 front 2024-04-25T11:39:07.867122+0000 (oldest deadline 2024-04-25T11:39:33.167199+0000) 2024-04-25T11:39:33.723 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:33 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:33.391+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:52.418405+0000 front 2024-04-25T11:38:52.418385+0000 (oldest deadline 2024-04-25T11:39:15.918284+0000) 2024-04-25T11:39:33.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:33 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:33.391+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:55.918916+0000 front 2024-04-25T11:38:55.919016+0000 (oldest deadline 2024-04-25T11:39:21.819129+0000) 2024-04-25T11:39:33.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:33 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:33.391+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:05.920608+0000 front 2024-04-25T11:39:05.920695+0000 (oldest deadline 2024-04-25T11:39:31.220387+0000) 2024-04-25T11:39:33.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:33 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:33.383+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:33.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:33 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:33.383+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:39:33.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:33 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:33.383+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:39:33.982 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 25 11:39:33 smithi069 podman[88830]: 2024-04-25 11:39:33.740730205 +0000 UTC m=+5.479692831 container died a793b52289b3c73e947629796faa9f014b1b794a7556dae1822c50b48f50e1d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-6, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T11:39:34.263 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 25 11:39:34 smithi069 podman[88830]: 2024-04-25 11:39:34.086008032 +0000 UTC m=+5.824970656 container cleanup a793b52289b3c73e947629796faa9f014b1b794a7556dae1822c50b48f50e1d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-6, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-25T11:39:34.263 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 25 11:39:34 smithi069 bash[88830]: ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-6 2024-04-25T11:39:34.342 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:33 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:33.968+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.703177+0000 front 2024-04-25T11:38:50.703148+0000 (oldest deadline 2024-04-25T11:39:16.603339+0000) 2024-04-25T11:39:34.343 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:33 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:33.968+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:56.603886+0000 front 2024-04-25T11:38:56.603908+0000 (oldest deadline 2024-04-25T11:39:21.903841+0000) 2024-04-25T11:39:34.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:34 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:34.015+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:34.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:34 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:34.015+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:51.169170+0000 front 2024-04-25T11:38:51.169194+0000 (oldest deadline 2024-04-25T11:39:17.069398+0000) 2024-04-25T11:39:34.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:34 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:34.015+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.170746+0000 front 2024-04-25T11:39:01.170670+0000 (oldest deadline 2024-04-25T11:39:22.870611+0000) 2024-04-25T11:39:34.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:34 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:34.015+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.472156+0000 front 2024-04-25T11:39:07.472172+0000 (oldest deadline 2024-04-25T11:39:30.972244+0000) 2024-04-25T11:39:34.666 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 25 11:39:34 smithi069 podman[88847]: 2024-04-25 11:39:34.379114015 +0000 UTC m=+0.639611919 container remove a793b52289b3c73e947629796faa9f014b1b794a7556dae1822c50b48f50e1d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-6, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T11:39:34.666 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:34 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:34.275+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.662848+0000 front 2024-04-25T11:38:44.663184+0000 (oldest deadline 2024-04-25T11:39:09.363059+0000) 2024-04-25T11:39:34.667 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:34 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:34.275+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:49.864482+0000 front 2024-04-25T11:38:49.864465+0000 (oldest deadline 2024-04-25T11:39:19.264749+0000) 2024-04-25T11:39:34.667 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:34 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:34.275+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.765972+0000 front 2024-04-25T11:38:59.765844+0000 (oldest deadline 2024-04-25T11:39:22.065909+0000) 2024-04-25T11:39:34.667 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:34 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:34.275+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.867091+0000 front 2024-04-25T11:39:07.867122+0000 (oldest deadline 2024-04-25T11:39:33.167199+0000) 2024-04-25T11:39:34.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:34 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:34.378+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:52.418405+0000 front 2024-04-25T11:38:52.418385+0000 (oldest deadline 2024-04-25T11:39:15.918284+0000) 2024-04-25T11:39:34.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:34 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:34.378+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:55.918916+0000 front 2024-04-25T11:38:55.919016+0000 (oldest deadline 2024-04-25T11:39:21.819129+0000) 2024-04-25T11:39:34.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:34 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:34.378+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:05.920608+0000 front 2024-04-25T11:39:05.920695+0000 (oldest deadline 2024-04-25T11:39:31.220387+0000) 2024-04-25T11:39:34.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:34 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:34.360+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:34.725 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:34 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:34.360+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:39:34.725 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:34 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:34.360+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:39:34.936 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 25 11:39:34 smithi069 podman[88934]: 2024-04-25 11:39:34.679837224 +0000 UTC m=+0.113769310 container create abec946f992ec824ad9190e407ff9a7d287b973daadf3a801e20a28d0eef268d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T11:39:34.937 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 25 11:39:34 smithi069 podman[88934]: 2024-04-25 11:39:34.584469762 +0000 UTC m=+0.018401859 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T11:39:34.937 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 25 11:39:34 smithi069 podman[88934]: 2024-04-25 11:39:34.915453346 +0000 UTC m=+0.349385431 container init abec946f992ec824ad9190e407ff9a7d287b973daadf3a801e20a28d0eef268d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240424) 2024-04-25T11:39:34.937 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 25 11:39:34 smithi069 podman[88934]: 2024-04-25 11:39:34.918508204 +0000 UTC m=+0.352440291 container start abec946f992ec824ad9190e407ff9a7d287b973daadf3a801e20a28d0eef268d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240424, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-25T11:39:35.263 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 25 11:39:35 smithi069 podman[88934]: 2024-04-25 11:39:35.015713097 +0000 UTC m=+0.449645183 container attach abec946f992ec824ad9190e407ff9a7d287b973daadf3a801e20a28d0eef268d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424) 2024-04-25T11:39:35.264 INFO:journalctl@ceph.osd.6.smithi069.stdout:Apr 25 11:39:35 smithi069 podman[88934]: 2024-04-25 11:39:35.2100288 +0000 UTC m=+0.643960891 container died abec946f992ec824ad9190e407ff9a7d287b973daadf3a801e20a28d0eef268d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-25T11:39:35.342 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:34 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:34.946+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.703177+0000 front 2024-04-25T11:38:50.703148+0000 (oldest deadline 2024-04-25T11:39:16.603339+0000) 2024-04-25T11:39:35.343 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:34 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:34.946+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:56.603886+0000 front 2024-04-25T11:38:56.603908+0000 (oldest deadline 2024-04-25T11:39:21.903841+0000) 2024-04-25T11:39:35.343 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:34 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:34.946+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:08.906147+0000 front 2024-04-25T11:39:08.905960+0000 (oldest deadline 2024-04-25T11:39:34.205905+0000) 2024-04-25T11:39:35.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:35 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:35.052+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:35.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:35 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:35.052+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:51.169170+0000 front 2024-04-25T11:38:51.169194+0000 (oldest deadline 2024-04-25T11:39:17.069398+0000) 2024-04-25T11:39:35.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:35 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:35.052+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.170746+0000 front 2024-04-25T11:39:01.170670+0000 (oldest deadline 2024-04-25T11:39:22.870611+0000) 2024-04-25T11:39:35.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:35 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:35.052+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.472156+0000 front 2024-04-25T11:39:07.472172+0000 (oldest deadline 2024-04-25T11:39:30.972244+0000) 2024-04-25T11:39:35.538 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:35 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:35.276+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.662848+0000 front 2024-04-25T11:38:44.663184+0000 (oldest deadline 2024-04-25T11:39:09.363059+0000) 2024-04-25T11:39:35.538 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:35 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:35.276+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:49.864482+0000 front 2024-04-25T11:38:49.864465+0000 (oldest deadline 2024-04-25T11:39:19.264749+0000) 2024-04-25T11:39:35.539 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:35 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:35.276+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.765972+0000 front 2024-04-25T11:38:59.765844+0000 (oldest deadline 2024-04-25T11:39:22.065909+0000) 2024-04-25T11:39:35.539 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:35 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:35.276+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.867091+0000 front 2024-04-25T11:39:07.867122+0000 (oldest deadline 2024-04-25T11:39:33.167199+0000) 2024-04-25T11:39:35.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:35 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:35.425+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:52.418405+0000 front 2024-04-25T11:38:52.418385+0000 (oldest deadline 2024-04-25T11:39:15.918284+0000) 2024-04-25T11:39:35.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:35 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:35.425+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:55.918916+0000 front 2024-04-25T11:38:55.919016+0000 (oldest deadline 2024-04-25T11:39:21.819129+0000) 2024-04-25T11:39:35.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:35 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:35.425+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:05.920608+0000 front 2024-04-25T11:39:05.920695+0000 (oldest deadline 2024-04-25T11:39:31.220387+0000) 2024-04-25T11:39:35.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:35 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:35.319+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:35.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:35 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:35.319+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:39:35.725 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:35 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:35.319+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:39:35.849 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@osd.6.service' 2024-04-25T11:39:35.887 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:39:35.887 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-25T11:39:35.887 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-25T11:39:35.887 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@osd.7 2024-04-25T11:39:36.215 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:36 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:36.073+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:36.216 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:36 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:36.073+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:51.169170+0000 front 2024-04-25T11:38:51.169194+0000 (oldest deadline 2024-04-25T11:39:17.069398+0000) 2024-04-25T11:39:36.216 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:36 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:36.073+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.170746+0000 front 2024-04-25T11:39:01.170670+0000 (oldest deadline 2024-04-25T11:39:22.870611+0000) 2024-04-25T11:39:36.216 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:36 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:36.073+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.472156+0000 front 2024-04-25T11:39:07.472172+0000 (oldest deadline 2024-04-25T11:39:30.972244+0000) 2024-04-25T11:39:36.216 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:35 smithi146 systemd[1]: Stopping Ceph osd.7 for f7ed73dc-02f4-11ef-bc93-c7b262605968... 2024-04-25T11:39:36.216 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:35 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:35.983+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.703177+0000 front 2024-04-25T11:38:50.703148+0000 (oldest deadline 2024-04-25T11:39:16.603339+0000) 2024-04-25T11:39:36.216 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:35 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:35.983+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:56.603886+0000 front 2024-04-25T11:38:56.603908+0000 (oldest deadline 2024-04-25T11:39:21.903841+0000) 2024-04-25T11:39:36.216 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:35 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:35.983+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:08.906147+0000 front 2024-04-25T11:39:08.905960+0000 (oldest deadline 2024-04-25T11:39:34.205905+0000) 2024-04-25T11:39:36.216 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:36 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:36.228+0000 7f8fdfb98700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:39:36.592 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:36 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:36.228+0000 7f8fdfb98700 -1 osd.7 88 *** Got signal Terminated *** 2024-04-25T11:39:36.593 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:36 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:36.228+0000 7f8fdfb98700 -1 osd.7 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T11:39:36.686 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:36 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:36.305+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.662848+0000 front 2024-04-25T11:38:44.663184+0000 (oldest deadline 2024-04-25T11:39:09.363059+0000) 2024-04-25T11:39:36.686 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:36 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:36.305+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:49.864482+0000 front 2024-04-25T11:38:49.864465+0000 (oldest deadline 2024-04-25T11:39:19.264749+0000) 2024-04-25T11:39:36.686 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:36 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:36.305+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.765972+0000 front 2024-04-25T11:38:59.765844+0000 (oldest deadline 2024-04-25T11:39:22.065909+0000) 2024-04-25T11:39:36.686 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:36 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:36.305+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.867091+0000 front 2024-04-25T11:39:07.867122+0000 (oldest deadline 2024-04-25T11:39:33.167199+0000) 2024-04-25T11:39:36.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:36 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:36.445+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:52.418405+0000 front 2024-04-25T11:38:52.418385+0000 (oldest deadline 2024-04-25T11:39:15.918284+0000) 2024-04-25T11:39:36.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:36 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:36.445+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:55.918916+0000 front 2024-04-25T11:38:55.919016+0000 (oldest deadline 2024-04-25T11:39:21.819129+0000) 2024-04-25T11:39:36.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:36 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:36.445+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:05.920608+0000 front 2024-04-25T11:39:05.920695+0000 (oldest deadline 2024-04-25T11:39:31.220387+0000) 2024-04-25T11:39:36.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:36 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:36.283+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:36.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:36 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:36.283+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:39:36.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:36 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:36.283+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:39:37.342 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:36 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:36.942+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.703177+0000 front 2024-04-25T11:38:50.703148+0000 (oldest deadline 2024-04-25T11:39:16.603339+0000) 2024-04-25T11:39:37.343 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:36 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:36.942+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:56.603886+0000 front 2024-04-25T11:38:56.603908+0000 (oldest deadline 2024-04-25T11:39:21.903841+0000) 2024-04-25T11:39:37.343 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:36 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:36.942+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:08.906147+0000 front 2024-04-25T11:39:08.905960+0000 (oldest deadline 2024-04-25T11:39:34.205905+0000) 2024-04-25T11:39:37.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:37 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:37.030+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:37.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:37 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:37.030+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:51.169170+0000 front 2024-04-25T11:38:51.169194+0000 (oldest deadline 2024-04-25T11:39:17.069398+0000) 2024-04-25T11:39:37.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:37 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:37.030+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.170746+0000 front 2024-04-25T11:39:01.170670+0000 (oldest deadline 2024-04-25T11:39:22.870611+0000) 2024-04-25T11:39:37.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:37 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:37.030+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.472156+0000 front 2024-04-25T11:39:07.472172+0000 (oldest deadline 2024-04-25T11:39:30.972244+0000) 2024-04-25T11:39:37.686 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:37 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:37.332+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.662848+0000 front 2024-04-25T11:38:44.663184+0000 (oldest deadline 2024-04-25T11:39:09.363059+0000) 2024-04-25T11:39:37.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:37 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:37.332+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:49.864482+0000 front 2024-04-25T11:38:49.864465+0000 (oldest deadline 2024-04-25T11:39:19.264749+0000) 2024-04-25T11:39:37.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:37 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:37.332+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.765972+0000 front 2024-04-25T11:38:59.765844+0000 (oldest deadline 2024-04-25T11:39:22.065909+0000) 2024-04-25T11:39:37.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:37 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:37.332+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.867091+0000 front 2024-04-25T11:39:07.867122+0000 (oldest deadline 2024-04-25T11:39:33.167199+0000) 2024-04-25T11:39:37.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:37 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:37.427+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:52.418405+0000 front 2024-04-25T11:38:52.418385+0000 (oldest deadline 2024-04-25T11:39:15.918284+0000) 2024-04-25T11:39:37.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:37 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:37.427+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:55.918916+0000 front 2024-04-25T11:38:55.919016+0000 (oldest deadline 2024-04-25T11:39:21.819129+0000) 2024-04-25T11:39:37.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:37 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:37.427+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:05.920608+0000 front 2024-04-25T11:39:05.920695+0000 (oldest deadline 2024-04-25T11:39:31.220387+0000) 2024-04-25T11:39:37.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:37 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:37.275+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:37.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:37 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:37.275+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:39:37.725 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:37 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:37.275+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:39:38.343 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:37 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:37.934+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.703177+0000 front 2024-04-25T11:38:50.703148+0000 (oldest deadline 2024-04-25T11:39:16.603339+0000) 2024-04-25T11:39:38.343 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:37 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:37.934+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:56.603886+0000 front 2024-04-25T11:38:56.603908+0000 (oldest deadline 2024-04-25T11:39:21.903841+0000) 2024-04-25T11:39:38.343 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:37 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:37.934+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:08.906147+0000 front 2024-04-25T11:39:08.905960+0000 (oldest deadline 2024-04-25T11:39:34.205905+0000) 2024-04-25T11:39:38.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:38 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:38.013+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:38.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:38 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:38.013+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:51.169170+0000 front 2024-04-25T11:38:51.169194+0000 (oldest deadline 2024-04-25T11:39:17.069398+0000) 2024-04-25T11:39:38.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:38 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:38.013+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.170746+0000 front 2024-04-25T11:39:01.170670+0000 (oldest deadline 2024-04-25T11:39:22.870611+0000) 2024-04-25T11:39:38.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:38 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:38.013+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.472156+0000 front 2024-04-25T11:39:07.472172+0000 (oldest deadline 2024-04-25T11:39:30.972244+0000) 2024-04-25T11:39:38.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:38 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:38.322+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.662848+0000 front 2024-04-25T11:38:44.663184+0000 (oldest deadline 2024-04-25T11:39:09.363059+0000) 2024-04-25T11:39:38.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:38 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:38.322+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:49.864482+0000 front 2024-04-25T11:38:49.864465+0000 (oldest deadline 2024-04-25T11:39:19.264749+0000) 2024-04-25T11:39:38.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:38 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:38.322+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.765972+0000 front 2024-04-25T11:38:59.765844+0000 (oldest deadline 2024-04-25T11:39:22.065909+0000) 2024-04-25T11:39:38.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:38 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:38.322+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.867091+0000 front 2024-04-25T11:39:07.867122+0000 (oldest deadline 2024-04-25T11:39:33.167199+0000) 2024-04-25T11:39:38.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:38 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:38.466+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:52.418405+0000 front 2024-04-25T11:38:52.418385+0000 (oldest deadline 2024-04-25T11:39:15.918284+0000) 2024-04-25T11:39:38.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:38 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:38.466+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:55.918916+0000 front 2024-04-25T11:38:55.919016+0000 (oldest deadline 2024-04-25T11:39:21.819129+0000) 2024-04-25T11:39:38.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:38 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:38.466+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:05.920608+0000 front 2024-04-25T11:39:05.920695+0000 (oldest deadline 2024-04-25T11:39:31.220387+0000) 2024-04-25T11:39:38.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:38 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:38.322+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:38.725 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:38 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:38.322+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:39:38.725 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:38 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:38.322+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:39:39.343 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:38 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:38.899+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.703177+0000 front 2024-04-25T11:38:50.703148+0000 (oldest deadline 2024-04-25T11:39:16.603339+0000) 2024-04-25T11:39:39.343 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:38 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:38.899+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:56.603886+0000 front 2024-04-25T11:38:56.603908+0000 (oldest deadline 2024-04-25T11:39:21.903841+0000) 2024-04-25T11:39:39.343 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:38 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:38.899+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:08.906147+0000 front 2024-04-25T11:39:08.905960+0000 (oldest deadline 2024-04-25T11:39:34.205905+0000) 2024-04-25T11:39:39.343 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:38 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:38.899+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:15.306942+0000 front 2024-04-25T11:39:15.307020+0000 (oldest deadline 2024-04-25T11:39:38.207105+0000) 2024-04-25T11:39:39.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:39 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:39.008+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:39.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:39 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:39.008+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:51.169170+0000 front 2024-04-25T11:38:51.169194+0000 (oldest deadline 2024-04-25T11:39:17.069398+0000) 2024-04-25T11:39:39.344 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:39 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:39.008+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.170746+0000 front 2024-04-25T11:39:01.170670+0000 (oldest deadline 2024-04-25T11:39:22.870611+0000) 2024-04-25T11:39:39.344 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:39 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:39.008+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.472156+0000 front 2024-04-25T11:39:07.472172+0000 (oldest deadline 2024-04-25T11:39:30.972244+0000) 2024-04-25T11:39:39.588 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:39 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:39.292+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.662848+0000 front 2024-04-25T11:38:44.663184+0000 (oldest deadline 2024-04-25T11:39:09.363059+0000) 2024-04-25T11:39:39.588 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:39 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:39.292+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:49.864482+0000 front 2024-04-25T11:38:49.864465+0000 (oldest deadline 2024-04-25T11:39:19.264749+0000) 2024-04-25T11:39:39.588 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:39 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:39.292+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.765972+0000 front 2024-04-25T11:38:59.765844+0000 (oldest deadline 2024-04-25T11:39:22.065909+0000) 2024-04-25T11:39:39.588 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:39 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:39.292+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.867091+0000 front 2024-04-25T11:39:07.867122+0000 (oldest deadline 2024-04-25T11:39:33.167199+0000) 2024-04-25T11:39:39.588 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:39 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:39.292+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:14.268262+0000 front 2024-04-25T11:39:14.268340+0000 (oldest deadline 2024-04-25T11:39:38.968155+0000) 2024-04-25T11:39:39.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:39 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:39.501+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:52.418405+0000 front 2024-04-25T11:38:52.418385+0000 (oldest deadline 2024-04-25T11:39:15.918284+0000) 2024-04-25T11:39:39.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:39 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:39.501+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:55.918916+0000 front 2024-04-25T11:38:55.919016+0000 (oldest deadline 2024-04-25T11:39:21.819129+0000) 2024-04-25T11:39:39.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:39 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:39.501+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:05.920608+0000 front 2024-04-25T11:39:05.920695+0000 (oldest deadline 2024-04-25T11:39:31.220387+0000) 2024-04-25T11:39:39.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:39 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:39.354+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:39.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:39 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:39.354+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:39:39.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:39 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:39.354+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:39:40.342 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:39 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:39.933+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.703177+0000 front 2024-04-25T11:38:50.703148+0000 (oldest deadline 2024-04-25T11:39:16.603339+0000) 2024-04-25T11:39:40.342 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:39 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:39.933+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:56.603886+0000 front 2024-04-25T11:38:56.603908+0000 (oldest deadline 2024-04-25T11:39:21.903841+0000) 2024-04-25T11:39:40.343 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:39 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:39.933+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:08.906147+0000 front 2024-04-25T11:39:08.905960+0000 (oldest deadline 2024-04-25T11:39:34.205905+0000) 2024-04-25T11:39:40.343 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:39 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:39.933+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:15.306942+0000 front 2024-04-25T11:39:15.307020+0000 (oldest deadline 2024-04-25T11:39:38.207105+0000) 2024-04-25T11:39:40.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:39 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:39.963+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:40.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:39 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:39.963+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:51.169170+0000 front 2024-04-25T11:38:51.169194+0000 (oldest deadline 2024-04-25T11:39:17.069398+0000) 2024-04-25T11:39:40.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:39 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:39.963+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.170746+0000 front 2024-04-25T11:39:01.170670+0000 (oldest deadline 2024-04-25T11:39:22.870611+0000) 2024-04-25T11:39:40.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:39 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:39.963+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.472156+0000 front 2024-04-25T11:39:07.472172+0000 (oldest deadline 2024-04-25T11:39:30.972244+0000) 2024-04-25T11:39:40.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:39 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:39.963+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:15.673283+0000 front 2024-04-25T11:39:15.673348+0000 (oldest deadline 2024-04-25T11:39:39.173434+0000) 2024-04-25T11:39:40.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:40 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:40.286+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.662848+0000 front 2024-04-25T11:38:44.663184+0000 (oldest deadline 2024-04-25T11:39:09.363059+0000) 2024-04-25T11:39:40.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:40 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:40.286+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:49.864482+0000 front 2024-04-25T11:38:49.864465+0000 (oldest deadline 2024-04-25T11:39:19.264749+0000) 2024-04-25T11:39:40.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:40 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:40.286+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.765972+0000 front 2024-04-25T11:38:59.765844+0000 (oldest deadline 2024-04-25T11:39:22.065909+0000) 2024-04-25T11:39:40.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:40 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:40.286+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.867091+0000 front 2024-04-25T11:39:07.867122+0000 (oldest deadline 2024-04-25T11:39:33.167199+0000) 2024-04-25T11:39:40.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:40 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:40.286+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:14.268262+0000 front 2024-04-25T11:39:14.268340+0000 (oldest deadline 2024-04-25T11:39:38.968155+0000) 2024-04-25T11:39:40.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:40 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:40.506+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:52.418405+0000 front 2024-04-25T11:38:52.418385+0000 (oldest deadline 2024-04-25T11:39:15.918284+0000) 2024-04-25T11:39:40.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:40 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:40.506+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:55.918916+0000 front 2024-04-25T11:38:55.919016+0000 (oldest deadline 2024-04-25T11:39:21.819129+0000) 2024-04-25T11:39:40.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:40 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:40.506+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:05.920608+0000 front 2024-04-25T11:39:05.920695+0000 (oldest deadline 2024-04-25T11:39:31.220387+0000) 2024-04-25T11:39:40.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:40 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:40.506+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:15.321402+0000 front 2024-04-25T11:39:15.321429+0000 (oldest deadline 2024-04-25T11:39:40.021407+0000) 2024-04-25T11:39:40.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:40 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:40.308+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:40.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:40 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:40.308+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:39:40.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:40 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:40.308+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:39:40.725 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:40 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:40.308+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:17.188600+0000 front 2024-04-25T11:39:17.188531+0000 (oldest deadline 2024-04-25T11:39:40.088473+0000) 2024-04-25T11:39:41.343 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:40 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:40.941+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.703177+0000 front 2024-04-25T11:38:50.703148+0000 (oldest deadline 2024-04-25T11:39:16.603339+0000) 2024-04-25T11:39:41.343 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:40 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:40.941+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:56.603886+0000 front 2024-04-25T11:38:56.603908+0000 (oldest deadline 2024-04-25T11:39:21.903841+0000) 2024-04-25T11:39:41.343 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:40 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:40.941+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:08.906147+0000 front 2024-04-25T11:39:08.905960+0000 (oldest deadline 2024-04-25T11:39:34.205905+0000) 2024-04-25T11:39:41.343 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:40 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7[50912]: 2024-04-25T11:39:40.941+0000 7f8fdc14e700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:15.306942+0000 front 2024-04-25T11:39:15.307020+0000 (oldest deadline 2024-04-25T11:39:38.207105+0000) 2024-04-25T11:39:41.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:40 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:40.921+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:41.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:40 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:40.921+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:51.169170+0000 front 2024-04-25T11:38:51.169194+0000 (oldest deadline 2024-04-25T11:39:17.069398+0000) 2024-04-25T11:39:41.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:40 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:40.921+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.170746+0000 front 2024-04-25T11:39:01.170670+0000 (oldest deadline 2024-04-25T11:39:22.870611+0000) 2024-04-25T11:39:41.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:40 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:40.921+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.472156+0000 front 2024-04-25T11:39:07.472172+0000 (oldest deadline 2024-04-25T11:39:30.972244+0000) 2024-04-25T11:39:41.344 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:40 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:40.921+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:15.673283+0000 front 2024-04-25T11:39:15.673348+0000 (oldest deadline 2024-04-25T11:39:39.173434+0000) 2024-04-25T11:39:41.686 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:41 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:41.316+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.662848+0000 front 2024-04-25T11:38:44.663184+0000 (oldest deadline 2024-04-25T11:39:09.363059+0000) 2024-04-25T11:39:41.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:41 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:41.316+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:49.864482+0000 front 2024-04-25T11:38:49.864465+0000 (oldest deadline 2024-04-25T11:39:19.264749+0000) 2024-04-25T11:39:41.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:41 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:41.316+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.765972+0000 front 2024-04-25T11:38:59.765844+0000 (oldest deadline 2024-04-25T11:39:22.065909+0000) 2024-04-25T11:39:41.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:41 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:41.316+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.867091+0000 front 2024-04-25T11:39:07.867122+0000 (oldest deadline 2024-04-25T11:39:33.167199+0000) 2024-04-25T11:39:41.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:41 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:41.316+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:14.268262+0000 front 2024-04-25T11:39:14.268340+0000 (oldest deadline 2024-04-25T11:39:38.968155+0000) 2024-04-25T11:39:41.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:41 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:41.538+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:52.418405+0000 front 2024-04-25T11:38:52.418385+0000 (oldest deadline 2024-04-25T11:39:15.918284+0000) 2024-04-25T11:39:41.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:41 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:41.538+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:55.918916+0000 front 2024-04-25T11:38:55.919016+0000 (oldest deadline 2024-04-25T11:39:21.819129+0000) 2024-04-25T11:39:41.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:41 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:41.538+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:05.920608+0000 front 2024-04-25T11:39:05.920695+0000 (oldest deadline 2024-04-25T11:39:31.220387+0000) 2024-04-25T11:39:41.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:41 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:41.538+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:15.321402+0000 front 2024-04-25T11:39:15.321429+0000 (oldest deadline 2024-04-25T11:39:40.021407+0000) 2024-04-25T11:39:41.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:41 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:41.315+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:41.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:41 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:41.315+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:39:41.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:41 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:41.315+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:39:41.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:41 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:41.315+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:17.188600+0000 front 2024-04-25T11:39:17.188531+0000 (oldest deadline 2024-04-25T11:39:40.088473+0000) 2024-04-25T11:39:41.916 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:41 smithi146 podman[68105]: 2024-04-25 11:39:41.645796555 +0000 UTC m=+5.548975778 container died 40ee3dc36199ff8c7b495dc95c0270b5c2bb0f0d76c9b869b152442d71d10213 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.build-date=20240424) 2024-04-25T11:39:42.191 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:42 smithi146 podman[68105]: 2024-04-25 11:39:42.020283861 +0000 UTC m=+5.923463085 container cleanup 40ee3dc36199ff8c7b495dc95c0270b5c2bb0f0d76c9b869b152442d71d10213 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T11:39:42.191 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:42 smithi146 bash[68105]: ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7 2024-04-25T11:39:42.191 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:41 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:41.941+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:42.191 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:41 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:41.941+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:51.169170+0000 front 2024-04-25T11:38:51.169194+0000 (oldest deadline 2024-04-25T11:39:17.069398+0000) 2024-04-25T11:39:42.191 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:41 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:41.941+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.170746+0000 front 2024-04-25T11:39:01.170670+0000 (oldest deadline 2024-04-25T11:39:22.870611+0000) 2024-04-25T11:39:42.191 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:41 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:41.941+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.472156+0000 front 2024-04-25T11:39:07.472172+0000 (oldest deadline 2024-04-25T11:39:30.972244+0000) 2024-04-25T11:39:42.192 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:41 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:41.941+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:15.673283+0000 front 2024-04-25T11:39:15.673348+0000 (oldest deadline 2024-04-25T11:39:39.173434+0000) 2024-04-25T11:39:42.592 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:42 smithi146 podman[68121]: 2024-04-25 11:39:42.204702553 +0000 UTC m=+0.553110672 container remove 40ee3dc36199ff8c7b495dc95c0270b5c2bb0f0d76c9b869b152442d71d10213 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-25T11:39:42.686 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:42 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:42.357+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.662848+0000 front 2024-04-25T11:38:44.663184+0000 (oldest deadline 2024-04-25T11:39:09.363059+0000) 2024-04-25T11:39:42.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:42 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:42.357+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:49.864482+0000 front 2024-04-25T11:38:49.864465+0000 (oldest deadline 2024-04-25T11:39:19.264749+0000) 2024-04-25T11:39:42.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:42 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:42.357+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.765972+0000 front 2024-04-25T11:38:59.765844+0000 (oldest deadline 2024-04-25T11:39:22.065909+0000) 2024-04-25T11:39:42.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:42 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:42.357+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.867091+0000 front 2024-04-25T11:39:07.867122+0000 (oldest deadline 2024-04-25T11:39:33.167199+0000) 2024-04-25T11:39:42.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:42 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:42.357+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:14.268262+0000 front 2024-04-25T11:39:14.268340+0000 (oldest deadline 2024-04-25T11:39:38.968155+0000) 2024-04-25T11:39:42.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:42 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:42.544+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:52.418405+0000 front 2024-04-25T11:38:52.418385+0000 (oldest deadline 2024-04-25T11:39:15.918284+0000) 2024-04-25T11:39:42.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:42 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:42.544+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:55.918916+0000 front 2024-04-25T11:38:55.919016+0000 (oldest deadline 2024-04-25T11:39:21.819129+0000) 2024-04-25T11:39:42.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:42 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:42.544+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:05.920608+0000 front 2024-04-25T11:39:05.920695+0000 (oldest deadline 2024-04-25T11:39:31.220387+0000) 2024-04-25T11:39:42.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:42 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:42.545+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:15.321402+0000 front 2024-04-25T11:39:15.321429+0000 (oldest deadline 2024-04-25T11:39:40.021407+0000) 2024-04-25T11:39:42.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:42 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:42.358+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:42.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:42 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:42.358+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:39:42.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:42 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:42.358+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:39:42.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:42 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:42.358+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:17.188600+0000 front 2024-04-25T11:39:17.188531+0000 (oldest deadline 2024-04-25T11:39:40.088473+0000) 2024-04-25T11:39:42.919 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:42 smithi146 podman[68214]: 2024-04-25 11:39:42.542464776 +0000 UTC m=+0.018496731 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T11:39:42.919 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:42 smithi146 podman[68214]: 2024-04-25 11:39:42.647244553 +0000 UTC m=+0.123276550 container create 3a9d263f0fabbaf37139b56506b2057c723991f146fef8cd9a446939e377664e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2) 2024-04-25T11:39:42.919 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:42 smithi146 podman[68214]: 2024-04-25 11:39:42.932873339 +0000 UTC m=+0.408905294 container init 3a9d263f0fabbaf37139b56506b2057c723991f146fef8cd9a446939e377664e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T11:39:43.216 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:42 smithi146 podman[68214]: 2024-04-25 11:39:42.936319559 +0000 UTC m=+0.412351519 container start 3a9d263f0fabbaf37139b56506b2057c723991f146fef8cd9a446939e377664e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T11:39:43.216 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:42 smithi146 podman[68214]: 2024-04-25 11:39:42.999484395 +0000 UTC m=+0.475516379 container attach 3a9d263f0fabbaf37139b56506b2057c723991f146fef8cd9a446939e377664e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T11:39:43.216 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:42 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:42.965+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:43.217 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:42 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:42.965+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:51.169170+0000 front 2024-04-25T11:38:51.169194+0000 (oldest deadline 2024-04-25T11:39:17.069398+0000) 2024-04-25T11:39:43.217 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:42 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:42.965+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.170746+0000 front 2024-04-25T11:39:01.170670+0000 (oldest deadline 2024-04-25T11:39:22.870611+0000) 2024-04-25T11:39:43.217 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:42 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:42.965+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.472156+0000 front 2024-04-25T11:39:07.472172+0000 (oldest deadline 2024-04-25T11:39:30.972244+0000) 2024-04-25T11:39:43.217 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:42 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:42.965+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:15.673283+0000 front 2024-04-25T11:39:15.673348+0000 (oldest deadline 2024-04-25T11:39:39.173434+0000) 2024-04-25T11:39:43.496 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:43 smithi146 podman[68214]: 2024-04-25 11:39:43.229963881 +0000 UTC m=+0.705995845 container died 3a9d263f0fabbaf37139b56506b2057c723991f146fef8cd9a446939e377664e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-25T11:39:43.686 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:43 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:43.363+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.662848+0000 front 2024-04-25T11:38:44.663184+0000 (oldest deadline 2024-04-25T11:39:09.363059+0000) 2024-04-25T11:39:43.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:43 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:43.363+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:49.864482+0000 front 2024-04-25T11:38:49.864465+0000 (oldest deadline 2024-04-25T11:39:19.264749+0000) 2024-04-25T11:39:43.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:43 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:43.363+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.765972+0000 front 2024-04-25T11:38:59.765844+0000 (oldest deadline 2024-04-25T11:39:22.065909+0000) 2024-04-25T11:39:43.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:43 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:43.363+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.867091+0000 front 2024-04-25T11:39:07.867122+0000 (oldest deadline 2024-04-25T11:39:33.167199+0000) 2024-04-25T11:39:43.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:43 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:43.363+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:14.268262+0000 front 2024-04-25T11:39:14.268340+0000 (oldest deadline 2024-04-25T11:39:38.968155+0000) 2024-04-25T11:39:43.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:43 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:43.579+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:52.418405+0000 front 2024-04-25T11:38:52.418385+0000 (oldest deadline 2024-04-25T11:39:15.918284+0000) 2024-04-25T11:39:43.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:43 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:43.579+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:55.918916+0000 front 2024-04-25T11:38:55.919016+0000 (oldest deadline 2024-04-25T11:39:21.819129+0000) 2024-04-25T11:39:43.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:43 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:43.579+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:05.920608+0000 front 2024-04-25T11:39:05.920695+0000 (oldest deadline 2024-04-25T11:39:31.220387+0000) 2024-04-25T11:39:43.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:43 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:43.579+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:15.321402+0000 front 2024-04-25T11:39:15.321429+0000 (oldest deadline 2024-04-25T11:39:40.021407+0000) 2024-04-25T11:39:43.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:43 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:43.387+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:43.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:43 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:43.387+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:39:43.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:43 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:43.387+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:39:43.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:43 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:43.387+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:17.188600+0000 front 2024-04-25T11:39:17.188531+0000 (oldest deadline 2024-04-25T11:39:40.088473+0000) 2024-04-25T11:39:43.757 INFO:journalctl@ceph.osd.7.smithi146.stdout:Apr 25 11:39:43 smithi146 podman[68240]: 2024-04-25 11:39:43.754805303 +0000 UTC m=+0.518831683 container remove 3a9d263f0fabbaf37139b56506b2057c723991f146fef8cd9a446939e377664e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-7-deactivate, ceph=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-25T11:39:43.762 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@osd.7.service' 2024-04-25T11:39:43.798 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:39:43.798 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-25T11:39:43.798 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-25T11:39:43.799 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@osd.8 2024-04-25T11:39:44.092 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:43 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:43.916+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:44.093 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:43 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:43.916+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:51.169170+0000 front 2024-04-25T11:38:51.169194+0000 (oldest deadline 2024-04-25T11:39:17.069398+0000) 2024-04-25T11:39:44.093 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:43 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:43.916+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.170746+0000 front 2024-04-25T11:39:01.170670+0000 (oldest deadline 2024-04-25T11:39:22.870611+0000) 2024-04-25T11:39:44.093 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:43 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:43.916+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.472156+0000 front 2024-04-25T11:39:07.472172+0000 (oldest deadline 2024-04-25T11:39:30.972244+0000) 2024-04-25T11:39:44.093 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:43 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:43.916+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:15.673283+0000 front 2024-04-25T11:39:15.673348+0000 (oldest deadline 2024-04-25T11:39:39.173434+0000) 2024-04-25T11:39:44.114 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:43 smithi152 systemd[1]: Stopping Ceph osd.8 for f7ed73dc-02f4-11ef-bc93-c7b262605968... 2024-04-25T11:39:44.421 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:44 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:44.127+0000 7f29d3258700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:39:44.421 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:44 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:44.127+0000 7f29d3258700 -1 osd.8 88 *** Got signal Terminated *** 2024-04-25T11:39:44.421 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:44 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:44.127+0000 7f29d3258700 -1 osd.8 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T11:39:44.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:44 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:44.322+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.662848+0000 front 2024-04-25T11:38:44.663184+0000 (oldest deadline 2024-04-25T11:39:09.363059+0000) 2024-04-25T11:39:44.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:44 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:44.322+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:49.864482+0000 front 2024-04-25T11:38:49.864465+0000 (oldest deadline 2024-04-25T11:39:19.264749+0000) 2024-04-25T11:39:44.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:44 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:44.322+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.765972+0000 front 2024-04-25T11:38:59.765844+0000 (oldest deadline 2024-04-25T11:39:22.065909+0000) 2024-04-25T11:39:44.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:44 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:44.322+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.867091+0000 front 2024-04-25T11:39:07.867122+0000 (oldest deadline 2024-04-25T11:39:33.167199+0000) 2024-04-25T11:39:44.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:44 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:44.322+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:14.268262+0000 front 2024-04-25T11:39:14.268340+0000 (oldest deadline 2024-04-25T11:39:38.968155+0000) 2024-04-25T11:39:44.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:44 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:44.605+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:52.418405+0000 front 2024-04-25T11:38:52.418385+0000 (oldest deadline 2024-04-25T11:39:15.918284+0000) 2024-04-25T11:39:44.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:44 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:44.605+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:55.918916+0000 front 2024-04-25T11:38:55.919016+0000 (oldest deadline 2024-04-25T11:39:21.819129+0000) 2024-04-25T11:39:44.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:44 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:44.605+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:05.920608+0000 front 2024-04-25T11:39:05.920695+0000 (oldest deadline 2024-04-25T11:39:31.220387+0000) 2024-04-25T11:39:44.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:44 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:44.605+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:15.321402+0000 front 2024-04-25T11:39:15.321429+0000 (oldest deadline 2024-04-25T11:39:40.021407+0000) 2024-04-25T11:39:44.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:44 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:44.434+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:44.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:44 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:44.434+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:39:44.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:44 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:44.434+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:39:44.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:44 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:44.434+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:17.188600+0000 front 2024-04-25T11:39:17.188531+0000 (oldest deadline 2024-04-25T11:39:40.088473+0000) 2024-04-25T11:39:45.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:44 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:44.943+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:45.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:44 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:44.943+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:51.169170+0000 front 2024-04-25T11:38:51.169194+0000 (oldest deadline 2024-04-25T11:39:17.069398+0000) 2024-04-25T11:39:45.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:44 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:44.943+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.170746+0000 front 2024-04-25T11:39:01.170670+0000 (oldest deadline 2024-04-25T11:39:22.870611+0000) 2024-04-25T11:39:45.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:44 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:44.943+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.472156+0000 front 2024-04-25T11:39:07.472172+0000 (oldest deadline 2024-04-25T11:39:30.972244+0000) 2024-04-25T11:39:45.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:44 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:44.943+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:15.673283+0000 front 2024-04-25T11:39:15.673348+0000 (oldest deadline 2024-04-25T11:39:39.173434+0000) 2024-04-25T11:39:45.686 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:45 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:45.335+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.662848+0000 front 2024-04-25T11:38:44.663184+0000 (oldest deadline 2024-04-25T11:39:09.363059+0000) 2024-04-25T11:39:45.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:45 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:45.335+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:49.864482+0000 front 2024-04-25T11:38:49.864465+0000 (oldest deadline 2024-04-25T11:39:19.264749+0000) 2024-04-25T11:39:45.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:45 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:45.335+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.765972+0000 front 2024-04-25T11:38:59.765844+0000 (oldest deadline 2024-04-25T11:39:22.065909+0000) 2024-04-25T11:39:45.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:45 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:45.335+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.867091+0000 front 2024-04-25T11:39:07.867122+0000 (oldest deadline 2024-04-25T11:39:33.167199+0000) 2024-04-25T11:39:45.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:45 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:45.335+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:14.268262+0000 front 2024-04-25T11:39:14.268340+0000 (oldest deadline 2024-04-25T11:39:38.968155+0000) 2024-04-25T11:39:45.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:45 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:45.632+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:52.418405+0000 front 2024-04-25T11:38:52.418385+0000 (oldest deadline 2024-04-25T11:39:15.918284+0000) 2024-04-25T11:39:45.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:45 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:45.632+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:55.918916+0000 front 2024-04-25T11:38:55.919016+0000 (oldest deadline 2024-04-25T11:39:21.819129+0000) 2024-04-25T11:39:45.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:45 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:45.632+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:05.920608+0000 front 2024-04-25T11:39:05.920695+0000 (oldest deadline 2024-04-25T11:39:31.220387+0000) 2024-04-25T11:39:45.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:45 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:45.632+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:15.321402+0000 front 2024-04-25T11:39:15.321429+0000 (oldest deadline 2024-04-25T11:39:40.021407+0000) 2024-04-25T11:39:45.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:45 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:45.474+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:45.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:45 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:45.474+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:39:45.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:45 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:45.474+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:39:45.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:45 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:45.474+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:17.188600+0000 front 2024-04-25T11:39:17.188531+0000 (oldest deadline 2024-04-25T11:39:40.088473+0000) 2024-04-25T11:39:46.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:45 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:45.957+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:46.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:45 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:45.957+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:51.169170+0000 front 2024-04-25T11:38:51.169194+0000 (oldest deadline 2024-04-25T11:39:17.069398+0000) 2024-04-25T11:39:46.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:45 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:45.957+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.170746+0000 front 2024-04-25T11:39:01.170670+0000 (oldest deadline 2024-04-25T11:39:22.870611+0000) 2024-04-25T11:39:46.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:45 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:45.957+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.472156+0000 front 2024-04-25T11:39:07.472172+0000 (oldest deadline 2024-04-25T11:39:30.972244+0000) 2024-04-25T11:39:46.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:45 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:45.957+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:15.673283+0000 front 2024-04-25T11:39:15.673348+0000 (oldest deadline 2024-04-25T11:39:39.173434+0000) 2024-04-25T11:39:46.686 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:46 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:46.304+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.662848+0000 front 2024-04-25T11:38:44.663184+0000 (oldest deadline 2024-04-25T11:39:09.363059+0000) 2024-04-25T11:39:46.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:46 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:46.304+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:49.864482+0000 front 2024-04-25T11:38:49.864465+0000 (oldest deadline 2024-04-25T11:39:19.264749+0000) 2024-04-25T11:39:46.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:46 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:46.304+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.765972+0000 front 2024-04-25T11:38:59.765844+0000 (oldest deadline 2024-04-25T11:39:22.065909+0000) 2024-04-25T11:39:46.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:46 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:46.304+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.867091+0000 front 2024-04-25T11:39:07.867122+0000 (oldest deadline 2024-04-25T11:39:33.167199+0000) 2024-04-25T11:39:46.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:46 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:46.304+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:14.268262+0000 front 2024-04-25T11:39:14.268340+0000 (oldest deadline 2024-04-25T11:39:38.968155+0000) 2024-04-25T11:39:46.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:46 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:46.666+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:52.418405+0000 front 2024-04-25T11:38:52.418385+0000 (oldest deadline 2024-04-25T11:39:15.918284+0000) 2024-04-25T11:39:46.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:46 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:46.666+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:55.918916+0000 front 2024-04-25T11:38:55.919016+0000 (oldest deadline 2024-04-25T11:39:21.819129+0000) 2024-04-25T11:39:46.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:46 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:46.666+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:05.920608+0000 front 2024-04-25T11:39:05.920695+0000 (oldest deadline 2024-04-25T11:39:31.220387+0000) 2024-04-25T11:39:46.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:46 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:46.666+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:15.321402+0000 front 2024-04-25T11:39:15.321429+0000 (oldest deadline 2024-04-25T11:39:40.021407+0000) 2024-04-25T11:39:46.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:46 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:46.444+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:46.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:46 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:46.444+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:39:46.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:46 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:46.444+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:39:46.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:46 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:46.444+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:17.188600+0000 front 2024-04-25T11:39:17.188531+0000 (oldest deadline 2024-04-25T11:39:40.088473+0000) 2024-04-25T11:39:47.342 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:46 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:46.972+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:47.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:46 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:46.972+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:51.169170+0000 front 2024-04-25T11:38:51.169194+0000 (oldest deadline 2024-04-25T11:39:17.069398+0000) 2024-04-25T11:39:47.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:46 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:46.972+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.170746+0000 front 2024-04-25T11:39:01.170670+0000 (oldest deadline 2024-04-25T11:39:22.870611+0000) 2024-04-25T11:39:47.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:46 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:46.972+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.472156+0000 front 2024-04-25T11:39:07.472172+0000 (oldest deadline 2024-04-25T11:39:30.972244+0000) 2024-04-25T11:39:47.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:46 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:46.972+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:15.673283+0000 front 2024-04-25T11:39:15.673348+0000 (oldest deadline 2024-04-25T11:39:39.173434+0000) 2024-04-25T11:39:47.686 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:47 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:47.346+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.662848+0000 front 2024-04-25T11:38:44.663184+0000 (oldest deadline 2024-04-25T11:39:09.363059+0000) 2024-04-25T11:39:47.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:47 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:47.346+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:49.864482+0000 front 2024-04-25T11:38:49.864465+0000 (oldest deadline 2024-04-25T11:39:19.264749+0000) 2024-04-25T11:39:47.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:47 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:47.346+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.765972+0000 front 2024-04-25T11:38:59.765844+0000 (oldest deadline 2024-04-25T11:39:22.065909+0000) 2024-04-25T11:39:47.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:47 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:47.346+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.867091+0000 front 2024-04-25T11:39:07.867122+0000 (oldest deadline 2024-04-25T11:39:33.167199+0000) 2024-04-25T11:39:47.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:47 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:47.346+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:14.268262+0000 front 2024-04-25T11:39:14.268340+0000 (oldest deadline 2024-04-25T11:39:38.968155+0000) 2024-04-25T11:39:47.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:47 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:47.422+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:47.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:47 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:47.422+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:39:47.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:47 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:47.422+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:39:47.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:47 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:47.422+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:17.188600+0000 front 2024-04-25T11:39:17.188531+0000 (oldest deadline 2024-04-25T11:39:40.088473+0000) 2024-04-25T11:39:47.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:47 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:47.422+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:21.789497+0000 front 2024-04-25T11:39:21.789585+0000 (oldest deadline 2024-04-25T11:39:46.489651+0000) 2024-04-25T11:39:47.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:47 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:47.660+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:52.418405+0000 front 2024-04-25T11:38:52.418385+0000 (oldest deadline 2024-04-25T11:39:15.918284+0000) 2024-04-25T11:39:47.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:47 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:47.660+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:55.918916+0000 front 2024-04-25T11:38:55.919016+0000 (oldest deadline 2024-04-25T11:39:21.819129+0000) 2024-04-25T11:39:47.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:47 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:47.660+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:05.920608+0000 front 2024-04-25T11:39:05.920695+0000 (oldest deadline 2024-04-25T11:39:31.220387+0000) 2024-04-25T11:39:47.725 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:47 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:47.660+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:15.321402+0000 front 2024-04-25T11:39:15.321429+0000 (oldest deadline 2024-04-25T11:39:40.021407+0000) 2024-04-25T11:39:47.725 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:47 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:47.660+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:23.522282+0000 front 2024-04-25T11:39:23.522331+0000 (oldest deadline 2024-04-25T11:39:47.022303+0000) 2024-04-25T11:39:48.342 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:48 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:48.005+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:48.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:48 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:48.005+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:51.169170+0000 front 2024-04-25T11:38:51.169194+0000 (oldest deadline 2024-04-25T11:39:17.069398+0000) 2024-04-25T11:39:48.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:48 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:48.005+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.170746+0000 front 2024-04-25T11:39:01.170670+0000 (oldest deadline 2024-04-25T11:39:22.870611+0000) 2024-04-25T11:39:48.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:48 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:48.005+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.472156+0000 front 2024-04-25T11:39:07.472172+0000 (oldest deadline 2024-04-25T11:39:30.972244+0000) 2024-04-25T11:39:48.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:48 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:48.005+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:15.673283+0000 front 2024-04-25T11:39:15.673348+0000 (oldest deadline 2024-04-25T11:39:39.173434+0000) 2024-04-25T11:39:48.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:48 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:48.005+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:25.074468+0000 front 2024-04-25T11:39:25.074569+0000 (oldest deadline 2024-04-25T11:39:47.974213+0000) 2024-04-25T11:39:48.686 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:48 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:48.337+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.662848+0000 front 2024-04-25T11:38:44.663184+0000 (oldest deadline 2024-04-25T11:39:09.363059+0000) 2024-04-25T11:39:48.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:48 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:48.337+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:49.864482+0000 front 2024-04-25T11:38:49.864465+0000 (oldest deadline 2024-04-25T11:39:19.264749+0000) 2024-04-25T11:39:48.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:48 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:48.337+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.765972+0000 front 2024-04-25T11:38:59.765844+0000 (oldest deadline 2024-04-25T11:39:22.065909+0000) 2024-04-25T11:39:48.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:48 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:48.337+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.867091+0000 front 2024-04-25T11:39:07.867122+0000 (oldest deadline 2024-04-25T11:39:33.167199+0000) 2024-04-25T11:39:48.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:48 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:48.337+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:14.268262+0000 front 2024-04-25T11:39:14.268340+0000 (oldest deadline 2024-04-25T11:39:38.968155+0000) 2024-04-25T11:39:48.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:48 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:48.446+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:48.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:48 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:48.446+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:39:48.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:48 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:48.446+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:39:48.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:48 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:48.446+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:17.188600+0000 front 2024-04-25T11:39:17.188531+0000 (oldest deadline 2024-04-25T11:39:40.088473+0000) 2024-04-25T11:39:48.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:48 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:48.446+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:21.789497+0000 front 2024-04-25T11:39:21.789585+0000 (oldest deadline 2024-04-25T11:39:46.489651+0000) 2024-04-25T11:39:48.724 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:48 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:48.695+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:52.418405+0000 front 2024-04-25T11:38:52.418385+0000 (oldest deadline 2024-04-25T11:39:15.918284+0000) 2024-04-25T11:39:48.725 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:48 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:48.695+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:55.918916+0000 front 2024-04-25T11:38:55.919016+0000 (oldest deadline 2024-04-25T11:39:21.819129+0000) 2024-04-25T11:39:48.725 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:48 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:48.695+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:05.920608+0000 front 2024-04-25T11:39:05.920695+0000 (oldest deadline 2024-04-25T11:39:31.220387+0000) 2024-04-25T11:39:48.725 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:48 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:48.695+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:15.321402+0000 front 2024-04-25T11:39:15.321429+0000 (oldest deadline 2024-04-25T11:39:40.021407+0000) 2024-04-25T11:39:48.725 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:48 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8[51382]: 2024-04-25T11:39:48.695+0000 7f29cb806700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:23.522282+0000 front 2024-04-25T11:39:23.522331+0000 (oldest deadline 2024-04-25T11:39:47.022303+0000) 2024-04-25T11:39:49.342 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:48 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:48.962+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:49.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:48 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:48.962+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:51.169170+0000 front 2024-04-25T11:38:51.169194+0000 (oldest deadline 2024-04-25T11:39:17.069398+0000) 2024-04-25T11:39:49.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:48 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:48.962+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.170746+0000 front 2024-04-25T11:39:01.170670+0000 (oldest deadline 2024-04-25T11:39:22.870611+0000) 2024-04-25T11:39:49.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:48 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:48.962+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.472156+0000 front 2024-04-25T11:39:07.472172+0000 (oldest deadline 2024-04-25T11:39:30.972244+0000) 2024-04-25T11:39:49.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:48 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:48.962+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:15.673283+0000 front 2024-04-25T11:39:15.673348+0000 (oldest deadline 2024-04-25T11:39:39.173434+0000) 2024-04-25T11:39:49.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:48 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:48.962+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:25.074468+0000 front 2024-04-25T11:39:25.074569+0000 (oldest deadline 2024-04-25T11:39:47.974213+0000) 2024-04-25T11:39:49.686 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:49 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:49.351+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.662848+0000 front 2024-04-25T11:38:44.663184+0000 (oldest deadline 2024-04-25T11:39:09.363059+0000) 2024-04-25T11:39:49.788 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:49 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:49.351+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:49.864482+0000 front 2024-04-25T11:38:49.864465+0000 (oldest deadline 2024-04-25T11:39:19.264749+0000) 2024-04-25T11:39:49.789 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:49 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:49.351+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.765972+0000 front 2024-04-25T11:38:59.765844+0000 (oldest deadline 2024-04-25T11:39:22.065909+0000) 2024-04-25T11:39:49.789 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:49 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:49.351+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.867091+0000 front 2024-04-25T11:39:07.867122+0000 (oldest deadline 2024-04-25T11:39:33.167199+0000) 2024-04-25T11:39:49.789 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:49 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:49.351+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:14.268262+0000 front 2024-04-25T11:39:14.268340+0000 (oldest deadline 2024-04-25T11:39:38.968155+0000) 2024-04-25T11:39:49.856 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:49 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:49.495+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:49.856 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:49 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:49.495+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:39:49.856 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:49 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:49.495+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:39:49.856 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:49 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:49.495+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:17.188600+0000 front 2024-04-25T11:39:17.188531+0000 (oldest deadline 2024-04-25T11:39:40.088473+0000) 2024-04-25T11:39:49.856 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:49 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:49.495+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:21.789497+0000 front 2024-04-25T11:39:21.789585+0000 (oldest deadline 2024-04-25T11:39:46.489651+0000) 2024-04-25T11:39:49.857 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:49 smithi152 podman[69732]: 2024-04-25 11:39:49.599343349 +0000 UTC m=+5.597770565 container died 4559c80314fbf5927b0776da9729284b70bdca02708df0fe76a1a9eafe4c4b91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD) 2024-04-25T11:39:50.206 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:49 smithi152 podman[69732]: 2024-04-25 11:39:49.986045806 +0000 UTC m=+5.984473022 container cleanup 4559c80314fbf5927b0776da9729284b70bdca02708df0fe76a1a9eafe4c4b91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0) 2024-04-25T11:39:50.207 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:49 smithi152 bash[69732]: ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8 2024-04-25T11:39:50.342 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:49 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:49.968+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:50.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:49 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:49.968+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:51.169170+0000 front 2024-04-25T11:38:51.169194+0000 (oldest deadline 2024-04-25T11:39:17.069398+0000) 2024-04-25T11:39:50.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:49 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:49.968+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.170746+0000 front 2024-04-25T11:39:01.170670+0000 (oldest deadline 2024-04-25T11:39:22.870611+0000) 2024-04-25T11:39:50.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:49 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:49.968+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.472156+0000 front 2024-04-25T11:39:07.472172+0000 (oldest deadline 2024-04-25T11:39:30.972244+0000) 2024-04-25T11:39:50.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:49 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:49.968+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:15.673283+0000 front 2024-04-25T11:39:15.673348+0000 (oldest deadline 2024-04-25T11:39:39.173434+0000) 2024-04-25T11:39:50.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:49 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:49.968+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:25.074468+0000 front 2024-04-25T11:39:25.074569+0000 (oldest deadline 2024-04-25T11:39:47.974213+0000) 2024-04-25T11:39:50.474 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:50 smithi152 podman[69761]: 2024-04-25 11:39:50.220442048 +0000 UTC m=+0.624567187 container remove 4559c80314fbf5927b0776da9729284b70bdca02708df0fe76a1a9eafe4c4b91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T11:39:50.474 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:50 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:50.456+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:50.474 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:50 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:50.456+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:39:50.474 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:50 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:50.456+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:39:50.474 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:50 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:50.456+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:17.188600+0000 front 2024-04-25T11:39:17.188531+0000 (oldest deadline 2024-04-25T11:39:40.088473+0000) 2024-04-25T11:39:50.474 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:50 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:50.456+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:21.789497+0000 front 2024-04-25T11:39:21.789585+0000 (oldest deadline 2024-04-25T11:39:46.489651+0000) 2024-04-25T11:39:50.686 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:50 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:50.353+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.662848+0000 front 2024-04-25T11:38:44.663184+0000 (oldest deadline 2024-04-25T11:39:09.363059+0000) 2024-04-25T11:39:50.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:50 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:50.353+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:49.864482+0000 front 2024-04-25T11:38:49.864465+0000 (oldest deadline 2024-04-25T11:39:19.264749+0000) 2024-04-25T11:39:50.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:50 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:50.353+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.765972+0000 front 2024-04-25T11:38:59.765844+0000 (oldest deadline 2024-04-25T11:39:22.065909+0000) 2024-04-25T11:39:50.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:50 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:50.353+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.867091+0000 front 2024-04-25T11:39:07.867122+0000 (oldest deadline 2024-04-25T11:39:33.167199+0000) 2024-04-25T11:39:50.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:50 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:50.353+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:14.268262+0000 front 2024-04-25T11:39:14.268340+0000 (oldest deadline 2024-04-25T11:39:38.968155+0000) 2024-04-25T11:39:50.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:50 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:50.353+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:20.669397+0000 front 2024-04-25T11:39:20.669415+0000 (oldest deadline 2024-04-25T11:39:50.069884+0000) 2024-04-25T11:39:50.794 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:50 smithi152 podman[69854]: 2024-04-25 11:39:50.432508135 +0000 UTC m=+0.017869179 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T11:39:50.794 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:50 smithi152 podman[69854]: 2024-04-25 11:39:50.554751791 +0000 UTC m=+0.140112825 container create fee9187155354a2005741061d5b6da8eabcb262a79700e5f41195ee8ab05b220 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-25T11:39:51.091 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:50 smithi152 podman[69854]: 2024-04-25 11:39:50.807749335 +0000 UTC m=+0.393110377 container init fee9187155354a2005741061d5b6da8eabcb262a79700e5f41195ee8ab05b220 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2) 2024-04-25T11:39:51.091 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:50 smithi152 podman[69854]: 2024-04-25 11:39:50.811126418 +0000 UTC m=+0.396487462 container start fee9187155354a2005741061d5b6da8eabcb262a79700e5f41195ee8ab05b220 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-25T11:39:51.091 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:50 smithi152 podman[69854]: 2024-04-25 11:39:50.890302196 +0000 UTC m=+0.475663244 container attach fee9187155354a2005741061d5b6da8eabcb262a79700e5f41195ee8ab05b220 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2) 2024-04-25T11:39:51.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:50 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:50.984+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:51.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:50 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:50.984+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:51.169170+0000 front 2024-04-25T11:38:51.169194+0000 (oldest deadline 2024-04-25T11:39:17.069398+0000) 2024-04-25T11:39:51.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:50 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:50.984+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.170746+0000 front 2024-04-25T11:39:01.170670+0000 (oldest deadline 2024-04-25T11:39:22.870611+0000) 2024-04-25T11:39:51.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:50 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:50.984+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.472156+0000 front 2024-04-25T11:39:07.472172+0000 (oldest deadline 2024-04-25T11:39:30.972244+0000) 2024-04-25T11:39:51.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:50 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:50.984+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:15.673283+0000 front 2024-04-25T11:39:15.673348+0000 (oldest deadline 2024-04-25T11:39:39.173434+0000) 2024-04-25T11:39:51.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:50 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:50.984+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:25.074468+0000 front 2024-04-25T11:39:25.074569+0000 (oldest deadline 2024-04-25T11:39:47.974213+0000) 2024-04-25T11:39:51.404 INFO:journalctl@ceph.osd.8.smithi152.stdout:Apr 25 11:39:51 smithi152 podman[69854]: 2024-04-25 11:39:51.104874058 +0000 UTC m=+0.690235113 container died fee9187155354a2005741061d5b6da8eabcb262a79700e5f41195ee8ab05b220 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-25T11:39:51.664 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:51 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:51.417+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:51.664 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:51 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:51.417+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:39:51.664 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:51 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:51.417+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:39:51.664 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:51 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:51.417+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:17.188600+0000 front 2024-04-25T11:39:17.188531+0000 (oldest deadline 2024-04-25T11:39:40.088473+0000) 2024-04-25T11:39:51.664 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:51 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:51.417+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:21.789497+0000 front 2024-04-25T11:39:21.789585+0000 (oldest deadline 2024-04-25T11:39:46.489651+0000) 2024-04-25T11:39:51.682 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@osd.8.service' 2024-04-25T11:39:51.686 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:51 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:51.325+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.662848+0000 front 2024-04-25T11:38:44.663184+0000 (oldest deadline 2024-04-25T11:39:09.363059+0000) 2024-04-25T11:39:51.686 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:51 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:51.325+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:49.864482+0000 front 2024-04-25T11:38:49.864465+0000 (oldest deadline 2024-04-25T11:39:19.264749+0000) 2024-04-25T11:39:51.686 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:51 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:51.325+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.765972+0000 front 2024-04-25T11:38:59.765844+0000 (oldest deadline 2024-04-25T11:39:22.065909+0000) 2024-04-25T11:39:51.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:51 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:51.325+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.867091+0000 front 2024-04-25T11:39:07.867122+0000 (oldest deadline 2024-04-25T11:39:33.167199+0000) 2024-04-25T11:39:51.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:51 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:51.325+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:14.268262+0000 front 2024-04-25T11:39:14.268340+0000 (oldest deadline 2024-04-25T11:39:38.968155+0000) 2024-04-25T11:39:51.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:51 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:51.325+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:20.669397+0000 front 2024-04-25T11:39:20.669415+0000 (oldest deadline 2024-04-25T11:39:50.069884+0000) 2024-04-25T11:39:51.719 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:39:51.720 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-25T11:39:51.720 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-25T11:39:51.720 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@osd.9 2024-04-25T11:39:52.042 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:51 smithi069 systemd[1]: Stopping Ceph osd.9 for f7ed73dc-02f4-11ef-bc93-c7b262605968... 2024-04-25T11:39:52.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:52 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:52.005+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:52.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:52 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:52.005+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:51.169170+0000 front 2024-04-25T11:38:51.169194+0000 (oldest deadline 2024-04-25T11:39:17.069398+0000) 2024-04-25T11:39:52.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:52 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:52.005+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.170746+0000 front 2024-04-25T11:39:01.170670+0000 (oldest deadline 2024-04-25T11:39:22.870611+0000) 2024-04-25T11:39:52.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:52 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:52.005+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.472156+0000 front 2024-04-25T11:39:07.472172+0000 (oldest deadline 2024-04-25T11:39:30.972244+0000) 2024-04-25T11:39:52.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:52 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:52.005+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:15.673283+0000 front 2024-04-25T11:39:15.673348+0000 (oldest deadline 2024-04-25T11:39:39.173434+0000) 2024-04-25T11:39:52.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:52 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:52.005+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:25.074468+0000 front 2024-04-25T11:39:25.074569+0000 (oldest deadline 2024-04-25T11:39:47.974213+0000) 2024-04-25T11:39:52.437 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:52 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:52.054+0000 7f4f3d037700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:39:52.437 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:52 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:52.054+0000 7f4f3d037700 -1 osd.9 88 *** Got signal Terminated *** 2024-04-25T11:39:52.437 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:52 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:52.054+0000 7f4f3d037700 -1 osd.9 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T11:39:52.437 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:52 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:52.303+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.662848+0000 front 2024-04-25T11:38:44.663184+0000 (oldest deadline 2024-04-25T11:39:09.363059+0000) 2024-04-25T11:39:52.437 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:52 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:52.303+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:49.864482+0000 front 2024-04-25T11:38:49.864465+0000 (oldest deadline 2024-04-25T11:39:19.264749+0000) 2024-04-25T11:39:52.437 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:52 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:52.303+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.765972+0000 front 2024-04-25T11:38:59.765844+0000 (oldest deadline 2024-04-25T11:39:22.065909+0000) 2024-04-25T11:39:52.437 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:52 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:52.303+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.867091+0000 front 2024-04-25T11:39:07.867122+0000 (oldest deadline 2024-04-25T11:39:33.167199+0000) 2024-04-25T11:39:52.437 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:52 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:52.303+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:14.268262+0000 front 2024-04-25T11:39:14.268340+0000 (oldest deadline 2024-04-25T11:39:38.968155+0000) 2024-04-25T11:39:52.437 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:52 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:52.303+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:20.669397+0000 front 2024-04-25T11:39:20.669415+0000 (oldest deadline 2024-04-25T11:39:50.069884+0000) 2024-04-25T11:39:52.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:52 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:52.410+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:52.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:52 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:52.410+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:39:52.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:52 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:52.410+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:39:52.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:52 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:52.410+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:17.188600+0000 front 2024-04-25T11:39:17.188531+0000 (oldest deadline 2024-04-25T11:39:40.088473+0000) 2024-04-25T11:39:52.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:52 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:52.410+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:21.789497+0000 front 2024-04-25T11:39:21.789585+0000 (oldest deadline 2024-04-25T11:39:46.489651+0000) 2024-04-25T11:39:53.244 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:52 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:52.984+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:53.244 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:52 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:52.984+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:51.169170+0000 front 2024-04-25T11:38:51.169194+0000 (oldest deadline 2024-04-25T11:39:17.069398+0000) 2024-04-25T11:39:53.244 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:52 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:52.984+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.170746+0000 front 2024-04-25T11:39:01.170670+0000 (oldest deadline 2024-04-25T11:39:22.870611+0000) 2024-04-25T11:39:53.244 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:52 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:52.984+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.472156+0000 front 2024-04-25T11:39:07.472172+0000 (oldest deadline 2024-04-25T11:39:30.972244+0000) 2024-04-25T11:39:53.244 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:52 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:52.984+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:15.673283+0000 front 2024-04-25T11:39:15.673348+0000 (oldest deadline 2024-04-25T11:39:39.173434+0000) 2024-04-25T11:39:53.245 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:52 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:52.984+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:25.074468+0000 front 2024-04-25T11:39:25.074569+0000 (oldest deadline 2024-04-25T11:39:47.974213+0000) 2024-04-25T11:39:53.686 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:53 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:53.279+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.662848+0000 front 2024-04-25T11:38:44.663184+0000 (oldest deadline 2024-04-25T11:39:09.363059+0000) 2024-04-25T11:39:53.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:53 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:53.279+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:49.864482+0000 front 2024-04-25T11:38:49.864465+0000 (oldest deadline 2024-04-25T11:39:19.264749+0000) 2024-04-25T11:39:53.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:53 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:53.279+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.765972+0000 front 2024-04-25T11:38:59.765844+0000 (oldest deadline 2024-04-25T11:39:22.065909+0000) 2024-04-25T11:39:53.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:53 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:53.279+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.867091+0000 front 2024-04-25T11:39:07.867122+0000 (oldest deadline 2024-04-25T11:39:33.167199+0000) 2024-04-25T11:39:53.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:53 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:53.279+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:14.268262+0000 front 2024-04-25T11:39:14.268340+0000 (oldest deadline 2024-04-25T11:39:38.968155+0000) 2024-04-25T11:39:53.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:53 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:53.279+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:20.669397+0000 front 2024-04-25T11:39:20.669415+0000 (oldest deadline 2024-04-25T11:39:50.069884+0000) 2024-04-25T11:39:53.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:53 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:53.411+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:53.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:53 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:53.411+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:39:53.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:53 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:53.411+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:39:53.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:53 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:53.411+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:17.188600+0000 front 2024-04-25T11:39:17.188531+0000 (oldest deadline 2024-04-25T11:39:40.088473+0000) 2024-04-25T11:39:53.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:53 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:53.411+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:21.789497+0000 front 2024-04-25T11:39:21.789585+0000 (oldest deadline 2024-04-25T11:39:46.489651+0000) 2024-04-25T11:39:54.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:54 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:53.999+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:54.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:54 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:53.999+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:51.169170+0000 front 2024-04-25T11:38:51.169194+0000 (oldest deadline 2024-04-25T11:39:17.069398+0000) 2024-04-25T11:39:54.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:54 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:53.999+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.170746+0000 front 2024-04-25T11:39:01.170670+0000 (oldest deadline 2024-04-25T11:39:22.870611+0000) 2024-04-25T11:39:54.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:54 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:53.999+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.472156+0000 front 2024-04-25T11:39:07.472172+0000 (oldest deadline 2024-04-25T11:39:30.972244+0000) 2024-04-25T11:39:54.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:54 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:53.999+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:15.673283+0000 front 2024-04-25T11:39:15.673348+0000 (oldest deadline 2024-04-25T11:39:39.173434+0000) 2024-04-25T11:39:54.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:54 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:53.999+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:25.074468+0000 front 2024-04-25T11:39:25.074569+0000 (oldest deadline 2024-04-25T11:39:47.974213+0000) 2024-04-25T11:39:54.686 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:54 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:54.293+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.662848+0000 front 2024-04-25T11:38:44.663184+0000 (oldest deadline 2024-04-25T11:39:09.363059+0000) 2024-04-25T11:39:54.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:54 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:54.293+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:49.864482+0000 front 2024-04-25T11:38:49.864465+0000 (oldest deadline 2024-04-25T11:39:19.264749+0000) 2024-04-25T11:39:54.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:54 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:54.293+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.765972+0000 front 2024-04-25T11:38:59.765844+0000 (oldest deadline 2024-04-25T11:39:22.065909+0000) 2024-04-25T11:39:54.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:54 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:54.293+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.867091+0000 front 2024-04-25T11:39:07.867122+0000 (oldest deadline 2024-04-25T11:39:33.167199+0000) 2024-04-25T11:39:54.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:54 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:54.293+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:14.268262+0000 front 2024-04-25T11:39:14.268340+0000 (oldest deadline 2024-04-25T11:39:38.968155+0000) 2024-04-25T11:39:54.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:54 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:54.293+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:20.669397+0000 front 2024-04-25T11:39:20.669415+0000 (oldest deadline 2024-04-25T11:39:50.069884+0000) 2024-04-25T11:39:54.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:54 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:54.376+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:54.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:54 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:54.376+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:39:54.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:54 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:54.376+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:39:54.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:54 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:54.376+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:17.188600+0000 front 2024-04-25T11:39:17.188531+0000 (oldest deadline 2024-04-25T11:39:40.088473+0000) 2024-04-25T11:39:54.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:54 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:54.376+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:21.789497+0000 front 2024-04-25T11:39:21.789585+0000 (oldest deadline 2024-04-25T11:39:46.489651+0000) 2024-04-25T11:39:55.342 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:55 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:55.036+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:55.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:55 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:55.036+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:51.169170+0000 front 2024-04-25T11:38:51.169194+0000 (oldest deadline 2024-04-25T11:39:17.069398+0000) 2024-04-25T11:39:55.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:55 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:55.036+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.170746+0000 front 2024-04-25T11:39:01.170670+0000 (oldest deadline 2024-04-25T11:39:22.870611+0000) 2024-04-25T11:39:55.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:55 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:55.036+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.472156+0000 front 2024-04-25T11:39:07.472172+0000 (oldest deadline 2024-04-25T11:39:30.972244+0000) 2024-04-25T11:39:55.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:55 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:55.036+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:15.673283+0000 front 2024-04-25T11:39:15.673348+0000 (oldest deadline 2024-04-25T11:39:39.173434+0000) 2024-04-25T11:39:55.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:55 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:55.036+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:25.074468+0000 front 2024-04-25T11:39:25.074569+0000 (oldest deadline 2024-04-25T11:39:47.974213+0000) 2024-04-25T11:39:55.686 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:55 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:55.283+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.662848+0000 front 2024-04-25T11:38:44.663184+0000 (oldest deadline 2024-04-25T11:39:09.363059+0000) 2024-04-25T11:39:55.686 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:55 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:55.283+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:49.864482+0000 front 2024-04-25T11:38:49.864465+0000 (oldest deadline 2024-04-25T11:39:19.264749+0000) 2024-04-25T11:39:55.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:55 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:55.283+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.765972+0000 front 2024-04-25T11:38:59.765844+0000 (oldest deadline 2024-04-25T11:39:22.065909+0000) 2024-04-25T11:39:55.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:55 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:55.283+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.867091+0000 front 2024-04-25T11:39:07.867122+0000 (oldest deadline 2024-04-25T11:39:33.167199+0000) 2024-04-25T11:39:55.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:55 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:55.283+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:14.268262+0000 front 2024-04-25T11:39:14.268340+0000 (oldest deadline 2024-04-25T11:39:38.968155+0000) 2024-04-25T11:39:55.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:55 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:55.283+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:20.669397+0000 front 2024-04-25T11:39:20.669415+0000 (oldest deadline 2024-04-25T11:39:50.069884+0000) 2024-04-25T11:39:55.723 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:55 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:55.376+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:55.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:55 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:55.376+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:39:55.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:55 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:55.376+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:39:55.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:55 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:55.376+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:17.188600+0000 front 2024-04-25T11:39:17.188531+0000 (oldest deadline 2024-04-25T11:39:40.088473+0000) 2024-04-25T11:39:55.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:55 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:55.376+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:21.789497+0000 front 2024-04-25T11:39:21.789585+0000 (oldest deadline 2024-04-25T11:39:46.489651+0000) 2024-04-25T11:39:55.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:55 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:55.376+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-25T11:39:31.190893+0000 front 2024-04-25T11:39:31.190759+0000 (oldest deadline 2024-04-25T11:39:54.690637+0000) 2024-04-25T11:39:56.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:56 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:56.034+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:56.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:56 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:56.034+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:51.169170+0000 front 2024-04-25T11:38:51.169194+0000 (oldest deadline 2024-04-25T11:39:17.069398+0000) 2024-04-25T11:39:56.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:56 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:56.034+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.170746+0000 front 2024-04-25T11:39:01.170670+0000 (oldest deadline 2024-04-25T11:39:22.870611+0000) 2024-04-25T11:39:56.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:56 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:56.034+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.472156+0000 front 2024-04-25T11:39:07.472172+0000 (oldest deadline 2024-04-25T11:39:30.972244+0000) 2024-04-25T11:39:56.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:56 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:56.034+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:15.673283+0000 front 2024-04-25T11:39:15.673348+0000 (oldest deadline 2024-04-25T11:39:39.173434+0000) 2024-04-25T11:39:56.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:56 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:56.034+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:25.074468+0000 front 2024-04-25T11:39:25.074569+0000 (oldest deadline 2024-04-25T11:39:47.974213+0000) 2024-04-25T11:39:56.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:56 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:56.034+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-25T11:39:33.275185+0000 front 2024-04-25T11:39:33.275270+0000 (oldest deadline 2024-04-25T11:39:55.575095+0000) 2024-04-25T11:39:56.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:56 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:56.262+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:44.662848+0000 front 2024-04-25T11:38:44.663184+0000 (oldest deadline 2024-04-25T11:39:09.363059+0000) 2024-04-25T11:39:56.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:56 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:56.262+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:49.864482+0000 front 2024-04-25T11:38:49.864465+0000 (oldest deadline 2024-04-25T11:39:19.264749+0000) 2024-04-25T11:39:56.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:56 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:56.262+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.765972+0000 front 2024-04-25T11:38:59.765844+0000 (oldest deadline 2024-04-25T11:39:22.065909+0000) 2024-04-25T11:39:56.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:56 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:56.262+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.867091+0000 front 2024-04-25T11:39:07.867122+0000 (oldest deadline 2024-04-25T11:39:33.167199+0000) 2024-04-25T11:39:56.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:56 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:56.262+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:14.268262+0000 front 2024-04-25T11:39:14.268340+0000 (oldest deadline 2024-04-25T11:39:38.968155+0000) 2024-04-25T11:39:56.687 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:56 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9[64213]: 2024-04-25T11:39:56.262+0000 7f4f355e5700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:20.669397+0000 front 2024-04-25T11:39:20.669415+0000 (oldest deadline 2024-04-25T11:39:50.069884+0000) 2024-04-25T11:39:56.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:56 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:56.348+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:56.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:56 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:56.348+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:39:56.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:56 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:56.348+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:39:56.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:56 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:56.348+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:17.188600+0000 front 2024-04-25T11:39:17.188531+0000 (oldest deadline 2024-04-25T11:39:40.088473+0000) 2024-04-25T11:39:56.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:56 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:56.348+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:21.789497+0000 front 2024-04-25T11:39:21.789585+0000 (oldest deadline 2024-04-25T11:39:46.489651+0000) 2024-04-25T11:39:56.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:56 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:56.348+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-25T11:39:31.190893+0000 front 2024-04-25T11:39:31.190759+0000 (oldest deadline 2024-04-25T11:39:54.690637+0000) 2024-04-25T11:39:57.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:57 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:57.021+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:57.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:57 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:57.021+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:51.169170+0000 front 2024-04-25T11:38:51.169194+0000 (oldest deadline 2024-04-25T11:39:17.069398+0000) 2024-04-25T11:39:57.344 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:57 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:57.021+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.170746+0000 front 2024-04-25T11:39:01.170670+0000 (oldest deadline 2024-04-25T11:39:22.870611+0000) 2024-04-25T11:39:57.344 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:57 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:57.021+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.472156+0000 front 2024-04-25T11:39:07.472172+0000 (oldest deadline 2024-04-25T11:39:30.972244+0000) 2024-04-25T11:39:57.344 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:57 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:57.021+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:15.673283+0000 front 2024-04-25T11:39:15.673348+0000 (oldest deadline 2024-04-25T11:39:39.173434+0000) 2024-04-25T11:39:57.344 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:57 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:57.021+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:25.074468+0000 front 2024-04-25T11:39:25.074569+0000 (oldest deadline 2024-04-25T11:39:47.974213+0000) 2024-04-25T11:39:57.344 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:57 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:57.021+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-25T11:39:33.275185+0000 front 2024-04-25T11:39:33.275270+0000 (oldest deadline 2024-04-25T11:39:55.575095+0000) 2024-04-25T11:39:57.658 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:57 smithi069 podman[89051]: 2024-04-25 11:39:57.392581776 +0000 UTC m=+5.459728950 container died 18114e4ddf16bff074995b476284357b6f6a672cc4893202ede0820dc5d2838d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-25T11:39:57.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:57 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:57.378+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:57.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:57 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:57.378+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:39:57.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:57 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:57.378+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:39:57.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:57 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:57.378+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:17.188600+0000 front 2024-04-25T11:39:17.188531+0000 (oldest deadline 2024-04-25T11:39:40.088473+0000) 2024-04-25T11:39:57.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:57 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:57.378+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:21.789497+0000 front 2024-04-25T11:39:21.789585+0000 (oldest deadline 2024-04-25T11:39:46.489651+0000) 2024-04-25T11:39:57.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:57 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:57.378+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-25T11:39:31.190893+0000 front 2024-04-25T11:39:31.190759+0000 (oldest deadline 2024-04-25T11:39:54.690637+0000) 2024-04-25T11:39:57.936 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:57 smithi069 podman[89051]: 2024-04-25 11:39:57.754430429 +0000 UTC m=+5.821577588 container cleanup 18114e4ddf16bff074995b476284357b6f6a672cc4893202ede0820dc5d2838d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T11:39:57.937 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:57 smithi069 bash[89051]: ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9 2024-04-25T11:39:58.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:58 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:58.057+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:58.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:58 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:58.057+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:51.169170+0000 front 2024-04-25T11:38:51.169194+0000 (oldest deadline 2024-04-25T11:39:17.069398+0000) 2024-04-25T11:39:58.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:58 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:58.057+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.170746+0000 front 2024-04-25T11:39:01.170670+0000 (oldest deadline 2024-04-25T11:39:22.870611+0000) 2024-04-25T11:39:58.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:58 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:58.057+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.472156+0000 front 2024-04-25T11:39:07.472172+0000 (oldest deadline 2024-04-25T11:39:30.972244+0000) 2024-04-25T11:39:58.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:58 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:58.057+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:15.673283+0000 front 2024-04-25T11:39:15.673348+0000 (oldest deadline 2024-04-25T11:39:39.173434+0000) 2024-04-25T11:39:58.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:58 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:58.057+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:25.074468+0000 front 2024-04-25T11:39:25.074569+0000 (oldest deadline 2024-04-25T11:39:47.974213+0000) 2024-04-25T11:39:58.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:58 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:58.057+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-25T11:39:33.275185+0000 front 2024-04-25T11:39:33.275270+0000 (oldest deadline 2024-04-25T11:39:55.575095+0000) 2024-04-25T11:39:58.364 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:58 smithi069 podman[89069]: 2024-04-25 11:39:58.053859452 +0000 UTC m=+0.660586328 container remove 18114e4ddf16bff074995b476284357b6f6a672cc4893202ede0820dc5d2838d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-25T11:39:58.662 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:58 smithi069 podman[89156]: 2024-04-25 11:39:58.281138221 +0000 UTC m=+0.017892693 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T11:39:58.662 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:58 smithi069 podman[89156]: 2024-04-25 11:39:58.389879121 +0000 UTC m=+0.126633612 container create f3d0e27ea56263af9fbbb9fedf38affb8cd0f360db74ddcd20e89d6b7580b04c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-25T11:39:58.662 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:58 smithi069 podman[89156]: 2024-04-25 11:39:58.600545138 +0000 UTC m=+0.337299587 container init f3d0e27ea56263af9fbbb9fedf38affb8cd0f360db74ddcd20e89d6b7580b04c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-25T11:39:58.662 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:58 smithi069 podman[89156]: 2024-04-25 11:39:58.603579167 +0000 UTC m=+0.340333617 container start f3d0e27ea56263af9fbbb9fedf38affb8cd0f360db74ddcd20e89d6b7580b04c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-25T11:39:58.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:58 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:58.356+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:58.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:58 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:58.356+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:39:58.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:58 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:58.356+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:39:58.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:58 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:58.356+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:17.188600+0000 front 2024-04-25T11:39:17.188531+0000 (oldest deadline 2024-04-25T11:39:40.088473+0000) 2024-04-25T11:39:58.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:58 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:58.356+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:21.789497+0000 front 2024-04-25T11:39:21.789585+0000 (oldest deadline 2024-04-25T11:39:46.489651+0000) 2024-04-25T11:39:58.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:58 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:58.356+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-25T11:39:31.190893+0000 front 2024-04-25T11:39:31.190759+0000 (oldest deadline 2024-04-25T11:39:54.690637+0000) 2024-04-25T11:39:58.937 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:58 smithi069 podman[89156]: 2024-04-25 11:39:58.675252779 +0000 UTC m=+0.412007239 container attach f3d0e27ea56263af9fbbb9fedf38affb8cd0f360db74ddcd20e89d6b7580b04c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T11:39:58.937 INFO:journalctl@ceph.osd.9.smithi069.stdout:Apr 25 11:39:58 smithi069 podman[89156]: 2024-04-25 11:39:58.905300637 +0000 UTC m=+0.642055100 container died f3d0e27ea56263af9fbbb9fedf38affb8cd0f360db74ddcd20e89d6b7580b04c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-25T11:39:59.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:59 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:59.019+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:39:59.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:59 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:59.019+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:51.169170+0000 front 2024-04-25T11:38:51.169194+0000 (oldest deadline 2024-04-25T11:39:17.069398+0000) 2024-04-25T11:39:59.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:59 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:59.019+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.170746+0000 front 2024-04-25T11:39:01.170670+0000 (oldest deadline 2024-04-25T11:39:22.870611+0000) 2024-04-25T11:39:59.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:59 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:59.019+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.472156+0000 front 2024-04-25T11:39:07.472172+0000 (oldest deadline 2024-04-25T11:39:30.972244+0000) 2024-04-25T11:39:59.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:59 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:59.019+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:15.673283+0000 front 2024-04-25T11:39:15.673348+0000 (oldest deadline 2024-04-25T11:39:39.173434+0000) 2024-04-25T11:39:59.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:59 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:59.019+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:25.074468+0000 front 2024-04-25T11:39:25.074569+0000 (oldest deadline 2024-04-25T11:39:47.974213+0000) 2024-04-25T11:39:59.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:59 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:59.019+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-25T11:39:33.275185+0000 front 2024-04-25T11:39:33.275270+0000 (oldest deadline 2024-04-25T11:39:55.575095+0000) 2024-04-25T11:39:59.489 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@osd.9.service' 2024-04-25T11:39:59.530 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:39:59.530 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-25T11:39:59.530 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-25T11:39:59.531 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@osd.10 2024-04-25T11:39:59.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:59 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:59.390+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:39:59.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:59 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:59.390+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:39:59.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:59 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:59.390+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:39:59.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:59 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:59.390+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:17.188600+0000 front 2024-04-25T11:39:17.188531+0000 (oldest deadline 2024-04-25T11:39:40.088473+0000) 2024-04-25T11:39:59.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:59 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:59.390+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:21.789497+0000 front 2024-04-25T11:39:21.789585+0000 (oldest deadline 2024-04-25T11:39:46.489651+0000) 2024-04-25T11:39:59.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:39:59 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:39:59.390+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-25T11:39:31.190893+0000 front 2024-04-25T11:39:31.190759+0000 (oldest deadline 2024-04-25T11:39:54.690637+0000) 2024-04-25T11:39:59.842 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:59 smithi146 systemd[1]: Stopping Ceph osd.10 for f7ed73dc-02f4-11ef-bc93-c7b262605968... 2024-04-25T11:40:00.342 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:59 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:59.864+0000 7f2629f12700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:40:00.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:59 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:59.864+0000 7f2629f12700 -1 osd.10 88 *** Got signal Terminated *** 2024-04-25T11:40:00.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:59 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:59.864+0000 7f2629f12700 -1 osd.10 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T11:40:00.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:59 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:59.992+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:40:00.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:59 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:59.992+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:51.169170+0000 front 2024-04-25T11:38:51.169194+0000 (oldest deadline 2024-04-25T11:39:17.069398+0000) 2024-04-25T11:40:00.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:59 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:59.992+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.170746+0000 front 2024-04-25T11:39:01.170670+0000 (oldest deadline 2024-04-25T11:39:22.870611+0000) 2024-04-25T11:40:00.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:59 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:59.992+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.472156+0000 front 2024-04-25T11:39:07.472172+0000 (oldest deadline 2024-04-25T11:39:30.972244+0000) 2024-04-25T11:40:00.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:59 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:59.992+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:15.673283+0000 front 2024-04-25T11:39:15.673348+0000 (oldest deadline 2024-04-25T11:39:39.173434+0000) 2024-04-25T11:40:00.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:59 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:59.992+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:25.074468+0000 front 2024-04-25T11:39:25.074569+0000 (oldest deadline 2024-04-25T11:39:47.974213+0000) 2024-04-25T11:40:00.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:39:59 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:39:59.992+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-25T11:39:33.275185+0000 front 2024-04-25T11:39:33.275270+0000 (oldest deadline 2024-04-25T11:39:55.575095+0000) 2024-04-25T11:40:00.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:00 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:00.407+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:40:00.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:00 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:00.407+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:40:00.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:00 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:00.407+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:40:00.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:00 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:00.407+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:17.188600+0000 front 2024-04-25T11:39:17.188531+0000 (oldest deadline 2024-04-25T11:39:40.088473+0000) 2024-04-25T11:40:00.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:00 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:00.407+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:21.789497+0000 front 2024-04-25T11:39:21.789585+0000 (oldest deadline 2024-04-25T11:39:46.489651+0000) 2024-04-25T11:40:00.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:00 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:00.407+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-25T11:39:31.190893+0000 front 2024-04-25T11:39:31.190759+0000 (oldest deadline 2024-04-25T11:39:54.690637+0000) 2024-04-25T11:40:01.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:40:01 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:40:01.019+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:40:01.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:40:01 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:40:01.020+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:51.169170+0000 front 2024-04-25T11:38:51.169194+0000 (oldest deadline 2024-04-25T11:39:17.069398+0000) 2024-04-25T11:40:01.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:40:01 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:40:01.020+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.170746+0000 front 2024-04-25T11:39:01.170670+0000 (oldest deadline 2024-04-25T11:39:22.870611+0000) 2024-04-25T11:40:01.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:40:01 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:40:01.020+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.472156+0000 front 2024-04-25T11:39:07.472172+0000 (oldest deadline 2024-04-25T11:39:30.972244+0000) 2024-04-25T11:40:01.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:40:01 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:40:01.020+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:15.673283+0000 front 2024-04-25T11:39:15.673348+0000 (oldest deadline 2024-04-25T11:39:39.173434+0000) 2024-04-25T11:40:01.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:40:01 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:40:01.020+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:25.074468+0000 front 2024-04-25T11:39:25.074569+0000 (oldest deadline 2024-04-25T11:39:47.974213+0000) 2024-04-25T11:40:01.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:40:01 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:40:01.020+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-25T11:39:33.275185+0000 front 2024-04-25T11:39:33.275270+0000 (oldest deadline 2024-04-25T11:39:55.575095+0000) 2024-04-25T11:40:01.474 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:01 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:01.373+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:40:01.474 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:01 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:01.373+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:40:01.474 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:01 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:01.373+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:40:01.474 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:01 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:01.373+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:17.188600+0000 front 2024-04-25T11:39:17.188531+0000 (oldest deadline 2024-04-25T11:39:40.088473+0000) 2024-04-25T11:40:01.474 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:01 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:01.373+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:21.789497+0000 front 2024-04-25T11:39:21.789585+0000 (oldest deadline 2024-04-25T11:39:46.489651+0000) 2024-04-25T11:40:01.474 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:01 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:01.373+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-25T11:39:31.190893+0000 front 2024-04-25T11:39:31.190759+0000 (oldest deadline 2024-04-25T11:39:54.690637+0000) 2024-04-25T11:40:02.342 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:40:01 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:40:01.991+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:40:02.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:40:01 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:40:01.991+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:51.169170+0000 front 2024-04-25T11:38:51.169194+0000 (oldest deadline 2024-04-25T11:39:17.069398+0000) 2024-04-25T11:40:02.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:40:01 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:40:01.991+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.170746+0000 front 2024-04-25T11:39:01.170670+0000 (oldest deadline 2024-04-25T11:39:22.870611+0000) 2024-04-25T11:40:02.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:40:01 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:40:01.991+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.472156+0000 front 2024-04-25T11:39:07.472172+0000 (oldest deadline 2024-04-25T11:39:30.972244+0000) 2024-04-25T11:40:02.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:40:01 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:40:01.991+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:15.673283+0000 front 2024-04-25T11:39:15.673348+0000 (oldest deadline 2024-04-25T11:39:39.173434+0000) 2024-04-25T11:40:02.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:40:01 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:40:01.991+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:25.074468+0000 front 2024-04-25T11:39:25.074569+0000 (oldest deadline 2024-04-25T11:39:47.974213+0000) 2024-04-25T11:40:02.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:40:01 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:40:01.991+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-25T11:39:33.275185+0000 front 2024-04-25T11:39:33.275270+0000 (oldest deadline 2024-04-25T11:39:55.575095+0000) 2024-04-25T11:40:02.725 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:02 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:02.346+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:40:02.725 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:02 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:02.346+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:40:02.725 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:02 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:02.346+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:40:02.725 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:02 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:02.346+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:17.188600+0000 front 2024-04-25T11:39:17.188531+0000 (oldest deadline 2024-04-25T11:39:40.088473+0000) 2024-04-25T11:40:02.725 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:02 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:02.346+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:21.789497+0000 front 2024-04-25T11:39:21.789585+0000 (oldest deadline 2024-04-25T11:39:46.489651+0000) 2024-04-25T11:40:02.725 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:02 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:02.346+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-25T11:39:31.190893+0000 front 2024-04-25T11:39:31.190759+0000 (oldest deadline 2024-04-25T11:39:54.690637+0000) 2024-04-25T11:40:03.342 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:40:02 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:40:02.974+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:40:03.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:40:02 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:40:02.974+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:51.169170+0000 front 2024-04-25T11:38:51.169194+0000 (oldest deadline 2024-04-25T11:39:17.069398+0000) 2024-04-25T11:40:03.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:40:02 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:40:02.974+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.170746+0000 front 2024-04-25T11:39:01.170670+0000 (oldest deadline 2024-04-25T11:39:22.870611+0000) 2024-04-25T11:40:03.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:40:02 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:40:02.974+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.472156+0000 front 2024-04-25T11:39:07.472172+0000 (oldest deadline 2024-04-25T11:39:30.972244+0000) 2024-04-25T11:40:03.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:40:02 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:40:02.974+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:15.673283+0000 front 2024-04-25T11:39:15.673348+0000 (oldest deadline 2024-04-25T11:39:39.173434+0000) 2024-04-25T11:40:03.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:40:02 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:40:02.974+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:25.074468+0000 front 2024-04-25T11:39:25.074569+0000 (oldest deadline 2024-04-25T11:39:47.974213+0000) 2024-04-25T11:40:03.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:40:02 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:40:02.974+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-25T11:39:33.275185+0000 front 2024-04-25T11:39:33.275270+0000 (oldest deadline 2024-04-25T11:39:55.575095+0000) 2024-04-25T11:40:03.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:03 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:03.370+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:40:03.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:03 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:03.370+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:40:03.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:03 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:03.370+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:40:03.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:03 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:03.370+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:17.188600+0000 front 2024-04-25T11:39:17.188531+0000 (oldest deadline 2024-04-25T11:39:40.088473+0000) 2024-04-25T11:40:03.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:03 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:03.370+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:21.789497+0000 front 2024-04-25T11:39:21.789585+0000 (oldest deadline 2024-04-25T11:39:46.489651+0000) 2024-04-25T11:40:03.725 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:03 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:03.370+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-25T11:39:31.190893+0000 front 2024-04-25T11:39:31.190759+0000 (oldest deadline 2024-04-25T11:39:54.690637+0000) 2024-04-25T11:40:04.230 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:40:03 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:40:03.939+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:41.168292+0000 front 2024-04-25T11:38:41.168228+0000 (oldest deadline 2024-04-25T11:39:06.468383+0000) 2024-04-25T11:40:04.230 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:40:03 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:40:03.939+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:51.169170+0000 front 2024-04-25T11:38:51.169194+0000 (oldest deadline 2024-04-25T11:39:17.069398+0000) 2024-04-25T11:40:04.230 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:40:03 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:40:03.939+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:39:01.170746+0000 front 2024-04-25T11:39:01.170670+0000 (oldest deadline 2024-04-25T11:39:22.870611+0000) 2024-04-25T11:40:04.231 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:40:03 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:40:03.939+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-04-25T11:39:07.472156+0000 front 2024-04-25T11:39:07.472172+0000 (oldest deadline 2024-04-25T11:39:30.972244+0000) 2024-04-25T11:40:04.231 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:40:03 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:40:03.939+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:15.673283+0000 front 2024-04-25T11:39:15.673348+0000 (oldest deadline 2024-04-25T11:39:39.173434+0000) 2024-04-25T11:40:04.231 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:40:03 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:40:03.939+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:25.074468+0000 front 2024-04-25T11:39:25.074569+0000 (oldest deadline 2024-04-25T11:39:47.974213+0000) 2024-04-25T11:40:04.231 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:40:03 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10[57535]: 2024-04-25T11:40:03.939+0000 7f26264c8700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-25T11:39:33.275185+0000 front 2024-04-25T11:39:33.275270+0000 (oldest deadline 2024-04-25T11:39:55.575095+0000) 2024-04-25T11:40:04.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:04 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:04.407+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:40:04.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:04 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:04.407+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:40:04.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:04 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:04.407+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:40:04.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:04 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:04.407+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:17.188600+0000 front 2024-04-25T11:39:17.188531+0000 (oldest deadline 2024-04-25T11:39:40.088473+0000) 2024-04-25T11:40:04.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:04 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:04.407+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:21.789497+0000 front 2024-04-25T11:39:21.789585+0000 (oldest deadline 2024-04-25T11:39:46.489651+0000) 2024-04-25T11:40:04.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:04 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:04.407+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-25T11:39:31.190893+0000 front 2024-04-25T11:39:31.190759+0000 (oldest deadline 2024-04-25T11:39:54.690637+0000) 2024-04-25T11:40:05.611 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:40:05 smithi146 podman[68334]: 2024-04-25 11:40:05.357909144 +0000 UTC m=+5.616624498 container died 17a6e62b0b9b530f0c6888e7d7514bb3c4e00938a4ea4c8b6c4f44365a47e7ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-25T11:40:05.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:05 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:05.378+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:40:05.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:05 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:05.378+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:40:05.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:05 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:05.378+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:40:05.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:05 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:05.378+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:17.188600+0000 front 2024-04-25T11:39:17.188531+0000 (oldest deadline 2024-04-25T11:39:40.088473+0000) 2024-04-25T11:40:05.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:05 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:05.378+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:21.789497+0000 front 2024-04-25T11:39:21.789585+0000 (oldest deadline 2024-04-25T11:39:46.489651+0000) 2024-04-25T11:40:05.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:05 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:05.378+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-25T11:39:31.190893+0000 front 2024-04-25T11:39:31.190759+0000 (oldest deadline 2024-04-25T11:39:54.690637+0000) 2024-04-25T11:40:05.725 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:05 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:05.378+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6820 osd.7 since back 2024-04-25T11:39:41.092503+0000 front 2024-04-25T11:39:41.092596+0000 (oldest deadline 2024-04-25T11:40:05.192269+0000) 2024-04-25T11:40:05.952 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:40:05 smithi146 podman[68334]: 2024-04-25 11:40:05.714908604 +0000 UTC m=+5.973623958 container cleanup 17a6e62b0b9b530f0c6888e7d7514bb3c4e00938a4ea4c8b6c4f44365a47e7ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-25T11:40:05.952 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:40:05 smithi146 bash[68334]: ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10 2024-04-25T11:40:06.310 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:40:05 smithi146 podman[68352]: 2024-04-25 11:40:05.966104568 +0000 UTC m=+0.605498380 container remove 17a6e62b0b9b530f0c6888e7d7514bb3c4e00938a4ea4c8b6c4f44365a47e7ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T11:40:06.593 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:40:06 smithi146 podman[68438]: 2024-04-25 11:40:06.226971752 +0000 UTC m=+0.018370189 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T11:40:06.593 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:40:06 smithi146 podman[68438]: 2024-04-25 11:40:06.325115682 +0000 UTC m=+0.116514127 container create 59a5121e96122e385cdc315423fb10db774ec0c05a660b4fe60d9dd3899e9ead (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-25T11:40:06.593 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:40:06 smithi146 podman[68438]: 2024-04-25 11:40:06.544410705 +0000 UTC m=+0.335809117 container init 59a5121e96122e385cdc315423fb10db774ec0c05a660b4fe60d9dd3899e9ead (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-25T11:40:06.593 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:40:06 smithi146 podman[68438]: 2024-04-25 11:40:06.5473667 +0000 UTC m=+0.338765114 container start 59a5121e96122e385cdc315423fb10db774ec0c05a660b4fe60d9dd3899e9ead (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10-deactivate, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, io.buildah.version=1.33.5) 2024-04-25T11:40:06.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:06 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:06.342+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:40:06.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:06 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:06.342+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:40:06.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:06 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:06.342+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:40:06.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:06 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:06.342+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:17.188600+0000 front 2024-04-25T11:39:17.188531+0000 (oldest deadline 2024-04-25T11:39:40.088473+0000) 2024-04-25T11:40:06.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:06 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:06.342+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:21.789497+0000 front 2024-04-25T11:39:21.789585+0000 (oldest deadline 2024-04-25T11:39:46.489651+0000) 2024-04-25T11:40:06.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:06 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:06.342+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-25T11:39:31.190893+0000 front 2024-04-25T11:39:31.190759+0000 (oldest deadline 2024-04-25T11:39:54.690637+0000) 2024-04-25T11:40:06.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:06 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:06.342+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6820 osd.7 since back 2024-04-25T11:39:41.092503+0000 front 2024-04-25T11:39:41.092596+0000 (oldest deadline 2024-04-25T11:40:05.192269+0000) 2024-04-25T11:40:07.066 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:40:06 smithi146 podman[68438]: 2024-04-25 11:40:06.628898997 +0000 UTC m=+0.420297403 container attach 59a5121e96122e385cdc315423fb10db774ec0c05a660b4fe60d9dd3899e9ead (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T11:40:07.067 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:40:06 smithi146 podman[68438]: 2024-04-25 11:40:06.847055105 +0000 UTC m=+0.638453523 container died 59a5121e96122e385cdc315423fb10db774ec0c05a660b4fe60d9dd3899e9ead (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10-deactivate, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-25T11:40:07.333 INFO:journalctl@ceph.osd.10.smithi146.stdout:Apr 25 11:40:07 smithi146 podman[68465]: 2024-04-25 11:40:07.322480763 +0000 UTC m=+0.469948076 container remove 59a5121e96122e385cdc315423fb10db774ec0c05a660b4fe60d9dd3899e9ead (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-25T11:40:07.338 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@osd.10.service' 2024-04-25T11:40:07.374 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:40:07.374 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-25T11:40:07.375 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-25T11:40:07.375 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@osd.11 2024-04-25T11:40:07.666 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:07 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:07.321+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:40:07.667 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:07 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:07.321+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:40:07.667 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:07 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:07.321+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:40:07.667 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:07 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:07.321+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:17.188600+0000 front 2024-04-25T11:39:17.188531+0000 (oldest deadline 2024-04-25T11:39:40.088473+0000) 2024-04-25T11:40:07.667 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:07 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:07.321+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:21.789497+0000 front 2024-04-25T11:39:21.789585+0000 (oldest deadline 2024-04-25T11:39:46.489651+0000) 2024-04-25T11:40:07.667 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:07 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:07.321+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-25T11:39:31.190893+0000 front 2024-04-25T11:39:31.190759+0000 (oldest deadline 2024-04-25T11:39:54.690637+0000) 2024-04-25T11:40:07.667 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:07 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:07.321+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6820 osd.7 since back 2024-04-25T11:39:41.092503+0000 front 2024-04-25T11:39:41.092596+0000 (oldest deadline 2024-04-25T11:40:05.192269+0000) 2024-04-25T11:40:07.667 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:07 smithi152 systemd[1]: Stopping Ceph osd.11 for f7ed73dc-02f4-11ef-bc93-c7b262605968... 2024-04-25T11:40:07.973 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:07 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:07.679+0000 7f392bfe0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:40:07.974 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:07 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:07.679+0000 7f392bfe0700 -1 osd.11 88 *** Got signal Terminated *** 2024-04-25T11:40:07.974 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:07 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:07.679+0000 7f392bfe0700 -1 osd.11 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T11:40:08.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:08 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:08.347+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:40:08.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:08 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:08.347+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:40:08.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:08 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:08.347+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:40:08.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:08 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:08.347+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:17.188600+0000 front 2024-04-25T11:39:17.188531+0000 (oldest deadline 2024-04-25T11:39:40.088473+0000) 2024-04-25T11:40:08.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:08 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:08.347+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:21.789497+0000 front 2024-04-25T11:39:21.789585+0000 (oldest deadline 2024-04-25T11:39:46.489651+0000) 2024-04-25T11:40:08.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:08 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:08.347+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-25T11:39:31.190893+0000 front 2024-04-25T11:39:31.190759+0000 (oldest deadline 2024-04-25T11:39:54.690637+0000) 2024-04-25T11:40:08.725 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:08 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:08.347+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6820 osd.7 since back 2024-04-25T11:39:41.092503+0000 front 2024-04-25T11:39:41.092596+0000 (oldest deadline 2024-04-25T11:40:05.192269+0000) 2024-04-25T11:40:09.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:09 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:09.367+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:40:09.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:09 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:09.367+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:40:09.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:09 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:09.367+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:40:09.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:09 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:09.367+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:17.188600+0000 front 2024-04-25T11:39:17.188531+0000 (oldest deadline 2024-04-25T11:39:40.088473+0000) 2024-04-25T11:40:09.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:09 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:09.367+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:21.789497+0000 front 2024-04-25T11:39:21.789585+0000 (oldest deadline 2024-04-25T11:39:46.489651+0000) 2024-04-25T11:40:09.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:09 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:09.367+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-25T11:39:31.190893+0000 front 2024-04-25T11:39:31.190759+0000 (oldest deadline 2024-04-25T11:39:54.690637+0000) 2024-04-25T11:40:09.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:09 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:09.367+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6820 osd.7 since back 2024-04-25T11:39:41.092503+0000 front 2024-04-25T11:39:41.092596+0000 (oldest deadline 2024-04-25T11:40:05.192269+0000) 2024-04-25T11:40:10.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:10 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:10.356+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:40:10.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:10 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:10.356+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:40:10.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:10 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:10.356+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:40:10.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:10 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:10.356+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:17.188600+0000 front 2024-04-25T11:39:17.188531+0000 (oldest deadline 2024-04-25T11:39:40.088473+0000) 2024-04-25T11:40:10.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:10 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:10.356+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:21.789497+0000 front 2024-04-25T11:39:21.789585+0000 (oldest deadline 2024-04-25T11:39:46.489651+0000) 2024-04-25T11:40:10.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:10 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:10.356+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-25T11:39:31.190893+0000 front 2024-04-25T11:39:31.190759+0000 (oldest deadline 2024-04-25T11:39:54.690637+0000) 2024-04-25T11:40:10.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:10 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:10.356+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6820 osd.7 since back 2024-04-25T11:39:41.092503+0000 front 2024-04-25T11:39:41.092596+0000 (oldest deadline 2024-04-25T11:40:05.192269+0000) 2024-04-25T11:40:11.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:11 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:11.315+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:40:11.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:11 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:11.315+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:40:11.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:11 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:11.315+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:40:11.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:11 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:11.315+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:17.188600+0000 front 2024-04-25T11:39:17.188531+0000 (oldest deadline 2024-04-25T11:39:40.088473+0000) 2024-04-25T11:40:11.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:11 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:11.315+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:21.789497+0000 front 2024-04-25T11:39:21.789585+0000 (oldest deadline 2024-04-25T11:39:46.489651+0000) 2024-04-25T11:40:11.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:11 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:11.315+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-25T11:39:31.190893+0000 front 2024-04-25T11:39:31.190759+0000 (oldest deadline 2024-04-25T11:39:54.690637+0000) 2024-04-25T11:40:11.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:11 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:11.315+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6820 osd.7 since back 2024-04-25T11:39:41.092503+0000 front 2024-04-25T11:39:41.092596+0000 (oldest deadline 2024-04-25T11:40:05.192269+0000) 2024-04-25T11:40:12.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:12 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:12.331+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-04-25T11:38:40.482427+0000 front 2024-04-25T11:38:40.482679+0000 (oldest deadline 2024-04-25T11:39:08.683005+0000) 2024-04-25T11:40:12.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:12 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:12.331+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-04-25T11:38:50.284785+0000 front 2024-04-25T11:38:50.284757+0000 (oldest deadline 2024-04-25T11:39:15.584930+0000) 2024-04-25T11:40:12.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:12 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:12.331+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-04-25T11:38:59.086033+0000 front 2024-04-25T11:38:59.085884+0000 (oldest deadline 2024-04-25T11:39:23.786100+0000) 2024-04-25T11:40:12.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:12 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:12.331+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-04-25T11:39:17.188600+0000 front 2024-04-25T11:39:17.188531+0000 (oldest deadline 2024-04-25T11:39:40.088473+0000) 2024-04-25T11:40:12.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:12 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:12.331+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-04-25T11:39:21.789497+0000 front 2024-04-25T11:39:21.789585+0000 (oldest deadline 2024-04-25T11:39:46.489651+0000) 2024-04-25T11:40:12.724 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:12 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:12.331+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-04-25T11:39:31.190893+0000 front 2024-04-25T11:39:31.190759+0000 (oldest deadline 2024-04-25T11:39:54.690637+0000) 2024-04-25T11:40:12.725 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:12 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11[58147]: 2024-04-25T11:40:12.331+0000 7f3928596700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.146:6820 osd.7 since back 2024-04-25T11:39:41.092503+0000 front 2024-04-25T11:39:41.092596+0000 (oldest deadline 2024-04-25T11:40:05.192269+0000) 2024-04-25T11:40:13.266 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:12 smithi152 podman[69960]: 2024-04-25 11:40:12.991988635 +0000 UTC m=+5.412694881 container died 7d80ca925b06b8ae37f3299872b530e277ad65984226b81f66544c9a2f455931 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T11:40:13.608 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:13 smithi152 podman[69960]: 2024-04-25 11:40:13.378773634 +0000 UTC m=+5.799479875 container cleanup 7d80ca925b06b8ae37f3299872b530e277ad65984226b81f66544c9a2f455931 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T11:40:13.608 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:13 smithi152 bash[69960]: ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11 2024-04-25T11:40:13.917 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:13 smithi152 podman[69979]: 2024-04-25 11:40:13.621701654 +0000 UTC m=+0.622030318 container remove 7d80ca925b06b8ae37f3299872b530e277ad65984226b81f66544c9a2f455931 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T11:40:14.202 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:13 smithi152 podman[70069]: 2024-04-25 11:40:13.83399103 +0000 UTC m=+0.018018440 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T11:40:14.203 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:13 smithi152 podman[70069]: 2024-04-25 11:40:13.938978238 +0000 UTC m=+0.123005637 container create f6b7c361d70f5ef6d63cef8be5582576fcc3c77b93ee7c39ba73f4190090d93e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T11:40:14.205 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:14 smithi152 podman[70069]: 2024-04-25 11:40:14.150194163 +0000 UTC m=+0.334221572 container init f6b7c361d70f5ef6d63cef8be5582576fcc3c77b93ee7c39ba73f4190090d93e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T11:40:14.205 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:14 smithi152 podman[70069]: 2024-04-25 11:40:14.153523633 +0000 UTC m=+0.337551033 container start f6b7c361d70f5ef6d63cef8be5582576fcc3c77b93ee7c39ba73f4190090d93e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, ceph=True) 2024-04-25T11:40:14.205 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:14 smithi152 podman[70069]: 2024-04-25 11:40:14.216215476 +0000 UTC m=+0.400242889 container attach f6b7c361d70f5ef6d63cef8be5582576fcc3c77b93ee7c39ba73f4190090d93e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T11:40:14.474 INFO:journalctl@ceph.osd.11.smithi152.stdout:Apr 25 11:40:14 smithi152 podman[70069]: 2024-04-25 11:40:14.447196553 +0000 UTC m=+0.631223953 container died f6b7c361d70f5ef6d63cef8be5582576fcc3c77b93ee7c39ba73f4190090d93e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T11:40:15.050 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@osd.11.service' 2024-04-25T11:40:15.088 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:40:15.088 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-25T11:40:15.088 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-25T11:40:15.088 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mds.a 2024-04-25T11:40:15.401 INFO:journalctl@ceph.mds.a.smithi069.stdout:Apr 25 11:40:15 smithi069 systemd[1]: Stopping Ceph mds.a for f7ed73dc-02f4-11ef-bc93-c7b262605968... 2024-04-25T11:40:15.686 INFO:journalctl@ceph.mds.a.smithi069.stdout:Apr 25 11:40:15 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-a[70877]: 2024-04-25T11:40:15.413+0000 7f220edb5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:40:15.686 INFO:journalctl@ceph.mds.a.smithi069.stdout:Apr 25 11:40:15 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-a[70877]: 2024-04-25T11:40:15.413+0000 7f220edb5700 -1 mds.a *** got signal Terminated *** 2024-04-25T11:40:19.686 INFO:journalctl@ceph.mds.a.smithi069.stdout:Apr 25 11:40:19 smithi069 podman[89274]: 2024-04-25 11:40:19.431253125 +0000 UTC m=+4.130204056 container died 4700ba9fb4542b04c7647c680d7ef56151496850689f124d848dcb75a2535af1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T11:40:20.150 INFO:journalctl@ceph.mds.a.smithi069.stdout:Apr 25 11:40:19 smithi069 podman[89274]: 2024-04-25 11:40:19.946591007 +0000 UTC m=+4.645541938 container cleanup 4700ba9fb4542b04c7647c680d7ef56151496850689f124d848dcb75a2535af1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-25T11:40:20.151 INFO:journalctl@ceph.mds.a.smithi069.stdout:Apr 25 11:40:19 smithi069 bash[89274]: ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-a 2024-04-25T11:40:20.400 INFO:journalctl@ceph.mds.a.smithi069.stdout:Apr 25 11:40:20 smithi069 podman[89289]: 2024-04-25 11:40:20.164436572 +0000 UTC m=+0.735127754 container remove 4700ba9fb4542b04c7647c680d7ef56151496850689f124d848dcb75a2535af1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-a, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-25T11:40:20.404 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mds.a.service' 2024-04-25T11:40:20.438 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:40:20.438 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-25T11:40:20.438 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-25T11:40:20.438 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mds.d 2024-04-25T11:40:20.686 INFO:journalctl@ceph.mds.d.smithi069.stdout:Apr 25 11:40:20 smithi069 systemd[1]: Stopping Ceph mds.d for f7ed73dc-02f4-11ef-bc93-c7b262605968... 2024-04-25T11:40:21.186 INFO:journalctl@ceph.mds.d.smithi069.stdout:Apr 25 11:40:20 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-d[69923]: 2024-04-25T11:40:20.800+0000 7fc45aee6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:40:21.186 INFO:journalctl@ceph.mds.d.smithi069.stdout:Apr 25 11:40:20 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-d[69923]: 2024-04-25T11:40:20.800+0000 7fc45aee6700 -1 mds.d *** got signal Terminated *** 2024-04-25T11:40:23.829 INFO:journalctl@ceph.mds.d.smithi069.stdout:Apr 25 11:40:23 smithi069 podman[89396]: 2024-04-25 11:40:23.516786197 +0000 UTC m=+2.853072388 container died 178baf59a50be0381351dea5b29d4fd5ffd281d77e467f6e8973b9a3ee6ea017 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-d, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424) 2024-04-25T11:40:24.129 INFO:journalctl@ceph.mds.d.smithi069.stdout:Apr 25 11:40:23 smithi069 podman[89396]: 2024-04-25 11:40:23.941718284 +0000 UTC m=+3.278004476 container cleanup 178baf59a50be0381351dea5b29d4fd5ffd281d77e467f6e8973b9a3ee6ea017 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T11:40:24.129 INFO:journalctl@ceph.mds.d.smithi069.stdout:Apr 25 11:40:23 smithi069 bash[89396]: ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-d 2024-04-25T11:40:24.365 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mds.d.service' 2024-04-25T11:40:24.385 INFO:journalctl@ceph.mds.d.smithi069.stdout:Apr 25 11:40:24 smithi069 podman[89409]: 2024-04-25 11:40:24.142792109 +0000 UTC m=+0.624398752 container remove 178baf59a50be0381351dea5b29d4fd5ffd281d77e467f6e8973b9a3ee6ea017 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-d, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T11:40:24.385 INFO:journalctl@ceph.mds.d.smithi069.stdout:Apr 25 11:40:24 smithi069 systemd[1]: ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mds.d.service: Deactivated successfully. 2024-04-25T11:40:24.386 INFO:journalctl@ceph.mds.d.smithi069.stdout:Apr 25 11:40:24 smithi069 systemd[1]: Stopped Ceph mds.d for f7ed73dc-02f4-11ef-bc93-c7b262605968. 2024-04-25T11:40:24.386 INFO:journalctl@ceph.mds.d.smithi069.stdout:Apr 25 11:40:24 smithi069 systemd[1]: ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mds.d.service: Consumed 1.105s CPU time. 2024-04-25T11:40:24.396 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:40:24.396 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-25T11:40:24.397 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-25T11:40:24.397 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mds.g 2024-04-25T11:40:24.686 INFO:journalctl@ceph.mds.g.smithi069.stdout:Apr 25 11:40:24 smithi069 systemd[1]: Stopping Ceph mds.g for f7ed73dc-02f4-11ef-bc93-c7b262605968... 2024-04-25T11:40:25.186 INFO:journalctl@ceph.mds.g.smithi069.stdout:Apr 25 11:40:24 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-g[70517]: 2024-04-25T11:40:24.745+0000 7f7ac904c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:40:25.186 INFO:journalctl@ceph.mds.g.smithi069.stdout:Apr 25 11:40:24 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-g[70517]: 2024-04-25T11:40:24.745+0000 7f7ac904c700 -1 mds.g *** got signal Terminated *** 2024-04-25T11:40:28.157 INFO:journalctl@ceph.mds.g.smithi069.stdout:Apr 25 11:40:27 smithi069 podman[89517]: 2024-04-25 11:40:27.839337271 +0000 UTC m=+3.216124097 container died bf0ac5d14701c33778f0c8a19479d9eeec80c1d48bd3c69ce2875d652f85172f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-g, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T11:40:28.432 INFO:journalctl@ceph.mds.g.smithi069.stdout:Apr 25 11:40:28 smithi069 podman[89517]: 2024-04-25 11:40:28.253763515 +0000 UTC m=+3.630550341 container cleanup bf0ac5d14701c33778f0c8a19479d9eeec80c1d48bd3c69ce2875d652f85172f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-g, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-25T11:40:28.433 INFO:journalctl@ceph.mds.g.smithi069.stdout:Apr 25 11:40:28 smithi069 bash[89517]: ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-g 2024-04-25T11:40:28.673 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mds.g.service' 2024-04-25T11:40:28.686 INFO:journalctl@ceph.mds.g.smithi069.stdout:Apr 25 11:40:28 smithi069 podman[89532]: 2024-04-25 11:40:28.446487823 +0000 UTC m=+0.606741691 container remove bf0ac5d14701c33778f0c8a19479d9eeec80c1d48bd3c69ce2875d652f85172f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-g, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-25T11:40:28.686 INFO:journalctl@ceph.mds.g.smithi069.stdout:Apr 25 11:40:28 smithi069 systemd[1]: ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mds.g.service: Deactivated successfully. 2024-04-25T11:40:28.686 INFO:journalctl@ceph.mds.g.smithi069.stdout:Apr 25 11:40:28 smithi069 systemd[1]: Stopped Ceph mds.g for f7ed73dc-02f4-11ef-bc93-c7b262605968. 2024-04-25T11:40:28.686 INFO:journalctl@ceph.mds.g.smithi069.stdout:Apr 25 11:40:28 smithi069 systemd[1]: ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mds.g.service: Consumed 2.286s CPU time. 2024-04-25T11:40:28.703 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:40:28.703 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-25T11:40:28.703 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-25T11:40:28.703 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mds.j 2024-04-25T11:40:29.027 INFO:journalctl@ceph.mds.j.smithi069.stdout:Apr 25 11:40:28 smithi069 systemd[1]: Stopping Ceph mds.j for f7ed73dc-02f4-11ef-bc93-c7b262605968... 2024-04-25T11:40:29.436 INFO:journalctl@ceph.mds.j.smithi069.stdout:Apr 25 11:40:29 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-j[69538]: 2024-04-25T11:40:29.040+0000 7fd87c972700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:40:29.436 INFO:journalctl@ceph.mds.j.smithi069.stdout:Apr 25 11:40:29 smithi069 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-j[69538]: 2024-04-25T11:40:29.040+0000 7fd87c972700 -1 mds.j *** got signal Terminated *** 2024-04-25T11:40:32.936 INFO:journalctl@ceph.mds.j.smithi069.stdout:Apr 25 11:40:32 smithi069 podman[89638]: 2024-04-25 11:40:32.607019083 +0000 UTC m=+3.672347326 container died 755177b434f5e856109567d6b2acb9dd02a337c13fb212098ec14d5445476324 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True) 2024-04-25T11:40:33.194 INFO:journalctl@ceph.mds.j.smithi069.stdout:Apr 25 11:40:33 smithi069 podman[89638]: 2024-04-25 11:40:33.048658786 +0000 UTC m=+4.113987020 container cleanup 755177b434f5e856109567d6b2acb9dd02a337c13fb212098ec14d5445476324 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-25T11:40:33.194 INFO:journalctl@ceph.mds.j.smithi069.stdout:Apr 25 11:40:33 smithi069 bash[89638]: ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-j 2024-04-25T11:40:33.412 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mds.j.service' 2024-04-25T11:40:33.442 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:40:33.442 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-25T11:40:33.443 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-25T11:40:33.443 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mds.b 2024-04-25T11:40:33.721 INFO:journalctl@ceph.mds.b.smithi146.stdout:Apr 25 11:40:33 smithi146 systemd[1]: Stopping Ceph mds.b for f7ed73dc-02f4-11ef-bc93-c7b262605968... 2024-04-25T11:40:34.092 INFO:journalctl@ceph.mds.b.smithi146.stdout:Apr 25 11:40:33 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-b[62892]: 2024-04-25T11:40:33.781+0000 7f09d68c8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:40:34.092 INFO:journalctl@ceph.mds.b.smithi146.stdout:Apr 25 11:40:33 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-b[62892]: 2024-04-25T11:40:33.781+0000 7f09d68c8700 -1 mds.b *** got signal Terminated *** 2024-04-25T11:40:36.092 INFO:journalctl@ceph.mds.b.smithi146.stdout:Apr 25 11:40:35 smithi146 podman[68545]: 2024-04-25 11:40:35.842459963 +0000 UTC m=+2.194447228 container died 26a675f1aaf4783579a8997fad18a4002a69e90fc6dfe997eca2184b5ba356d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-25T11:40:36.451 INFO:journalctl@ceph.mds.b.smithi146.stdout:Apr 25 11:40:36 smithi146 podman[68545]: 2024-04-25 11:40:36.189181922 +0000 UTC m=+2.541169196 container cleanup 26a675f1aaf4783579a8997fad18a4002a69e90fc6dfe997eca2184b5ba356d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-b, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T11:40:36.452 INFO:journalctl@ceph.mds.b.smithi146.stdout:Apr 25 11:40:36 smithi146 bash[68545]: ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-b 2024-04-25T11:40:36.598 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mds.b.service' 2024-04-25T11:40:36.632 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:40:36.632 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-25T11:40:36.632 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-25T11:40:36.632 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mds.e 2024-04-25T11:40:36.963 INFO:journalctl@ceph.mds.e.smithi146.stdout:Apr 25 11:40:36 smithi146 systemd[1]: Stopping Ceph mds.e for f7ed73dc-02f4-11ef-bc93-c7b262605968... 2024-04-25T11:40:37.342 INFO:journalctl@ceph.mds.e.smithi146.stdout:Apr 25 11:40:36 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-e[62273]: 2024-04-25T11:40:36.976+0000 7f21e3fae700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:40:37.342 INFO:journalctl@ceph.mds.e.smithi146.stdout:Apr 25 11:40:36 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-e[62273]: 2024-04-25T11:40:36.976+0000 7f21e3fae700 -1 mds.e *** got signal Terminated *** 2024-04-25T11:40:40.092 INFO:journalctl@ceph.mds.e.smithi146.stdout:Apr 25 11:40:39 smithi146 podman[68677]: 2024-04-25 11:40:39.855821434 +0000 UTC m=+2.992058023 container died 6f4c14e69976442102f92ca019018e7da4a42aaa0126c0f4c0f1993ea64233d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-e, org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-25T11:40:40.472 INFO:journalctl@ceph.mds.e.smithi146.stdout:Apr 25 11:40:40 smithi146 podman[68677]: 2024-04-25 11:40:40.232006865 +0000 UTC m=+3.368243464 container cleanup 6f4c14e69976442102f92ca019018e7da4a42aaa0126c0f4c0f1993ea64233d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-e, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True) 2024-04-25T11:40:40.472 INFO:journalctl@ceph.mds.e.smithi146.stdout:Apr 25 11:40:40 smithi146 bash[68677]: ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-e 2024-04-25T11:40:40.604 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mds.e.service' 2024-04-25T11:40:40.636 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:40:40.636 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-25T11:40:40.636 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-25T11:40:40.636 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mds.h 2024-04-25T11:40:40.842 INFO:journalctl@ceph.mds.h.smithi146.stdout:Apr 25 11:40:40 smithi146 systemd[1]: Stopping Ceph mds.h for f7ed73dc-02f4-11ef-bc93-c7b262605968... 2024-04-25T11:40:41.342 INFO:journalctl@ceph.mds.h.smithi146.stdout:Apr 25 11:40:40 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-h[62580]: 2024-04-25T11:40:40.963+0000 7efff3b37700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:40:41.342 INFO:journalctl@ceph.mds.h.smithi146.stdout:Apr 25 11:40:40 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-h[62580]: 2024-04-25T11:40:40.963+0000 7efff3b37700 -1 mds.h *** got signal Terminated *** 2024-04-25T11:40:44.092 INFO:journalctl@ceph.mds.h.smithi146.stdout:Apr 25 11:40:43 smithi146 podman[68798]: 2024-04-25 11:40:43.853556778 +0000 UTC m=+2.980460709 container died e2ddf7bbd96375b6f8dfdd19fb9df75158c8f510f8528f68a1812911b16d5357 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-h, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T11:40:44.350 INFO:journalctl@ceph.mds.h.smithi146.stdout:Apr 25 11:40:44 smithi146 podman[68798]: 2024-04-25 11:40:44.196665417 +0000 UTC m=+3.323569389 container cleanup e2ddf7bbd96375b6f8dfdd19fb9df75158c8f510f8528f68a1812911b16d5357 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-h, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424) 2024-04-25T11:40:44.350 INFO:journalctl@ceph.mds.h.smithi146.stdout:Apr 25 11:40:44 smithi146 bash[68798]: ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-h 2024-04-25T11:40:44.561 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mds.h.service' 2024-04-25T11:40:44.592 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:40:44.593 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-25T11:40:44.593 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-25T11:40:44.593 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mds.k 2024-04-25T11:40:44.920 INFO:journalctl@ceph.mds.k.smithi146.stdout:Apr 25 11:40:44 smithi146 systemd[1]: Stopping Ceph mds.k for f7ed73dc-02f4-11ef-bc93-c7b262605968... 2024-04-25T11:40:45.342 INFO:journalctl@ceph.mds.k.smithi146.stdout:Apr 25 11:40:44 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-k[61952]: 2024-04-25T11:40:44.932+0000 7fbbf20dc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:40:45.342 INFO:journalctl@ceph.mds.k.smithi146.stdout:Apr 25 11:40:44 smithi146 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-k[61952]: 2024-04-25T11:40:44.932+0000 7fbbf20dc700 -1 mds.k *** got signal Terminated *** 2024-04-25T11:40:48.092 INFO:journalctl@ceph.mds.k.smithi146.stdout:Apr 25 11:40:47 smithi146 podman[68917]: 2024-04-25 11:40:47.845171242 +0000 UTC m=+3.018543532 container died 115cfd316cf7007d5d8afe65351a5983e51320ad640ca5f1760be725594a23ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-k, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T11:40:48.362 INFO:journalctl@ceph.mds.k.smithi146.stdout:Apr 25 11:40:48 smithi146 podman[68917]: 2024-04-25 11:40:48.183547515 +0000 UTC m=+3.356919806 container cleanup 115cfd316cf7007d5d8afe65351a5983e51320ad640ca5f1760be725594a23ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-k, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-25T11:40:48.362 INFO:journalctl@ceph.mds.k.smithi146.stdout:Apr 25 11:40:48 smithi146 bash[68917]: ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-k 2024-04-25T11:40:48.499 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mds.k.service' 2024-04-25T11:40:48.530 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:40:48.530 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-25T11:40:48.531 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-25T11:40:48.531 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mds.c 2024-04-25T11:40:48.808 INFO:journalctl@ceph.mds.c.smithi152.stdout:Apr 25 11:40:48 smithi152 systemd[1]: Stopping Ceph mds.c for f7ed73dc-02f4-11ef-bc93-c7b262605968... 2024-04-25T11:40:49.224 INFO:journalctl@ceph.mds.c.smithi152.stdout:Apr 25 11:40:48 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-c[63595]: 2024-04-25T11:40:48.843+0000 7f674425e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:40:49.224 INFO:journalctl@ceph.mds.c.smithi152.stdout:Apr 25 11:40:48 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-c[63595]: 2024-04-25T11:40:48.843+0000 7f674425e700 -1 mds.c *** got signal Terminated *** 2024-04-25T11:40:52.172 INFO:journalctl@ceph.mds.c.smithi152.stdout:Apr 25 11:40:51 smithi152 podman[70175]: 2024-04-25 11:40:51.829046143 +0000 UTC m=+3.097856680 container stop f812ac2e4ae3797c8a5876c893af557326c18547d70f6f9cce7c70b95104138b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-c, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-25T11:40:52.172 INFO:journalctl@ceph.mds.c.smithi152.stdout:Apr 25 11:40:51 smithi152 podman[70175]: 2024-04-25 11:40:51.859020176 +0000 UTC m=+3.127830723 container died f812ac2e4ae3797c8a5876c893af557326c18547d70f6f9cce7c70b95104138b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-c, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-25T11:40:52.438 INFO:journalctl@ceph.mds.c.smithi152.stdout:Apr 25 11:40:52 smithi152 podman[70175]: 2024-04-25 11:40:52.267520122 +0000 UTC m=+3.536330655 container cleanup f812ac2e4ae3797c8a5876c893af557326c18547d70f6f9cce7c70b95104138b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-c, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T11:40:52.438 INFO:journalctl@ceph.mds.c.smithi152.stdout:Apr 25 11:40:52 smithi152 bash[70175]: ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-c 2024-04-25T11:40:52.673 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mds.c.service' 2024-04-25T11:40:52.694 INFO:journalctl@ceph.mds.c.smithi152.stdout:Apr 25 11:40:52 smithi152 podman[70201]: 2024-04-25 11:40:52.452228549 +0000 UTC m=+0.615782117 container remove f812ac2e4ae3797c8a5876c893af557326c18547d70f6f9cce7c70b95104138b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-c, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-25T11:40:52.694 INFO:journalctl@ceph.mds.c.smithi152.stdout:Apr 25 11:40:52 smithi152 systemd[1]: ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mds.c.service: Deactivated successfully. 2024-04-25T11:40:52.695 INFO:journalctl@ceph.mds.c.smithi152.stdout:Apr 25 11:40:52 smithi152 systemd[1]: Stopped Ceph mds.c for f7ed73dc-02f4-11ef-bc93-c7b262605968. 2024-04-25T11:40:52.695 INFO:journalctl@ceph.mds.c.smithi152.stdout:Apr 25 11:40:52 smithi152 systemd[1]: ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mds.c.service: Consumed 3.191s CPU time. 2024-04-25T11:40:52.708 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:40:52.708 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-25T11:40:52.708 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-25T11:40:52.708 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mds.f 2024-04-25T11:40:52.973 INFO:journalctl@ceph.mds.f.smithi152.stdout:Apr 25 11:40:52 smithi152 systemd[1]: Stopping Ceph mds.f for f7ed73dc-02f4-11ef-bc93-c7b262605968... 2024-04-25T11:40:53.473 INFO:journalctl@ceph.mds.f.smithi152.stdout:Apr 25 11:40:53 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-f[63285]: 2024-04-25T11:40:53.037+0000 7ff1339b2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:40:53.474 INFO:journalctl@ceph.mds.f.smithi152.stdout:Apr 25 11:40:53 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-f[63285]: 2024-04-25T11:40:53.037+0000 7ff1339b2700 -1 mds.f *** got signal Terminated *** 2024-04-25T11:40:56.091 INFO:journalctl@ceph.mds.f.smithi152.stdout:Apr 25 11:40:55 smithi152 podman[70309]: 2024-04-25 11:40:55.844416363 +0000 UTC m=+2.910175556 container died 840d6a5b4e77670b00b477263121551be0ad311e6013a88363b5ea71c6be2deb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True) 2024-04-25T11:40:56.433 INFO:journalctl@ceph.mds.f.smithi152.stdout:Apr 25 11:40:56 smithi152 podman[70309]: 2024-04-25 11:40:56.204812078 +0000 UTC m=+3.270571280 container cleanup 840d6a5b4e77670b00b477263121551be0ad311e6013a88363b5ea71c6be2deb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-f, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-25T11:40:56.433 INFO:journalctl@ceph.mds.f.smithi152.stdout:Apr 25 11:40:56 smithi152 bash[70309]: ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-f 2024-04-25T11:40:56.570 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mds.f.service' 2024-04-25T11:40:56.603 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:40:56.604 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-25T11:40:56.604 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-25T11:40:56.604 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mds.i 2024-04-25T11:40:56.974 INFO:journalctl@ceph.mds.i.smithi152.stdout:Apr 25 11:40:56 smithi152 systemd[1]: Stopping Ceph mds.i for f7ed73dc-02f4-11ef-bc93-c7b262605968... 2024-04-25T11:40:56.974 INFO:journalctl@ceph.mds.i.smithi152.stdout:Apr 25 11:40:56 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-i[63906]: 2024-04-25T11:40:56.940+0000 7fdd6d5cb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:40:56.974 INFO:journalctl@ceph.mds.i.smithi152.stdout:Apr 25 11:40:56 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-i[63906]: 2024-04-25T11:40:56.940+0000 7fdd6d5cb700 -1 mds.i *** got signal Terminated *** 2024-04-25T11:41:00.094 INFO:journalctl@ceph.mds.i.smithi152.stdout:Apr 25 11:40:59 smithi152 podman[70429]: 2024-04-25 11:40:59.84193001 +0000 UTC m=+3.006332704 container died 9c74b6b22d60ac474b098fcf8899c5ef7f34cff878a93f2d3349b068063447e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-i, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-25T11:41:00.445 INFO:journalctl@ceph.mds.i.smithi152.stdout:Apr 25 11:41:00 smithi152 podman[70429]: 2024-04-25 11:41:00.199269241 +0000 UTC m=+3.363671924 container cleanup 9c74b6b22d60ac474b098fcf8899c5ef7f34cff878a93f2d3349b068063447e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-i, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-25T11:41:00.445 INFO:journalctl@ceph.mds.i.smithi152.stdout:Apr 25 11:41:00 smithi152 bash[70429]: ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-i 2024-04-25T11:41:00.603 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mds.i.service' 2024-04-25T11:41:00.634 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:41:00.634 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-25T11:41:00.634 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-25T11:41:00.635 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mds.l 2024-04-25T11:41:00.973 INFO:journalctl@ceph.mds.l.smithi152.stdout:Apr 25 11:41:00 smithi152 systemd[1]: Stopping Ceph mds.l for f7ed73dc-02f4-11ef-bc93-c7b262605968... 2024-04-25T11:41:00.974 INFO:journalctl@ceph.mds.l.smithi152.stdout:Apr 25 11:41:00 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-l[62974]: 2024-04-25T11:41:00.952+0000 7fee6ecee700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:41:00.974 INFO:journalctl@ceph.mds.l.smithi152.stdout:Apr 25 11:41:00 smithi152 ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-l[62974]: 2024-04-25T11:41:00.952+0000 7fee6ecee700 -1 mds.l *** got signal Terminated *** 2024-04-25T11:41:04.082 INFO:journalctl@ceph.mds.l.smithi152.stdout:Apr 25 11:41:03 smithi152 podman[70551]: 2024-04-25 11:41:03.835440298 +0000 UTC m=+2.964852481 container died 98e10171bbd1d7e0e9d7f360e1e39d5265c7556f2eece45bd7cb5b2f3a31b2a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-l, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-25T11:41:04.415 INFO:journalctl@ceph.mds.l.smithi152.stdout:Apr 25 11:41:04 smithi152 podman[70551]: 2024-04-25 11:41:04.19531998 +0000 UTC m=+3.324732161 container cleanup 98e10171bbd1d7e0e9d7f360e1e39d5265c7556f2eece45bd7cb5b2f3a31b2a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-l, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T11:41:04.415 INFO:journalctl@ceph.mds.l.smithi152.stdout:Apr 25 11:41:04 smithi152 bash[70551]: ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968-mds-l 2024-04-25T11:41:04.556 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f7ed73dc-02f4-11ef-bc93-c7b262605968@mds.l.service' 2024-04-25T11:41:04.586 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:41:04.586 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-25T11:41:04.586 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T11:41:04.776 INFO:teuthology.orchestra.run.smithi069.stdout:Deleting cluster with fsid: f7ed73dc-02f4-11ef-bc93-c7b262605968 2024-04-25T11:41:07.249 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T11:41:07.440 INFO:teuthology.orchestra.run.smithi146.stdout:Deleting cluster with fsid: f7ed73dc-02f4-11ef-bc93-c7b262605968 2024-04-25T11:41:09.889 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T11:41:10.078 INFO:teuthology.orchestra.run.smithi152.stdout:Deleting cluster with fsid: f7ed73dc-02f4-11ef-bc93-c7b262605968 2024-04-25T11:41:12.364 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T11:41:12.395 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T11:41:12.427 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T11:41:12.453 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-25T11:41:12.454 DEBUG:teuthology.misc:Transferring archived files from smithi069:/var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673128/remote/smithi069/crash 2024-04-25T11:41:12.455 DEBUG:teuthology.orchestra.run.smithi069:> sudo tar c -f - -C /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/crash -- . 2024-04-25T11:41:12.489 INFO:teuthology.orchestra.run.smithi069.stderr:tar: /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T11:41:12.490 INFO:teuthology.orchestra.run.smithi069.stderr:tar: Error is not recoverable: exiting now 2024-04-25T11:41:12.492 DEBUG:teuthology.misc:Transferring archived files from smithi146:/var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673128/remote/smithi146/crash 2024-04-25T11:41:12.493 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar c -f - -C /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/crash -- . 2024-04-25T11:41:12.616 INFO:teuthology.orchestra.run.smithi146.stderr:tar: /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T11:41:12.616 INFO:teuthology.orchestra.run.smithi146.stderr:tar: Error is not recoverable: exiting now 2024-04-25T11:41:12.618 DEBUG:teuthology.misc:Transferring archived files from smithi152:/var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673128/remote/smithi152/crash 2024-04-25T11:41:12.619 DEBUG:teuthology.orchestra.run.smithi152:> sudo tar c -f - -C /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/crash -- . 2024-04-25T11:41:12.662 INFO:teuthology.orchestra.run.smithi152.stderr:tar: /var/lib/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T11:41:12.662 INFO:teuthology.orchestra.run.smithi152.stderr:tar: Error is not recoverable: exiting now 2024-04-25T11:41:12.663 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-25T11:41:12.664 DEBUG:teuthology.orchestra.run.smithi069:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-25T11:41:12.699 INFO:tasks.cephadm:Compressing logs... 2024-04-25T11:41:12.700 DEBUG:teuthology.orchestra.run.smithi069:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T11:41:12.742 DEBUG:teuthology.orchestra.run.smithi146:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T11:41:12.745 DEBUG:teuthology.orchestra.run.smithi152:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T11:41:12.767 INFO:teuthology.orchestra.run.smithi069.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-04-25T11:41:12.767 INFO:teuthology.orchestra.run.smithi069.stderr:: No such file or directory 2024-04-25T11:41:12.768 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph.log 2024-04-25T11:41:12.768 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T11:41:12.768 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-25T11:41:12.768 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-25T11:41:12.768 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-25T11:41:12.775 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-25T11:41:12.775 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-25T11:41:12.775 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-25T11:41:12.775 INFO:teuthology.orchestra.run.smithi069.stderr: 87.9% -- replaced with /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T11:41:12.776 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-25T11:41:12.776 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mon.a.log: 90.3% -- replaced with /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T11:41:12.776 INFO:teuthology.orchestra.run.smithi069.stderr: 94.7% -- replaced with /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-25T11:41:12.780 INFO:teuthology.orchestra.run.smithi146.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph.log 2024-04-25T11:41:12.780 INFO:teuthology.orchestra.run.smithi146.stderr:: No such file or directory 2024-04-25T11:41:12.780 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T11:41:12.780 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-25T11:41:12.780 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-25T11:41:12.781 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-25T11:41:12.781 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-25T11:41:12.781 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-25T11:41:12.781 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-25T11:41:12.781 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mgr.y.log: 87.9%gzip -5 --verbose -- /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-25T11:41:12.781 INFO:teuthology.orchestra.run.smithi146.stderr: -- replaced with /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T11:41:12.781 INFO:teuthology.orchestra.run.smithi146.stderr: 90.4% -- replaced with /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T11:41:12.781 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mds.h.log: 93.0% -- replaced with /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-25T11:41:12.782 INFO:teuthology.orchestra.run.smithi152.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory 2024-04-25T11:41:12.782 INFO:teuthology.orchestra.run.smithi152.stderr: /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph.log 2024-04-25T11:41:12.783 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-25T11:41:12.783 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-25T11:41:12.783 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T11:41:12.783 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-25T11:41:12.783 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-25T11:41:12.783 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-25T11:41:12.783 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-25T11:41:12.783 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-25T11:41:12.783 INFO:teuthology.orchestra.run.smithi152.stderr: 87.9% -- replaced with /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T11:41:12.783 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T11:41:12.783 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mds.i.log: 90.4% -- replaced with /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T11:41:12.783 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-25T11:41:12.783 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph.cephadm.log: 82.6% -- replaced with /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T11:41:12.784 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-25T11:41:12.784 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-osd.11.log: 93.0% -- replaced with /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-25T11:41:12.784 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T11:41:12.785 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T11:41:12.785 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-25T11:41:12.785 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.0% -- replaced with /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T11:41:12.787 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T11:41:12.787 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-25T11:41:12.787 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T11:41:12.787 INFO:teuthology.orchestra.run.smithi146.stderr: 82.7% -- replaced with /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T11:41:12.790 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-25T11:41:12.796 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T11:41:12.798 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T11:41:12.799 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-25T11:41:12.808 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-25T11:41:12.811 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-25T11:41:12.813 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T11:41:12.814 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T11:41:12.817 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-25T11:41:12.818 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mds.l.log: 91.0% -- replaced with /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T11:41:12.820 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/cephadm.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T11:41:12.828 INFO:teuthology.orchestra.run.smithi152.stderr: 93.6% -- replaced with /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-25T11:41:12.829 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mds.d.log: 94.6%gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T11:41:12.829 INFO:teuthology.orchestra.run.smithi069.stderr: -- replaced with /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-25T11:41:12.831 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T11:41:12.835 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-04-25T11:41:12.836 INFO:teuthology.orchestra.run.smithi146.stderr: 93.4% -- replaced with /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-25T11:41:12.837 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/cephadm.log: 91.1% -- replaced with /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T11:41:12.837 INFO:teuthology.orchestra.run.smithi069.stderr: 94.5% -- replaced with /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-25T11:41:12.838 INFO:teuthology.orchestra.run.smithi152.stderr: 94.7% -- replaced with /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-25T11:41:12.839 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/ceph-client.admin.log: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-04-25T11:41:12.848 INFO:teuthology.orchestra.run.smithi069.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T11:41:12.857 INFO:teuthology.orchestra.run.smithi069.stderr: 93.4% -- replaced with /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-25T11:41:12.867 INFO:teuthology.orchestra.run.smithi146.stderr: 93.4% -- replaced with /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-25T11:41:12.878 INFO:teuthology.orchestra.run.smithi152.stderr: 94.5% -- replaced with /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-25T11:41:12.888 INFO:teuthology.orchestra.run.smithi146.stderr: 94.5% -- replaced with /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-25T11:41:12.976 INFO:teuthology.orchestra.run.smithi152.stderr: 94.7% -- replaced with /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-25T11:41:13.047 INFO:teuthology.orchestra.run.smithi152.stderr: 92.1% -- replaced with /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-25T11:41:13.071 INFO:teuthology.orchestra.run.smithi069.stderr: 90.4% -- replaced with /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-25T11:41:13.141 INFO:teuthology.orchestra.run.smithi069.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-04-25T11:41:13.209 INFO:teuthology.orchestra.run.smithi146.stderr: 92.1% -- replaced with /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-25T11:41:13.524 INFO:teuthology.orchestra.run.smithi069.stderr: 92.4% -- replaced with /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-25T11:41:13.802 INFO:teuthology.orchestra.run.smithi146.stderr: 92.0% -- replaced with /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-25T11:41:13.847 INFO:teuthology.orchestra.run.smithi069.stderr: 90.5% -- replaced with /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-25T11:41:13.906 INFO:teuthology.orchestra.run.smithi146.stderr: 92.4% -- replaced with /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-25T11:41:14.044 INFO:teuthology.orchestra.run.smithi152.stderr: 93.1% -- replaced with /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-25T11:41:14.138 INFO:teuthology.orchestra.run.smithi069.stderr: 93.1% -- replaced with /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-25T11:41:14.141 INFO:teuthology.orchestra.run.smithi069.stderr: 93.2% -- replaced with /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-25T11:41:14.166 INFO:teuthology.orchestra.run.smithi069.stderr: 93.4% -- replaced with /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-25T11:41:14.168 INFO:teuthology.orchestra.run.smithi069.stderr: 2024-04-25T11:41:14.169 INFO:teuthology.orchestra.run.smithi069.stderr:real 0m1.413s 2024-04-25T11:41:14.169 INFO:teuthology.orchestra.run.smithi069.stderr:user 0m6.483s 2024-04-25T11:41:14.169 INFO:teuthology.orchestra.run.smithi069.stderr:sys 0m0.240s 2024-04-25T11:41:14.310 INFO:teuthology.orchestra.run.smithi146.stderr: 91.6% -- replaced with /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-25T11:41:14.348 INFO:teuthology.orchestra.run.smithi146.stderr: 91.6% -- replaced with /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-25T11:41:14.477 INFO:teuthology.orchestra.run.smithi152.stderr: 91.5% -- replaced with /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-25T11:41:14.559 INFO:teuthology.orchestra.run.smithi152.stderr: 92.6% -- replaced with /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-25T11:41:14.640 INFO:teuthology.orchestra.run.smithi146.stderr: 92.9% -- replaced with /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-25T11:41:14.642 INFO:teuthology.orchestra.run.smithi146.stderr: 2024-04-25T11:41:14.642 INFO:teuthology.orchestra.run.smithi146.stderr:real 0m1.883s 2024-04-25T11:41:14.642 INFO:teuthology.orchestra.run.smithi146.stderr:user 0m7.507s 2024-04-25T11:41:14.642 INFO:teuthology.orchestra.run.smithi146.stderr:sys 0m0.264s 2024-04-25T11:41:15.162 INFO:teuthology.orchestra.run.smithi152.stderr: 91.2% -- replaced with /var/log/ceph/f7ed73dc-02f4-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-25T11:41:15.165 INFO:teuthology.orchestra.run.smithi152.stderr: 2024-04-25T11:41:15.165 INFO:teuthology.orchestra.run.smithi152.stderr:real 0m2.406s 2024-04-25T11:41:15.165 INFO:teuthology.orchestra.run.smithi152.stderr:user 0m7.585s 2024-04-25T11:41:15.165 INFO:teuthology.orchestra.run.smithi152.stderr:sys 0m0.252s 2024-04-25T11:41:15.165 INFO:tasks.cephadm:Archiving logs... 2024-04-25T11:41:15.165 DEBUG:teuthology.misc:Transferring archived files from smithi069:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673128/remote/smithi069/log 2024-04-25T11:41:15.167 DEBUG:teuthology.orchestra.run.smithi069:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T11:41:16.096 DEBUG:teuthology.misc:Transferring archived files from smithi146:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673128/remote/smithi146/log 2024-04-25T11:41:16.097 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T11:41:17.241 DEBUG:teuthology.misc:Transferring archived files from smithi152:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673128/remote/smithi152/log 2024-04-25T11:41:17.242 DEBUG:teuthology.orchestra.run.smithi152:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T11:41:18.554 INFO:tasks.cephadm:Removing cluster... 2024-04-25T11:41:18.555 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 --force 2024-04-25T11:41:18.744 INFO:teuthology.orchestra.run.smithi069.stdout:Deleting cluster with fsid: f7ed73dc-02f4-11ef-bc93-c7b262605968 2024-04-25T11:41:19.004 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 --force 2024-04-25T11:41:19.185 INFO:teuthology.orchestra.run.smithi146.stdout:Deleting cluster with fsid: f7ed73dc-02f4-11ef-bc93-c7b262605968 2024-04-25T11:41:19.450 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f7ed73dc-02f4-11ef-bc93-c7b262605968 --force 2024-04-25T11:41:19.640 INFO:teuthology.orchestra.run.smithi152.stdout:Deleting cluster with fsid: f7ed73dc-02f4-11ef-bc93-c7b262605968 2024-04-25T11:41:19.916 INFO:tasks.cephadm:Removing cephadm ... 2024-04-25T11:41:19.916 DEBUG:teuthology.orchestra.run.smithi069:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T11:41:19.935 DEBUG:teuthology.orchestra.run.smithi146:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T11:41:19.950 DEBUG:teuthology.orchestra.run.smithi152:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T11:41:19.965 INFO:tasks.cephadm:Teardown complete 2024-04-25T11:41:19.965 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T11:41:19.976 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T11:41:19.976 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T11:41:19.979 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T11:41:19.993 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T11:41:20.105 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T11:41:20.115 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T11:41:20.115 DEBUG:teuthology.orchestra.run.smithi069:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T11:41:20.117 DEBUG:teuthology.orchestra.run.smithi146:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T11:41:20.119 DEBUG:teuthology.orchestra.run.smithi152:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T11:41:20.131 INFO:teuthology.orchestra.run.smithi146.stderr:bash: line 1: ntpq: command not found 2024-04-25T11:41:20.131 INFO:teuthology.orchestra.run.smithi069.stderr:bash: line 1: ntpq: command not found 2024-04-25T11:41:20.133 INFO:teuthology.orchestra.run.smithi152.stderr:bash: line 1: ntpq: command not found 2024-04-25T11:41:20.136 INFO:teuthology.orchestra.run.smithi146.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T11:41:20.136 INFO:teuthology.orchestra.run.smithi146.stdout:=============================================================================== 2024-04-25T11:41:20.136 INFO:teuthology.orchestra.run.smithi146.stdout:^* hv01.front.sepia.ceph.com 3 6 377 33 +671us[ +657us] +/- 40ms 2024-04-25T11:41:20.136 INFO:teuthology.orchestra.run.smithi146.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 32 -553us[ -553us] +/- 80ms 2024-04-25T11:41:20.136 INFO:teuthology.orchestra.run.smithi146.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 31 -659us[ -659us] +/- 79ms 2024-04-25T11:41:20.136 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:41:20.137 INFO:teuthology.orchestra.run.smithi069.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T11:41:20.137 INFO:teuthology.orchestra.run.smithi069.stdout:=============================================================================== 2024-04-25T11:41:20.138 INFO:teuthology.orchestra.run.smithi069.stdout:^* hv01.front.sepia.ceph.com 3 6 377 35 +649us[ +629us] +/- 40ms 2024-04-25T11:41:20.138 INFO:teuthology.orchestra.run.smithi069.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 36 -560us[ -580us] +/- 80ms 2024-04-25T11:41:20.138 INFO:teuthology.orchestra.run.smithi069.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 34 -680us[ -680us] +/- 79ms 2024-04-25T11:41:20.138 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:41:20.139 INFO:teuthology.orchestra.run.smithi152.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T11:41:20.139 INFO:teuthology.orchestra.run.smithi152.stdout:=============================================================================== 2024-04-25T11:41:20.139 INFO:teuthology.orchestra.run.smithi152.stdout:^* hv01.front.sepia.ceph.com 3 6 377 33 +668us[ +671us] +/- 40ms 2024-04-25T11:41:20.139 INFO:teuthology.orchestra.run.smithi152.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 34 -554us[ -552us] +/- 80ms 2024-04-25T11:41:20.139 INFO:teuthology.orchestra.run.smithi152.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 36 -670us[ -667us] +/- 79ms 2024-04-25T11:41:20.139 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:41:20.139 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T11:41:20.150 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T11:41:20.150 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T11:41:20.160 DEBUG:teuthology.orchestra.run.smithi069:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T11:41:20.257 DEBUG:teuthology.orchestra.run.smithi146:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T11:41:20.320 DEBUG:teuthology.orchestra.run.smithi152:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T11:41:20.383 DEBUG:teuthology.orchestra.run.smithi069:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T11:41:20.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:41:20.412 DEBUG:teuthology.orchestra.run.smithi146:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T11:41:20.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:41:20.439 DEBUG:teuthology.orchestra.run.smithi152:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T11:41:20.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:41:20.467 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T11:41:20.477 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T11:41:20.487 INFO:teuthology.task.internal:Duration was 1942.959715 seconds 2024-04-25T11:41:20.488 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T11:41:20.498 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T11:41:20.498 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T11:41:20.501 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T11:41:20.503 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T11:41:20.537 INFO:teuthology.orchestra.run.smithi146.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T11:41:20.537 INFO:teuthology.orchestra.run.smithi069.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T11:41:20.543 INFO:teuthology.orchestra.run.smithi152.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T11:41:20.883 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T11:41:20.883 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi069.front.sepia.ceph.com 2024-04-25T11:41:20.883 DEBUG:teuthology.orchestra.run.smithi069:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T11:41:20.903 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi146.front.sepia.ceph.com 2024-04-25T11:41:20.904 DEBUG:teuthology.orchestra.run.smithi146:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T11:41:20.947 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi152.front.sepia.ceph.com 2024-04-25T11:41:20.948 DEBUG:teuthology.orchestra.run.smithi152:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T11:41:20.976 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T11:41:20.976 DEBUG:teuthology.orchestra.run.smithi069:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T11:41:20.979 DEBUG:teuthology.orchestra.run.smithi146:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T11:41:20.990 DEBUG:teuthology.orchestra.run.smithi152:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T11:41:21.041 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T11:41:21.042 DEBUG:teuthology.orchestra.run.smithi069:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T11:41:21.046 DEBUG:teuthology.orchestra.run.smithi146:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T11:41:21.055 DEBUG:teuthology.orchestra.run.smithi152:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T11:41:21.428 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T11:41:21.439 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T11:41:21.439 DEBUG:teuthology.orchestra.run.smithi069:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T11:41:21.463 DEBUG:teuthology.orchestra.run.smithi146:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T11:41:21.487 DEBUG:teuthology.orchestra.run.smithi152:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T11:41:21.510 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T11:41:21.521 DEBUG:teuthology.orchestra.run.smithi069:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T11:41:21.523 DEBUG:teuthology.orchestra.run.smithi146:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T11:41:21.529 DEBUG:teuthology.orchestra.run.smithi152:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T11:41:21.542 INFO:teuthology.orchestra.run.smithi069.stdout:kernel.core_pattern = core 2024-04-25T11:41:21.549 INFO:teuthology.orchestra.run.smithi146.stdout:kernel.core_pattern = core 2024-04-25T11:41:21.572 INFO:teuthology.orchestra.run.smithi152.stdout:kernel.core_pattern = core 2024-04-25T11:41:21.613 DEBUG:teuthology.orchestra.run.smithi069:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T11:41:21.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:41:21.628 DEBUG:teuthology.orchestra.run.smithi146:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T11:41:21.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:41:21.641 DEBUG:teuthology.orchestra.run.smithi152:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T11:41:21.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:41:21.667 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T11:41:21.679 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T11:41:21.679 DEBUG:teuthology.misc:Transferring archived files from smithi069:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673128/remote/smithi069 2024-04-25T11:41:21.680 DEBUG:teuthology.orchestra.run.smithi069:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T11:41:21.720 DEBUG:teuthology.misc:Transferring archived files from smithi146:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673128/remote/smithi146 2024-04-25T11:41:21.721 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T11:41:21.755 DEBUG:teuthology.misc:Transferring archived files from smithi152:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673128/remote/smithi152 2024-04-25T11:41:21.755 DEBUG:teuthology.orchestra.run.smithi152:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T11:41:21.790 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T11:41:21.791 DEBUG:teuthology.orchestra.run.smithi069:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T11:41:21.793 DEBUG:teuthology.orchestra.run.smithi146:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T11:41:21.795 DEBUG:teuthology.orchestra.run.smithi152:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T11:41:21.839 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T11:41:21.852 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T11:41:21.852 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T11:41:21.863 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T11:41:21.863 DEBUG:teuthology.orchestra.run.smithi069:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T11:41:21.866 DEBUG:teuthology.orchestra.run.smithi146:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T11:41:21.868 DEBUG:teuthology.orchestra.run.smithi152:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T11:41:21.877 INFO:teuthology.orchestra.run.smithi069.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 11:41 /home/ubuntu/cephtest 2024-04-25T11:41:21.880 INFO:teuthology.orchestra.run.smithi146.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 11:41 /home/ubuntu/cephtest 2024-04-25T11:41:21.893 INFO:teuthology.orchestra.run.smithi152.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 11:41 /home/ubuntu/cephtest 2024-04-25T11:41:21.895 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T11:41:21.906 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T11:41:21.932 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/fs/test_o_trunc}} duration: 1942.9597153663635 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-04-25T11:41:21.932 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T11:41:22.012 INFO:teuthology.run:pass